ajdc 0.0.1 → 0.1.0

data/skills/ajdc/SKILL.md

@@ -0,0 +1,117 @@
+---
+name: ajdc
+description: Design, implement, refactor and test durable workflows on Active Job with AJ/DC (`ActiveJob::Durable`). Use when a feature is a multi-step background process, a "do this later" timer (cron sweep, `set(wait_until:)`, a `*_sent_at` claim column), an approval or webhook wait (a state column plus a controller action that re-enqueues), a job that re-enqueues itself, a status enum that tracks a job's position, or a job that must survive a crash and be resumable. Also use when the user asks whether to use a workflow at all.
+---
+
+# AJ/DC: durable workflows on Active Job
+
+AJ/DC makes an `ActiveJob::Continuable` job durable: the run and its steps are rows in the
+database, so progress survives a crash (not only a graceful stop), a run can wait for a time or
+for a signal without holding a worker or a queue slot, and every run is identified and queryable.
+Same `step` DSL, same job runner, same `perform_later`.
+
+Read `references/api.md` for the full surface. This file is the decision procedure.
+
+## 1. Decide whether a workflow is the right tool
+
+```
+Is it background work with more than one step, or a wait?
+├── no → a plain job. Stop.
+└── yes
+    ├── 1. Does the process wait: for a time, or for someone (approval, webhook, reply)?
+    ├── 2. Does anyone need to find a run by its subject, see where it is, or
+    │      continue it after a failure from where it stopped?
+    ├── both no → ActiveJob::Continuable (Rails 8.1) if a loop must survive a deploy;
+    │             otherwise a plain job. Stop.
+    └── at least one yes → ActiveJob::Durable. Then:
+        ├── it waits (1): does the wait belong to one entity, with steps before or after it?
+        │   ├── yes → a run per entity: wait:, wait_until:, await
+        │   └── no, it is a policy over a table → keep the cron sweep; hybrid for far-future
+        │        dates (references/decision-guide.md §1)
+        └── someone watches (2): name the identity (identified_by), decide whether a second
+            live run is allowed (unique_by), decide what a person fixes (halt_on).
+```
+
+Questions 1 and 2 are the only decisive ones. Uniqueness and per-step error rules are choices
+made once inside, not reasons to enter.
+
+## 2. Recognize the shape in existing code
+
+Each row answers question 1 or 2 of the tree.
+
+| You see | It is | Durable shape |
+|---|---|---|
+| Job A enqueues job B enqueues job C, each flips a status | pipeline | one job, `step :a; step :b; step :c` |
+| a state machine gem on a record plus a job that re-enqueues itself | pipeline | steps, `isolated: true` where each step must be its own execution |
+| `set(wait_until:)` plus an `updated_at` argument compared at wake | timer with hand-made invalidation | `step :x, wait_until:` plus `unique_by ..., on_conflict: :replace` |
+| a cron entry that exists only to check a date column | timer as sweep | `step :x, wait_until: record.date` per entity, or keep the sweep (see guide) |
+| `*_sent_at` / `*_notified_at` claim columns per reminder | timers as columns | one step per reminder, `wait_until:` |
+| a controller action that flips a state and calls `perform_later` | signal | `await :name`, `run.wake_up(:name, value)` from the action |
+| a webhook handler that finds a record by token and continues the flow | signal | `await :name, wait:` with a deadline, `unique_by` on the record |
+| a class that serializes "the last run" into a settings row | run record | delete it; `MyJob.workflow_runs.first` |
+
+Secondary signals. They confirm that a workflow fits and name a feature to use once inside;
+they do not start one:
+
+- a dedupe guard (`Model.exists?(external_id:)`) next to a per-id `limits_concurrency` key:
+  `unique_by`, usually `:skip`;
+- a `rescue` inside a step body that flips a status column and returns, so a person can retry
+  later: `halt_on` for that error class;
+- a `retry_on`/`discard_on` list that grows with every new error class: the three verbs,
+  `retry_on`, `discard_on`, `halt_on`, and the rest stays as it is.
+
+## 3. Implement
+
+If the gem is not installed yet, follow `references/installation.md` first: it decides between
+the primary and a separate database (the transaction property depends on it) and adds the
+recurring wake job, without which no timer or deadline fires.
+
+1. Decide where each piece of state lives. Facts about the record (`paid`, `signed`) stay on
+   the model. The process position (`analyzing`, `needs_review`, `generating`) moves to the run.
+   Never mirror one into the other.
+2. `include ActiveJob::Durable` instead of `ActiveJob::Continuable`. Keep `step`, cursors and
+   `attribute` as they are. Run the existing tests; nothing should change.
+3. Name the identity. Default key = all arguments. Declare `identified_by` when other arguments
+   must not split the key, `unique_by` when only one live run per identity may exist. Pick
+   `on_conflict`: `:skip` for sweeps and double clicks, `:reject` for money, `:replace` for
+   reschedules and renewals. See `references/api.md` §2.
+4. Move waits into steps: `wait:` for "N after the previous step", `wait_until:` for "at this
+   time"; both take a value or a callable. Check that `ActiveJob::Durable::WakeJob` is scheduled
+   (`references/installation.md` §3).
+5. Move signals into `await :name, wait: deadline` plus a method `name(value)`; deliver with
+   `run.wake_up(:name, value)` from the controller, webhook or console. `nil` means the deadline
+   passed. Add a model method that finds the run (`payout.job_run`) instead of a class-level
+   helper.
+6. Classify errors: `retry_on` (the machine tries again), `discard_on` (give up), `halt_on` (a
+   person fixes it, then `run.resume!`). `halt!(reason)` from inside a step is the same stop
+   without an exception.
+7. Replace `after_transition`/broadcast calls inside steps with `after_step` /
+   `before_step` / `around_step`; read `current_step.name` inside the callback.
+8. Replace hand-written run records, status enums that track position, and sweeps that only
+   check a date. Keep the model facts.
+9. Check the transaction story in `references/transactions.md` for every `perform_later`,
+   `wake_up`, `cancel!` you add inside a `transaction`/`with_lock` block.
+10. Write the tests per `references/testing.md` before the refactor, against the old behaviour,
+   then switch.
+
+## 4. Do not
+
+- Do not add a `status` enum to a model to mirror the run. Query `workflow_runs`.
+- Do not `await` inside a loop with a fixed name; step names are unique per run, so the second
+  pass would skip it. For a repeated decision that needs no value (a tool approval already
+  recorded on a model), `halt!` in the loop and `resume!` from the outside. For a known set of
+  valued decisions, one `await` per name: `await :"decision_#{approver.id}"`. For an unknown
+  number of valued decisions, store the value on a model and use `halt!`; see
+  `examples/agent-loop.md` and `references/decision-guide.md` §3.
+- Do not `sleep` inside a step; use `wait:` on the next step.
+- Do not call `wake_up`, `resume!` or `cancel!` on a run loaded before a long operation; reload
+  or re-find it, the guarded update fails on a stale status.
+- Do not put `wait:` and `wait_until:` on the same step.
+- Do not replace a sweep that is a policy over millions of rows with millions of parked runs
+  without reading `references/decision-guide.md` §2.
+
+## 5. Report
+
+When you finish, list: the jobs converted, the columns/classes/cron entries removed, the
+`unique_by` strategy chosen and why, the recurring `WakeJob` entry added, and the queries the
+UI or ops now use (`workflow_runs.awaiting`, `.halted`, `.stuck_for(...)`).

data/skills/ajdc/examples/README.md

@@ -0,0 +1,11 @@
+# Examples: before and after
+
+Each file shows a real shape found in Rails apps and its `ActiveJob::Durable` version. Names
+are generic. Read the one that matches the shape you found (see SKILL.md §2).
+
+- `pipeline.md`: a chain of jobs with a status enum, or a state machine with a self-re-enqueuing job
+- `timers.md`: cron sweeps and `set(wait_until:)` with invalidation
+- `signals.md`: a state column plus a controller action; a webhook that continues a flow
+- `halting.md`: an error a person fixes; rescue-inside-the-step shapes
+- `uniqueness.md`: one live run per identity; `:skip`, `:replace`, `:reject`; identity inside an argument
+- `agent-loop.md`: an LLM loop with a tool approval

data/skills/ajdc/examples/agent-loop.md

@@ -0,0 +1,57 @@
+# Agent loop with a tool approval
+
+## Before (ruby_llm durable-agents guide): one loop step, a finished job on approval
+
+```ruby
+class AgentRunJob < ApplicationJob
+  include ActiveJob::Continuable
+
+  def perform(chat_id)
+    step :agent_loop do |job_step|
+      chat = Chat.find(chat_id)
+      until chat.complete?
+        chat.step
+        job_step.checkpoint!
+      end
+    end
+  end
+end
+
+class ApprovalsController < ApplicationController
+  def create
+    chat = Chat.find(params[:chat_id])
+    params[:approved] == "true" ? chat.approve(params[:tool_call_id]) : chat.deny(params[:tool_call_id])
+    CompleteJob.perform_later(chat.id)
+  end
+end
+```
+
+## After
+
+```ruby
+class AgentRunJob < ApplicationJob
+  include ActiveJob::Durable
+
+  def perform(chat)
+    step :run do |step|
+      until chat.complete?
+        chat.step
+        halt!(:tool_approval) if chat.awaiting_approval?
+        step.checkpoint!
+      end
+    end
+  end
+end
+
+class ApprovalsController < ApplicationController
+  def create
+    chat = Chat.find(params[:chat_id])
+    params[:approved] == "true" ? chat.approve(params[:tool_call_id]) : chat.deny(params[:tool_call_id])
+    chat.agent_run.resume!
+  end
+end
+```
+
+`halt!`, not `await`: the pending tool call is already a row in ruby_llm's tables, so the run
+needs a wake, not a payload. `resume!` re-enters the loop step; `chat.step` runs the approved
+tool and skips the ones with results. Halted runs: `AgentRunJob.workflow_runs.halted`.

data/skills/ajdc/examples/halting.md

@@ -0,0 +1,84 @@
+# Halting: an error a person fixes, then the run continues
+
+## Before: rescue inside the step, a status column, a restart from scratch
+
+The shape from Spree's CSV import: `discard_on` would never see an error raised after progress,
+so the step rescues, marks the import failed, and stops. A retry starts the job over.
+
+```ruby
+class Imports::ProcessJob < ApplicationJob
+  include ActiveJob::Continuable
+
+  def perform(import)
+    @import = import
+    step :create_rows, start: 1
+    return if @csv_failed
+    step :dispatch_row_groups
+  end
+
+  private
+    def create_rows(step)
+      CSV.foreach(@import.file, headers: true).with_index(1) do |row, number|
+        next if number < step.cursor
+        ImportRow.create!(import: @import, number:, data: row.to_h)
+        step.set!(number)
+      end
+    rescue StorageQuotaExceeded, CSV::MalformedCSVError => e
+      @import.update_columns(status: :failed, processing_errors: e.message)
+      @csv_failed = true
+    end
+end
+
+# admin "Retry": import.update!(status: :pending); Imports::ProcessJob.perform_later(import)   # from row 1
+```
+
+## After: `halt_on`, `resume!` from the cursor
+
+```ruby
+class Imports::ProcessJob < ApplicationJob
+  include ActiveJob::Durable
+
+  discard_on CSV::MalformedCSVError          # nothing can fix the file
+  halt_on StorageQuotaExceeded               # a person upgrades the plan, then the run continues
+
+  def perform(import)
+    @import = import
+    step :create_rows, start: 1
+    step :dispatch_row_groups
+  end
+
+  private
+    def create_rows(step)
+      CSV.foreach(@import.file, headers: true).with_index(1) do |row, number|
+        next if number < step.cursor
+        ImportRow.create!(import: @import, number:, data: row.to_h)
+        step.set!(number)
+      end
+    end
+end
+
+# after the plan upgrade:
+run = Imports::ProcessJob.workflow_runs.for(import).halted.sole
+run.error_class         # => "StorageQuotaExceeded"
+run.current_step        # => "create_rows"
+run.resume!             # continues from the last committed row number
+```
+
+Three verbs for three kinds of failure: `retry_on` is the machine trying again, `discard_on` is
+giving up, `halt_on` is a person deciding. The rescue, the flag and the status column are gone;
+the import's own facts stay on the import. A halted run is in `workflow_runs.attention` with its
+error until someone resumes or cancels it.
+
+## `halt!`: the same stop without an exception
+
+```ruby
+def check(name)
+  result = public_send(:"check_#{name}")
+  self.error_msg = result.reason
+  halt!(result.level) unless result.level == :success      # halt_reason = "warning" / "critical"
+  metadata[name] = result.data
+end
+```
+
+`resume!` re-runs the halted step. Use `halt!` when the stop is a decision, not an error; use
+`await` instead when the decision carries a value the run needs (see `signals.md`).

data/skills/ajdc/examples/pipeline.md

@@ -0,0 +1,73 @@
+# Pipeline
+
+## Before: a chain of jobs and a status enum
+
+```ruby
+class AnalyzeCardJob < ApplicationJob
+  def perform(card)
+    card.analyzing!
+    if NSFWDetector.check(card.image) then card.analyzed!; GenerateCardJob.perform_later(card)
+    else card.fail!("NSFW check failed") end
+  end
+end
+
+class GenerateCardJob < ApplicationJob
+  def perform(card)
+    card.generating!
+    card.attach_rendition(CardPainter.paint(card))
+    card.generated!
+  end
+end
+```
+
+## Before: a state machine and a job that re-enqueues itself
+
+```ruby
+class Cable::DiagnosticJob < ApplicationJob
+  def perform(diagnostic)
+    level = diagnostic.check!                       # runs the check for the current state
+    return diagnostic.update!(state: :halted, halted_reason: level) unless level == :success
+    diagnostic.next!
+    self.class.perform_later(diagnostic) unless diagnostic.completed?
+  end
+end
+```
+
+## After
+
+```ruby
+class Cable::DiagnosticJob < ApplicationJob
+  include ActiveJob::Durable
+
+  attribute :metadata, default: {}
+  attribute :error_msg, :string
+
+  after_step :broadcast_update
+
+  def perform(cable)
+    @cable = cable
+    step :provider_status,  isolated: true
+    step :websocket_status, isolated: true
+    step :admin_api_status, isolated: true
+  end
+
+  private
+    def provider_status  = check(:provider_status)
+    def websocket_status = check(:websocket_status)
+    def admin_api_status = check(:admin_api_status)
+
+    def check(name)
+      result = public_send(:"check_#{name}")
+      self.error_msg = result.reason
+      halt!(result.level) unless result.level == :success
+      metadata[name] = result.data
+    end
+
+    def broadcast_update = @cable.broadcast_replace(partial: "cables/diagnostic", locals: { step: current_step.name })
+end
+```
+
+What moved: the transitions table is the three `step` lines; "one check per job run" is
+`isolated: true`; the broadcast is one `after_step`; the halt has a reason and is resumable; the
+UI reads the run (`status`, `current_step`, `state`) instead of a cached record. The card's own
+facts (`generated`, `failed`) stay on the card.

data/skills/ajdc/examples/signals.md

@@ -0,0 +1,139 @@
+# Signals
+
+## Before: a state column, a controller action, a worker with no memory, a vacuum
+
+```ruby
+class BulkImport < ApplicationRecord
+  CONFIRM_PERIOD = 10.minutes
+  enum :state, { unconfirmed: 0, scheduled: 1, in_progress: 2, finished: 3 }, prefix: true
+  scope :confirmation_missed, -> { state_unconfirmed.where(created_at: ..CONFIRM_PERIOD.ago) }
+end
+
+class Settings::ImportsController
+  def confirm
+    @bulk_import.update!(state: :scheduled)
+    BulkImportWorker.perform_async(@bulk_import.id)
+    redirect_to settings_imports_path
+  end
+end
+
+class Vacuum::ImportsVacuum                                     # daily
+  def perform = BulkImport.confirmation_missed.in_batches.delete_all
+end
+```
+
+## After
+
+```ruby
+class BulkImportJob < ApplicationJob
+  include ActiveJob::Durable
+
+  attribute :confirmed, :boolean, default: false
+
+  def perform(import)
+    await :confirmation, wait: 10.minutes
+    return import.destroy! unless confirmed
+    step :apply do
+      BulkImportService.new.call(import)
+    end
+  end
+
+  def confirmation(signal) = self.confirmed = signal.presence
+end
+
+# Form::Import#save, after the rows are inserted:  BulkImportJob.perform_later(bulk_import)
+# Settings::ImportsController#confirm:             @bulk_import.job_run.wake_up(:confirmation, true)
+# BulkImport#job_run:                              BulkImportJob.workflow_runs.for(self).live.sole
+```
+
+The vacuum is gone because the deadline fires on the clock's next tick. Ops:
+`BulkImportJob.workflow_runs.awaiting.count`, `.at_step(:apply).stuck_for(1.hour)`, `.attention`.
+
+## Before: a payout with two entry points tied by a token
+
+```ruby
+class WithdrawFundsJob < ApplicationJob
+  def perform(payout, wallet_id, amount) = Wallet::Withdraw.(wallet_id:, amount:, payout:)
+end
+
+class PaymentWebhooksController
+  def create
+    ProcessPaymentJob.perform_later(*params.slice(:token, :status, :amount))
+    head :ok
+  end
+end
+
+class ProcessPaymentJob < ApplicationJob
+  def perform(token, status, amount)
+    payout = Payout.find_by!(token:)
+    Ledger.credit!(payout.account, amount)
+    payout.completed!
+  end
+end
+```
+
+## After
+
+```ruby
+class PayoutJob < ApplicationJob
+  include ActiveJob::Durable
+
+  unique_by :payout, on_conflict: :reject
+  attribute :payment
+
+  def perform(payout, wallet_id, amount)
+    @payout = payout
+    step :withdraw do
+      Wallet::Withdraw.(wallet_id:, amount:, payout:)
+    end
+    await :payment, wait: 3.days
+    step :credit
+  end
+
+  def payment(signal)
+    halt!(:payment_overdue) if signal.nil?
+    halt!(:payment_failed)  if signal["status"] == "failed"
+    self.payment = signal
+  end
+
+  def credit
+    Ledger.credit!(@payout.account, payment["amount"])
+    @payout.completed!
+  end
+end
+
+class PaymentWebhooksController
+  def create
+    payout = Payout.find_by!(token: params[:token])
+    payout.job_run.wake_up(:payment, params.slice(:status, :amount).to_h)
+    head :ok
+  end
+end
+```
+
+A halted run sits in `PayoutJob.workflow_runs.attention` with a reason; nothing is silently
+stuck. `unique_by :payout` is what lets the webhook find the run with the payout alone.
+
+## Human review with a deadline
+
+```ruby
+class CardGenerationJob < ApplicationJob
+  include ActiveJob::Durable
+
+  attribute :verdict, :string
+
+  def perform(card)
+    @card = card
+    step :prepare,  isolated: true
+    step :moderate, isolated: true
+    await :review, wait: 1.hour if verdict == "unsure"
+    step :generate, isolated: true unless verdict == "rejected"
+  end
+
+  def moderate       = self.verdict = Moderator.verdict(@card)
+  def review(value)  = self.verdict = value || "rejected"          # nil at the deadline
+end
+
+# Admin action "Approve": card.generation_run.wake_up(:review, "approved")
+# Review queue:           CardGenerationJob.workflow_runs.awaiting.at_step(:review)
+```

data/skills/ajdc/examples/timers.md

@@ -0,0 +1,136 @@
+# Timers
+
+## Before: three cron entries and three sweeps
+
+```yaml
+license_expiration_reminder: { class: License::ReminderJob, args: [60], schedule: "26 * * * *" }
+license_expiration:          { class: License::ExpirationJob,   schedule: "13 */2 * * *" }
+license_revoke:              { class: License::RevokeAccessJob, schedule: "51 */12 * * *" }
+```
+
+```ruby
+class License::ReminderJob < ApplicationJob
+  def perform(interval, now = Time.current)
+    future = now + 2.weeks
+    shift = future.to_i % interval                                   # bucket so each license gets one reminder
+    License.where(expires_at: (future - shift)...(future + interval - shift), status: [:trial, :active])
+           .find_each { LicenseDelivery.with(license: it).license_expiring_two_weeks.deliver_later }
+  end
+end
+```
+
+## After: one run per license
+
+```ruby
+class License::LifecycleJob < ApplicationJob
+  include ActiveJob::Durable
+
+  unique_by :license, on_conflict: :replace
+
+  def perform(license)
+    @license = license
+    step :remind, wait_until: license.expires_at - 2.weeks
+    step :expire, wait_until: license.expires_at
+    step :revoke, wait: 2.weeks
+  end
+
+  def remind = LicenseDelivery.with(license: @license).license_expiring_two_weeks.deliver_later
+  def expire = @license.expired!.then { LicenseDelivery.with(license: @license).license_expired.deliver_later }
+  def revoke = @license.revoke_later.then { LicenseDelivery.with(license: @license).access_revoked.deliver_later }
+end
+
+# License: after_create_commit, and after renewal:  License::LifecycleJob.perform_later(self)
+```
+
+Renewal restarts the lifecycle with `:replace`, because `remind` is already completed and the
+new term needs its own reminder. A moved `expires_at` while waiting re-arms by itself.
+
+## Before: a parked job with hand-made invalidation, then a retreat to a sweep
+
+```ruby
+class AppointmentNotificationJob < ApplicationJob
+  def self.schedule(appointment)
+    set(wait_until: appointment.start_at - 30.minutes).perform_later(appointment.id, appointment.updated_at)
+  end
+
+  def perform(appointment_id, updated_at)
+    appointment = PatientAppointment.lock.find(appointment_id)
+    return unless appointment.updated_at.to_i == updated_at.to_i     # rescheduled meanwhile? drop
+    return if appointment.notification_sent_at                       # claim column
+    appointment.update!(notification_sent_at: Time.zone.now)
+    AppointmentNotification.with(appointment:).deliver_later(appointment.patient)
+  end
+end
+```
+
+## After
+
+```ruby
+class AppointmentReminderJob < ApplicationJob
+  include ActiveJob::Durable
+
+  unique_by :appointment, on_conflict: :replace
+
+  def perform(appointment)
+    step :remind, wait_until: appointment.start_at - 30.minutes do
+      next if appointment.hide_from_patient?
+      AppointmentNotification.with(appointment:).deliver_later(appointment.patient)
+    end
+  end
+end
+
+# PatientAppointment: after_create_commit, and after_update_commit when start_at changed:
+#   AppointmentReminderJob.perform_later(self)
+```
+
+No claim column, no `updated_at` argument, no sweep. A parked run has an identity.
+
+## Before: a grace period as a scope and a sweep
+
+```ruby
+class Account::IncinerateDueJob < ApplicationJob
+  include ActiveJob::Continuable
+  def perform
+    step :incineration do |step|
+      Account.due_for_incineration.find_each { it.incinerate; step.checkpoint! }   # cancellation older than 30 days
+    end
+  end
+end
+```
+
+## After: the model starts and cancels the run, inside its lock
+
+```ruby
+module Account::Cancellable
+  def cancel
+    with_lock do
+      next unless cancellable? && active?
+      create_cancellation!(initiated_by: Current.user)
+      Account::IncinerationJob.perform_later(self)                     # row now, message after commit
+    end
+  end
+
+  def reactivate
+    with_lock do
+      next unless cancelled?
+      cancellation.destroy
+      Account::IncinerationJob.workflow_runs.for(self).live.sole.cancel!
+    end
+  end
+end
+
+class Account::IncinerationJob < ApplicationJob
+  include ActiveJob::Durable
+
+  unique_by :account
+
+  def perform(account)
+    step :incinerate, wait: 30.days do
+      account.incinerate
+    end
+  end
+end
+```
+
+The sweep was a fine design; this version buys one row per pending deletion that anyone can
+query and a reactivation that cannot race the sweep.