active_durable 0.7.0 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: ca565450baafb042b5ec363d2d7c101f24edd3281b180d43c173eaeff8dd8df8
4
- data.tar.gz: 96ceecb07cf0d85b0d8aaf604dececee56301983d16055de408e533c53059da6
3
+ metadata.gz: 47ef8eb3bb5a133221ed52ce914ec952ff7f73f552b5352d4558b96e6738458c
4
+ data.tar.gz: af132b5a177125726a6aed8699e7cf4391c70e9e4fbac566531ab87c3204f998
5
5
  SHA512:
6
- metadata.gz: a6dafa621dba36aac0429fc264922d876d378390aa7fccebf05a085bbcb1f2e001d7bf57d202075299e47cbbb20d29af86df62000adca22367bec76ab18dd16b
7
- data.tar.gz: 7b4c90ddb5104811ab17be9532fe2c4ca029bae139eeeaafdef1bd04a222fa4c2896c34b2ce44376d3ff13222473e76276a991c125fcdbe057937d7e6a649896
6
+ metadata.gz: fe0584f97ce0594ee30c2186b8a10ac91e57505172b758cf8e667b290a6cb387e0da3566db685c8070798bdab84f32f225a7f2f1787aabc9b106bbe069ceb11e
7
+ data.tar.gz: 863396b2acc54f007f65c80379e3bcbac0512eb05888d3a91372cddfd88333e5b3536d25798949eaf2200a85fd021ae540a65ffd4d1d1078716e0221247a609b
data/CHANGELOG.md CHANGED
@@ -5,6 +5,43 @@ All notable changes to this project are documented here. The format follows
5
5
  [Semantic Versioning](https://semver.org/). Schema changes ship as new migrations: after updating the gem, run
6
6
  `bin/rails generate active_durable:upgrade` and `bin/rails db:migrate`.
7
7
 
8
+ ## [0.8.0] - 2026-10-22
9
+
10
+ For sagas that live long and talk to people, such as a WhatsApp agent that books appointments: a mailbox for
11
+ messages, a fresh notebook for long conversations, and the tenant in every run.
12
+
13
+ Upgrading from 0.7: `bin/rails generate active_durable:upgrade && bin/rails db:migrate`. It adds three nullable
14
+ columns (`durable_executions.context`, `durable_executions.continued_as` with its index, `durable_steps.signal_name`);
15
+ nothing is rewritten. Note the first change below: hash keys now come back sorted.
16
+
17
+ ### Added
18
+
19
+ - `flow.continue_as_new(**input)` closes the execution and starts a new one of the same recipe (its latest version)
20
+ with that input, the same context and the signals nobody read yet, in one transaction. Ids follow the chain
21
+ `order-1`, `order-1~2`, `order-1~3`. The new `continued` status; `Durable.find(id, latest: true)`.
22
+ `Durable.signal` to any id of the chain reaches the newest execution, also during the hand-off.
23
+ - `flow.wait_for(name, signal:, all:)`: several waits share one signal name, like a mailbox, and `all: true` takes
24
+ every pending signal at once, returning their payloads in order.
25
+ - `config.context`: an object with `capture` (when an execution starts) and `wrap(context) { }` (around every run,
26
+ undo, hook and `flow.parallel` branch). `Durable.start(..., context: {...})` gives it explicitly. Reruns keep it.
27
+ - `ActiveDurable::Testing.drain(id, expire_waits: false)` stops at a wait instead of fast-forwarding to its
28
+ timeout, to test a conversation turn by turn. `drain` follows `continue_as_new`.
29
+ - The dashboard shows `continued` executions and links to the one that carries on.
30
+
31
+ ### Changed
32
+
33
+ - Hash keys in step results, inputs, signal payloads and `flow.parallel` results are sorted, on the first run and
34
+ when read back. PostgreSQL's jsonb and MySQL's JSON store keys shorter-first, so a replay used to read a hash whose
35
+ keys were in another order than the first run returned. That broke anything that compares bytes, such as an AI
36
+ provider that checks the conversation it is sent again. Executions in flight during the upgrade see their keys in
37
+ a new order once.
38
+ - `Durable.signal` locks the execution row while it writes the signal.
39
+ - `Testing.drain` delivers each signal given in `signals:` once per call, instead of every time a step of that name
40
+ waits.
41
+ - `Durable.start` takes `context:`, so `context` can no longer be an input key.
42
+ - `ActiveDurable.retry`, `compensate` and `rerun` on a continued execution raise, naming the one that carries on.
43
+ - `ActiveDurable.prune` deletes a continued execution only together with the one that carried it on.
44
+
8
45
  ## [0.7.0] - 2026-10-07
9
46
 
10
47
  Fixes from an architecture and security review. Most of them stop a saga from paying twice or undoing the wrong
data/README.md CHANGED
@@ -213,6 +213,7 @@ ActiveDurable.configure do |config|
213
213
  config.sweep_grace = 1.minute # the sweeper leaves executions this young alone
214
214
  config.keep_finished_for = 30.days # then ActiveDurable::PruneJob deletes finished ones
215
215
  config.code_errors += ["Payments::Misconfigured"] # your own errors that mean a bug (see "A bug is not a failure")
216
+ config.context = TenantContext # who the work is for, in every run (see "Context")
216
217
 
217
218
  # Who may open the dashboard outside development and test. With Devise (HTTP basic auth: see Dashboard):
218
219
  config.dashboard_authorize = ->(controller) { controller.request.env["warden"]&.user&.admin? }
@@ -413,7 +414,8 @@ Three rules follow from replaying the recipe:
413
414
  | `flow.pivot(name) { \|ticket\| ... }` | the step after which there is no going back | retried, then the saga is undone |
414
415
  | `flow.parallel(name) { \|branches\| ... }` | several steps at the same time | each branch like a step |
415
416
  | `flow.sleep(name, 3.days)` | waiting without holding a worker | — |
416
- | `flow.wait_for(name, timeout:)` | waiting for `Durable.signal` | a timeout fails the saga |
417
+ | `flow.wait_for(name, timeout:)` | waiting for `Durable.signal` (`signal:` and `all:` for a mailbox) | a timeout fails the saga |
418
+ | `flow.continue_as_new(**input)` | a long-lived saga, such as a chat, that starts a fresh notebook | refused inside a step or with undos pending |
417
419
  | `flow.on(:completed) { ... }` | updating your own records when the saga ends (also `:compensated`) | blocks; a retry runs the hook again |
418
420
 
419
421
  Steps take `undo:`, `retry:` (`3`, `false` or `{ attempts:, backoff: }`) and `undo_on_failure:`.
@@ -537,6 +539,18 @@ Durable.signal("loan-42", :kyc_done, verified: true)
537
539
 
538
540
  Pass `id:` to `Durable.start` to choose the execution id (the call becomes idempotent).
539
541
 
542
+ **A mailbox.** By default the signal has the step's name. A conversation waits for the customer on every turn, so
543
+ each turn gets its own step name and all of them share one signal name with `signal:`. A message that arrives while
544
+ the saga is busy waits in the mailbox for the next turn. With `all: true` the wait takes every pending message at
545
+ once and returns them in order, so five messages in a row are one turn, not five:
546
+
547
+ ```ruby
548
+ messages = flow.wait_for("reply.#{turn}", signal: :message, all: true, timeout: 1.day)
549
+
550
+ # in the WhatsApp webhook
551
+ Durable.signal(conversation.execution_id, :message, text: params[:text])
552
+ ```
553
+
540
554
  </details>
541
555
 
542
556
  <details>
@@ -562,6 +576,57 @@ the finished branches are undone in the order they finished. Give your connectio
562
576
 
563
577
  </details>
564
578
 
579
+ <details>
580
+ <summary><b>Context: the tenant in every run</b></summary>
581
+
582
+ <br>
583
+
584
+ A saga runs in a worker, days after the request that started it, so `Current.tenant` and friends are gone.
585
+ `config.context` carries them: `capture` runs when the execution starts and its JSON is stored with it; `wrap` runs
586
+ around every run of it, including undos, hooks and each `flow.parallel` branch thread.
587
+
588
+ ```ruby
589
+ module TenantContext
590
+ def self.capture = { "tenant_id" => Current.tenant_id }
591
+ def self.wrap(context, &run) = Current.set(tenant_id: context["tenant_id"], &run)
592
+ end
593
+
594
+ ActiveDurable.configure { |config| config.context = TenantContext }
595
+
596
+ Durable.start(:checkout, order_id: order.id) # captures the current tenant
597
+ Durable.start(:welcome, tenant_id: t.id, context: { "tenant_id" => t.id }) # or gives it explicitly
598
+ ```
599
+
600
+ `wrap` must call the block once. Executions started without a context get `{}`.
601
+
602
+ </details>
603
+
604
+ <details>
605
+ <summary><b>Long conversations: continue as new</b></summary>
606
+
607
+ <br>
608
+
609
+ Every run replays the recipe from the top, reading the notebook. A chat that has gone on for 200 turns reads 1,000
610
+ entries before it can answer. `flow.continue_as_new` closes the execution and starts a fresh one of the same recipe
611
+ (its latest version) with the input you give it, usually a summary, plus the same context and the messages nobody
612
+ read yet. All of it happens in one transaction:
613
+
614
+ ```ruby
615
+ Durable.define(:chat) do |flow, conversation_id:, turn: 0, summary: nil|
616
+ # ... the turn ...
617
+ flow.continue_as_new(conversation_id: conversation_id, turn: turn + 1, summary: summary) if turn % 20 == 19
618
+ end
619
+ ```
620
+
621
+ The new execution's id is the old one plus `~2` (then `~3`), so its steps get new tickets. `Durable.signal` to any id
622
+ of the chain reaches the newest execution, even one sent during the hand-off, and
623
+ `Durable.find(id, latest: true)` returns it. Compute the input from step results, so a replay computes the same.
624
+ `continue_as_new` is refused inside a step and while undos are pending (those steps could never be undone); after
625
+ `flow.pivot` it is allowed. The old execution's hooks do not run, and it is pruned together with the one that
626
+ carries on.
627
+
628
+ </details>
629
+
565
630
  <details>
566
631
  <summary><b>Changing a recipe while sagas are in flight</b></summary>
567
632
 
@@ -581,6 +646,91 @@ New executions use the highest version; each execution keeps the one it started
581
646
 
582
647
  </details>
583
648
 
649
+ ## Example: a WhatsApp assistant that books appointments
650
+
651
+ First in plain words, then the code.
652
+
653
+ **What it does.** A customer writes to a bakery on WhatsApp: "Hi, can I come on Friday to order a cake?". An
654
+ assistant (an AI that can see the calendar) answers, offers the free times and books the one the customer picks. The
655
+ conversation can take two minutes or two days. The customer may send three messages in a row, or disappear and come
656
+ back tomorrow. Meanwhile your servers restart, you deploy, a process dies.
657
+
658
+ **The notebook.** Think of a cook who ticks every step of a recipe in a notebook. When someone interrupts, the cook
659
+ does not trust their memory: they reread the notebook from the top, skip every step with a tick, and carry on from
660
+ the first one without it. ActiveDurable does the same with your code. Each step's result is written in your database,
661
+ and every time the saga wakes up (a message arrives, a timer rings) it rereads the recipe and skips what is done. That
662
+ is why the assistant never books twice, even if the server dies right after booking: the booking has its tick. (A
663
+ step that died halfway runs again with the same ticket, so the calendar and WhatsApp can recognize a repeat.)
664
+
665
+ **Turns.** A turn is one exchange: the customer writes, the assistant answers. Each turn adds three ticks: the
666
+ customer's messages, the AI's answer (which may book) and the WhatsApp message sent back. The AI's answer is written
667
+ down too, so rereading never asks the AI again: no double cost, and the very same text.
668
+
669
+ **The mailbox.** Every message goes to one mailbox, `:message`. If the customer writes while the assistant is
670
+ thinking, the message waits there for the next turn. If they send five in a row, the assistant takes all five at
671
+ once: one turn, not five.
672
+
673
+ **A fresh page.** After 20 turns the notebook has 60 ticks to reread every time something happens. Instead of making
674
+ the same page longer, `continue_as_new` starts a new page with a short summary ("Ana wants a cake on Friday, saw the
675
+ times at 10 and 3"), keeping the messages nobody read yet.
676
+
677
+ **Who it is for.** One app serves many bakeries. `config.context` writes down which one the conversation belongs to
678
+ when it starts, and gives it back on every run, so each step sees the right calendar.
679
+
680
+ ```ruby
681
+ # config/initializers/active_durable.rb
682
+ module TenantContext
683
+ def self.capture = { "tenant_id" => Current.tenant_id }
684
+ def self.wrap(context, &run) = Current.set(tenant_id: context["tenant_id"], &run)
685
+ end
686
+ ActiveDurable.configure { |config| config.context = TenantContext }
687
+
688
+ # app/sagas/booking_assistant_saga.rb
689
+ BookingAssistantSaga = Durable.define(:booking_assistant) do |flow, conversation_id:, turn: 0, summary: nil|
690
+ conversation = Conversation.find(conversation_id)
691
+ history = summary ? [{ "role" => "user", "content" => "What happened so far: #{summary}" }] : []
692
+ booked = nil
693
+
694
+ 20.times do |i|
695
+ n = turn + i
696
+ # 1. The customer's messages: every one that arrived since the last turn.
697
+ messages = flow.wait_for("customer.#{n}", signal: :message, all: true)
698
+ history << { "role" => "user", "content" => messages.map { |m| m["text"] }.join("\n") }
699
+
700
+ # 2. The AI's answer. When it decides to book, it calls Calendar.book(slot, key: ticket): booked once.
701
+ reply = flow.step("assistant.#{n}") { |ticket| BookingAssistant.answer(history, ticket: ticket) }
702
+ history << { "role" => "assistant", "content" => reply["text"] }
703
+
704
+ # 3. The answer. If the process dies right after sending it, this step runs again with the same ticket:
705
+ # WhatsApp.send_text keeps the ticket with the message it sent and skips sending it twice.
706
+ flow.step("send.#{n}") { |ticket| WhatsApp.send_text(conversation.phone, reply["text"], key: ticket) }
707
+
708
+ if reply["booked"]
709
+ booked = reply
710
+ break
711
+ end
712
+ end
713
+ next booked if booked
714
+
715
+ # 20 turns without a booking: a fresh page with a summary.
716
+ summary = flow.step("summary.#{turn}") { BookingAssistant.summarize(history) }
717
+ flow.continue_as_new(conversation_id: conversation_id, turn: turn + 20, summary: summary["text"])
718
+ end
719
+
720
+ # app/controllers/whatsapp_webhooks_controller.rb
721
+ def create
722
+ conversation = Conversation.open_for(params[:from]) # a new one once the last one ended; tenant: Current.tenant_id
723
+ id = "chat-#{conversation.id}"
724
+ Durable.start(:booking_assistant, id: id, conversation_id: conversation.id) # idempotent: starts it once
725
+ Durable.signal(id, :message, text: params[:text]) # reaches the newest page of the chain
726
+ head :ok
727
+ end
728
+ ```
729
+
730
+ To close conversations where the customer went quiet, give the wait a `timeout: 1.day` and rescue
731
+ `ActiveDurable::StepFailed`. To test it turn by turn: `Testing.drain(id, expire_waits: false)`, then `Durable.signal`,
732
+ then drain again.
733
+
584
734
  ## Dashboard
585
735
 
586
736
  ```ruby
@@ -654,7 +804,8 @@ point, kills it right there, and finishes it with a new worker. It is the fastes
654
804
  idempotent.
655
805
 
656
806
  `ActiveDurable::Testing.drain(id, signals: { name => payload })` runs an execution synchronously, fast-forwarding
657
- sleeps, retries and expired leases.
807
+ sleeps, retries and expired leases, and follows `continue_as_new`. It fast-forwards wait timeouts too; to test a
808
+ conversation turn by turn, pass `expire_waits: false`, send a signal and drain again.
658
809
 
659
810
  ## Observability
660
811
 
@@ -760,6 +911,9 @@ while sagas were halfway through. All of them settled, the refunds and hooks ran
760
911
  OpenTelemetry: keep secrets and personal data out of them.
761
912
  - Ids cannot be blank or contain `/`. Ids and step names compare exactly, also on MySQL.
762
913
  - `flow.transaction` is atomic only when the notebook lives in the same database as your data.
914
+ - Step results, inputs and signals come back with their hash keys sorted, on the first run and on every replay, since
915
+ PostgreSQL and MySQL do not keep the order you wrote them in.
916
+ - A replay reads the whole notebook. For sagas that grow without end, such as a chat, use `flow.continue_as_new`.
763
917
 
764
918
  ## Development
765
919
 
@@ -8,14 +8,14 @@ module ActiveDurable
8
8
 
9
9
  STATUS_TONES = {
10
10
  "completed" => "good", "compensated" => "undo", "blocked" => "bad", "superseded" => "muted",
11
- "running" => "busy", "pending" => "busy", "sleeping" => "wait", "waiting" => "wait",
11
+ "continued" => "muted", "running" => "busy", "pending" => "busy", "sleeping" => "wait", "waiting" => "wait",
12
12
  "retrying" => "wait", "failed" => "bad", "undone" => "undo"
13
13
  }.freeze
14
14
 
15
15
  STATUS_WORDS = {
16
16
  "pending" => "Queued", "running" => "Running", "sleeping" => "Sleeping", "waiting" => "Waiting",
17
17
  "completed" => "Completed", "compensated" => "Undone", "blocked" => "Needs a person",
18
- "superseded" => "Superseded"
18
+ "superseded" => "Superseded", "continued" => "Continued"
19
19
  }.freeze
20
20
 
21
21
  def status_pill(status)
@@ -17,6 +17,9 @@
17
17
  <% if @execution.forked_from %>
18
18
  Rerun of <%= execution_link(@execution.forked_from, class: "mono") %>.
19
19
  <% end %>
20
+ <% if @execution.continued? %>
21
+ Continued as <%= execution_link(@execution.continued_as, class: "mono") %>.
22
+ <% end %>
20
23
  </p>
21
24
  </div>
22
25
  <span class="badge"><%= ActiveDurable::DashboardHelper::STATUS_WORDS.fetch(@execution.status, @execution.status) %><%= ", undoing" if compensating_now?(@execution) %></span>
@@ -52,6 +52,21 @@ module ActiveDurable
52
52
  NoMatchingPatternError LocalJumpError RegexpError EncodingError
53
53
  ].freeze
54
54
 
55
+ # Who the work is for (the tenant, the user, the request id), carried from where an execution starts to every
56
+ # run of it. An object with two methods:
57
+ #
58
+ # module TenantContext
59
+ # def self.capture = { "tenant_id" => Current.tenant_id } # when the execution starts
60
+ # def self.wrap(context, &run) = Current.set(tenant_id: context["tenant_id"], &run) # around every run
61
+ # end
62
+ # config.context = TenantContext
63
+ #
64
+ # capture returns plain JSON (or nil), stored with the execution; ActiveDurable.start(..., context: {...})
65
+ # gives it explicitly instead. wrap runs around the recipe, the undos, the flow.on hooks and each
66
+ # flow.parallel branch thread, and must call the block exactly once. Executions started without a context
67
+ # get {}. Needs the context column (rails generate active_durable:upgrade).
68
+ attr_accessor :context
69
+
55
70
  attr_writer :logger
56
71
 
57
72
  def initialize
@@ -67,6 +82,7 @@ module ActiveDurable
67
82
  @clock = -> { Time.current }
68
83
  @dashboard_authorize = nil
69
84
  @code_errors = DEFAULT_CODE_ERRORS.dup
85
+ @context = nil
70
86
  @logger = nil
71
87
  end
72
88
 
@@ -9,14 +9,17 @@ module ActiveDurable
9
9
  # wake-up time (a sleep or a retry) or waiting for a signal.
10
10
  ACTIVE = %w[pending running sleeping waiting].freeze
11
11
  # Statuses where nothing happens without a person: done, undone, blocked (needs {ActiveDurable.retry},
12
- # {ActiveDurable.compensate} or a fix), or replaced by a rerun.
13
- TERMINAL = %w[completed compensated blocked superseded].freeze
14
- # Statuses that never change again: signals are refused and ActiveDurable.prune may delete them.
15
- FINISHED = %w[completed compensated superseded].freeze
12
+ # {ActiveDurable.compensate} or a fix), replaced by a rerun, or carried on by another execution
13
+ # (flow.continue_as_new, see {#continued_as}).
14
+ TERMINAL = %w[completed compensated blocked superseded continued].freeze
15
+ # Statuses that never change again. Signals are refused (a continued execution passes them on), and
16
+ # ActiveDurable.prune may delete them (a continued one only together with the execution that carried it on).
17
+ FINISHED = %w[completed compensated superseded continued].freeze
16
18
  # Every status.
17
19
  STATUSES = (ACTIVE + TERMINAL).freeze
18
20
 
19
21
  attribute :input, JSON_TYPE, default: -> { {} }
22
+ attribute :context, JSON_TYPE
20
23
  attribute :output, JSON_TYPE
21
24
  attribute :error, JSON_TYPE
22
25
 
@@ -34,6 +37,16 @@ module ActiveDurable
34
37
  # the sweeper finds the pending row and enqueues it.
35
38
  after_create_commit { ActiveDurable.enqueue(id) }
36
39
 
40
+ # @return [Boolean] whether flow.continue_as_new closed it; {#continued_as} is the id that carries on
41
+ def continued?
42
+ status == "continued"
43
+ end
44
+
45
+ # @api private
46
+ def self.continuations?
47
+ column_names.include?("continued_as")
48
+ end
49
+
37
50
  # @return [Boolean] whether it will move on by itself
38
51
  def active?
39
52
  ACTIVE.include?(status)
@@ -199,36 +199,79 @@ module ActiveDurable
199
199
  # Waits, without holding a worker, for {ActiveDurable.signal}(execution_id, name, payload). A signal sent
200
200
  # before the saga gets here is kept.
201
201
  #
202
- # @param name [Symbol, String]
202
+ # By default the signal has the step's name. With `signal:`, several waits share one signal name, like a
203
+ # mailbox: a conversation can wait for :message on every turn, and a message that arrives while the saga is
204
+ # busy waits there for the next turn. With `all: true`, the wait takes every pending signal of that name at
205
+ # once and returns their payloads in the order they arrived, so a burst of five messages is one turn, not five.
206
+ #
207
+ # @param name [Symbol, String] the step's name
203
208
  # @param timeout [ActiveSupport::Duration, Numeric, nil] seconds; then the step fails and the saga is undone
204
- # @return [Object] the signal's payload
209
+ # @param signal [Symbol, String, nil] the signal to wait for, when it is not the step's name
210
+ # @param all [Boolean] take every pending signal and return an Array of payloads
211
+ # @return [Object] the signal's payload, or an Array of payloads with `all: true`
205
212
  # @raise [StepFailed] when the timeout passes first
206
- def wait_for(name, timeout: nil)
213
+ # @example A chat that waits for the customer on every turn
214
+ # messages = flow.wait_for("reply.#{turn}", signal: :message, all: true, timeout: 1.day)
215
+ def wait_for(name, timeout: nil, signal: nil, all: false)
207
216
  name, position = visit!(name, "wait")
217
+ signal_name = signal.nil? ? name : signal.to_s
218
+ check_signal_name!(name, signal_name)
208
219
  entry = @notebook[name]
209
220
  return entry.result.deep_dup if entry&.completed?
210
221
  raise StepFailed.new(name, entry.error&.fetch("message", nil)) if entry&.failed?
211
222
  raise StopForward, name if compensating?
212
223
 
213
- signal = SignalRecord.next_for(execution_id, name)
214
- return consume_signal(signal, name, position).deep_dup if signal
224
+ signals = pending_signals(signal_name, all)
225
+ return consume_signals(signals, name, position, all).deep_dup if signals.any?
215
226
 
216
- unless entry&.waiting? # first time here, or retried after a timeout: wait again, with a new deadline
217
- deadline = timeout && (now + timeout)
218
- @notebook.wait!(name, kind: "wait", position: position, wake_at: deadline)
219
- @runner.suspend!(deadline, "waiting")
220
- end
227
+ keep_waiting(name, position, entry, timeout, signal_name)
228
+ end
221
229
 
222
- if entry.wake_at && now >= entry.wake_at
223
- error = WaitTimeout.new("no :#{name} signal arrived before #{entry.wake_at.utc.iso8601}")
224
- @notebook.fail!(name, kind: "wait", position: position, attempts: 1,
225
- error: ActiveDurable.dump_error(error, step: name))
226
- raise StepFailed.new(name, error)
230
+ # Closes this execution and starts a new one of the same recipe (its latest version) with `input`, the same
231
+ # context and the signals still waiting to be read. It keeps a long-lived saga, such as a conversation, from
232
+ # replaying an ever longer notebook: carry over a summary of what matters, not the whole history.
233
+ #
234
+ # The new execution's id is this one's plus "~2" (then "~3"...), so its steps get new tickets.
235
+ # {ActiveDurable.signal} sent to any id of the chain reaches the newest one; {ActiveDurable.find} with
236
+ # `latest: true` returns it. Call it in the recipe, not inside a step, and compute `input` from step results
237
+ # so a replay computes the same. Hooks of this execution do not run.
238
+ #
239
+ # @param input [Hash] the input of the new execution
240
+ # @raise [InvalidRecipe] inside a step or a parallel branch, or while undos are pending: those steps
241
+ # could never be undone. Call it before them, or after flow.pivot.
242
+ # @example A conversation that starts a fresh notebook every 20 turns
243
+ # flow.continue_as_new(summary: summary, turn: turn) if turn % 20 == 19
244
+ def continue_as_new(**input)
245
+ raise InvalidRecipe, "flow.continue_as_new must be called in the recipe, not inside a step" if inside_step?
246
+ raise blocked_by if blocked_by
247
+ raise StopForward, "continue_as_new" if compensating?
248
+
249
+ if @undo_stack.any? && !@pivoted
250
+ names = @undo_stack.map { |entry| ":#{entry.name}" }.join(", ")
251
+ raise InvalidRecipe, "flow.continue_as_new would leave #{names} without their undo. Continue before " \
252
+ "steps with undo:, or after flow.pivot."
227
253
  end
228
254
 
229
- @runner.suspend!(entry.wake_at, "waiting")
255
+ @runner.continue_as_new!(Serializer.normalize(input, "the input of continue_as_new"))
256
+ end
257
+
258
+ # @api private
259
+ def inside_step?
260
+ Thread.current[INSIDE_STEP] == true
261
+ end
262
+
263
+ # @api private
264
+ def inside_step
265
+ previous = Thread.current[INSIDE_STEP]
266
+ Thread.current[INSIDE_STEP] = true
267
+ yield
268
+ ensure
269
+ Thread.current[INSIDE_STEP] = previous
230
270
  end
231
271
 
272
+ # @api private
273
+ INSIDE_STEP = :active_durable_inside_step
274
+
232
275
  # Called when the recipe returns: every step the notebook knows about must have been reached.
233
276
  #
234
277
  # @api private
@@ -292,9 +335,11 @@ module ActiveDurable
292
335
  ActiveDurable.crash_point(:before_step, name)
293
336
  result = ActiveDurable.instrument("step", execution_id: execution_id, step: name, kind: kind) do
294
337
  if kind == "transaction"
295
- @notebook.transaction(records: name) { record_result(name, kind, position, block.call(ticket)) }
338
+ @notebook.transaction(records: name) do
339
+ record_result(name, kind, position, inside_step { block.call(ticket) })
340
+ end
296
341
  else
297
- record_result(name, kind, position, block.call(ticket))
342
+ record_result(name, kind, position, inside_step { block.call(ticket) })
298
343
  end
299
344
  end
300
345
  ActiveDurable.crash_point(:after_record, name)
@@ -411,15 +456,49 @@ module ActiveDurable
411
456
  "recipe reached :#{name} (#{kind}). #{RECIPE_CHANGED_HINT}"
412
457
  end
413
458
 
414
- def consume_signal(signal, name, position)
415
- payload = signal.payload
459
+ def pending_signals(signal_name, all)
460
+ signals = SignalRecord.pending_for(execution_id, signal_name)
461
+ all ? signals.to_a : signals.limit(1).to_a
462
+ end
463
+
464
+ # Suspends the execution until a signal arrives, or fails the wait once its deadline has passed.
465
+ def keep_waiting(name, position, entry, timeout, signal_name)
466
+ unless entry&.waiting? # first time here, or retried after a timeout: wait again, with a new deadline
467
+ deadline = timeout && (now + timeout)
468
+ @notebook.wait!(name, kind: "wait", position: position, wake_at: deadline,
469
+ signal_name: (signal_name unless signal_name == name))
470
+ @runner.suspend!(deadline, "waiting")
471
+ end
472
+
473
+ if entry.wake_at && now >= entry.wake_at
474
+ error = WaitTimeout.new("no :#{signal_name} signal arrived before #{entry.wake_at.utc.iso8601}")
475
+ @notebook.fail!(name, kind: "wait", position: position, attempts: 1,
476
+ error: ActiveDurable.dump_error(error, step: name))
477
+ raise StepFailed.new(name, error)
478
+ end
479
+
480
+ @runner.suspend!(entry.wake_at, "waiting")
481
+ end
482
+
483
+ # Takes the signals and completes the wait in one transaction: either both happen or neither does.
484
+ def consume_signals(signals, name, position, all)
485
+ result = all ? signals.map(&:payload) : signals.first.payload
486
+ ids = signals.map(&:id)
416
487
  @notebook.transaction do
417
- taken = SignalRecord.where(id: signal.id, consumed_at: nil).update_all(consumed_at: now)
418
- raise LeaseLost, "signal #{signal.id} for #{execution_id} was already consumed" if taken.zero?
488
+ taken = SignalRecord.where(id: ids, consumed_at: nil).update_all(consumed_at: now)
489
+ raise LeaseLost, "a signal for #{execution_id} was already consumed" if taken != ids.size
419
490
 
420
- @notebook.complete!(name, kind: "wait", position: position, result: payload)
491
+ @notebook.complete!(name, kind: "wait", position: position, result: result)
421
492
  end
422
- payload
493
+ result
494
+ end
495
+
496
+ def check_signal_name!(name, signal_name)
497
+ raise InvalidRecipe, "flow.wait_for :#{name} needs a signal name that is not blank" if signal_name.empty?
498
+ return if signal_name == name || Step.signal_names?
499
+
500
+ raise InvalidRecipe, "flow.wait_for :#{name}, signal: :#{signal_name} needs the signal_name column. Run " \
501
+ "rails generate active_durable:upgrade and rails db:migrate."
423
502
  end
424
503
  end
425
504
  end
@@ -99,7 +99,8 @@ module ActiveDurable
99
99
  wakes = outcomes.values.filter_map { |state, value| value if state == :retry }
100
100
  @runner.suspend!(wakes.min, "sleeping") if wakes.any?
101
101
 
102
- results = branches.to_h { |branch| [branch.name, outcomes.fetch(branch.full_name).last] }
102
+ # By branch name in sorted order, which is also how a replay reads it back from the notebook.
103
+ results = Serializer.canonical(branches.to_h { |branch| [branch.name, outcomes.fetch(branch.full_name).last] })
103
104
  @notebook.complete!(name, kind: "parallel", position: position, result: results)
104
105
  remember_branches(name, branches)
105
106
  results.deep_dup
@@ -155,9 +156,12 @@ module ActiveDurable
155
156
  end
156
157
 
157
158
  def in_branch_context(wrappers, &block)
159
+ context = @runner.execution.context
158
160
  run = lambda do
159
161
  Record.connection_pool.with_connection do
160
- wrappers.reverse.inject(block) { |inner, (wrapper, captured)| -> { wrapper.wrap(captured, &inner) } }.call
162
+ ActiveDurable.with_context(context) do
163
+ wrappers.reverse.inject(block) { |inner, (wrapper, captured)| -> { wrapper.wrap(captured, &inner) } }.call
164
+ end
161
165
  end
162
166
  end
163
167
  if defined?(Rails) && Rails.respond_to?(:application) && Rails.application
@@ -175,10 +179,10 @@ module ActiveDurable
175
179
  kind: branch.kind) do
176
180
  if branch.kind == "transaction"
177
181
  @notebook.transaction(records: branch.full_name) do
178
- record_result(branch.full_name, branch.kind, nil, branch.block.call(ticket))
182
+ record_result(branch.full_name, branch.kind, nil, inside_step { branch.block.call(ticket) })
179
183
  end
180
184
  else
181
- record_result(branch.full_name, branch.kind, nil, branch.block.call(ticket))
185
+ record_result(branch.full_name, branch.kind, nil, inside_step { branch.block.call(ticket) })
182
186
  end
183
187
  end
184
188
  ActiveDurable.crash_point(:after_record, branch.full_name)
@@ -53,8 +53,10 @@ module ActiveDurable
53
53
  write!(name, kind: kind, position: position, status: "blocked", attempts: attempts, error: error, wake_at: nil)
54
54
  end
55
55
 
56
- def wait!(name, kind:, position:, wake_at:)
57
- write!(name, kind: kind, position: position, status: "waiting", wake_at: wake_at)
56
+ def wait!(name, kind:, position:, wake_at:, signal_name: nil)
57
+ attributes = { kind: kind, position: position, status: "waiting", wake_at: wake_at }
58
+ attributes[:signal_name] = signal_name if Step.signal_names?
59
+ write!(name, **attributes)
58
60
  end
59
61
 
60
62
  # Runs the block in a database transaction and remembers the notebook writes made inside it only once it
@@ -58,7 +58,7 @@ module ActiveDurable
58
58
  start = rerun_start(original, from)
59
59
  execution = Execution.create!(id: rerun_id(original), recipe: original.recipe,
60
60
  recipe_version: original.recipe_version, input: original.input,
61
- status: "pending", forked_from: original.id)
61
+ context: original.context, status: "pending", forked_from: original.id)
62
62
  copy_steps(original, execution, before: start.position)
63
63
  original.update_columns(status: "superseded", updated_at: ActiveDurable.now) if original.status == "blocked"
64
64
  end
@@ -78,6 +78,7 @@ module ActiveDurable
78
78
  def with_idle_execution(execution_id, allowed:)
79
79
  execution = Record.transaction do
80
80
  record = Execution.lock.find(execution_id)
81
+ raise Error, "#{record.id} continued as #{record.continued_as}; use that one" if record.continued?
81
82
  unless allowed.include?(record.status)
82
83
  raise Error, "#{record.id} is #{record.status}; this works on #{allowed.join(", ")} executions"
83
84
  end
@@ -98,6 +99,7 @@ module ActiveDurable
98
99
  end
99
100
 
100
101
  def check_rerunnable!(original)
102
+ raise Error, "#{original.id} continued as #{original.continued_as}; rerun that one" if original.continued?
101
103
  return if original.status == "completed"
102
104
  return if original.status == "blocked" && !original.compensating
103
105
 
@@ -2,7 +2,8 @@
2
2
 
3
3
  module ActiveDurable
4
4
  # Deletes finished executions (completed, compensated, superseded) older than a cutoff, with their notebook and
5
- # their signals. Active and blocked executions are never touched: they still need a worker or a person.
5
+ # their signals. Active and blocked executions are never touched: they still need a worker or a person. A
6
+ # continued execution goes together with the one that carried it on, never before: its id may still be in use.
6
7
  #
7
8
  # @api private
8
9
  module Pruner
@@ -16,9 +17,12 @@ module ActiveDurable
16
17
  cutoff = now - older_than.to_f
17
18
  deleted = 0
18
19
  loop do
19
- ids = Execution.where(status: Execution::FINISHED).where("updated_at < ?", cutoff).limit(batch_size).pluck(:id)
20
+ ids = Execution.where(status: Execution::FINISHED - ["continued"]).where("updated_at < ?", cutoff)
21
+ .limit(batch_size).pluck(:id)
20
22
  break if ids.empty?
21
23
 
24
+ ids += predecessors(ids)
25
+
22
26
  Record.transaction do
23
27
  Step.where(execution_id: ids).delete_all
24
28
  SignalRecord.where(execution_id: ids).delete_all
@@ -28,5 +32,18 @@ module ActiveDurable
28
32
  end
29
33
  deleted
30
34
  end
35
+
36
+ # The continued executions that led to these ones, all the way back.
37
+ def predecessors(ids)
38
+ return [] unless Execution.continuations?
39
+
40
+ found = []
41
+ layer = ids
42
+ until layer.empty?
43
+ layer = Execution.where(continued_as: layer).pluck(:id)
44
+ found.concat(layer)
45
+ end
46
+ found
47
+ end
31
48
  end
32
49
  end
@@ -8,8 +8,16 @@ module ActiveDurable
8
8
  class Record < ActiveRecord::Base
9
9
  self.abstract_class = true
10
10
 
11
+ # JSON that reads back with its hash keys sorted, whatever order the database stored them in (see
12
+ # Serializer.canonical).
13
+ class CanonicalJson < ActiveRecord::Type::Json
14
+ def deserialize(value)
15
+ Serializer.canonical(super)
16
+ end
17
+ end
18
+
11
19
  # A JSON type object (not the :json symbol): resolving a symbol needs a database connection,
12
20
  # and these classes may load before the app has configured one.
13
- JSON_TYPE = ActiveRecord::Type::Json.new
21
+ JSON_TYPE = CanonicalJson.new
14
22
  end
15
23
  end
@@ -8,7 +8,7 @@ module ActiveDurable
8
8
  class Runner
9
9
  SUSPEND = :active_durable_suspend
10
10
 
11
- # Returns :completed, :compensated, :blocked, :sleeping, :waiting, :busy or :lease_lost.
11
+ # Returns :completed, :compensated, :continued, :blocked, :sleeping, :waiting, :busy or :lease_lost.
12
12
  def self.run(execution_id)
13
13
  lease = Lease.claim(execution_id)
14
14
  return :busy unless lease
@@ -25,8 +25,10 @@ module ActiveDurable
25
25
  end
26
26
 
27
27
  def run
28
- ActiveDurable.instrument("execution", execution_id: execution.id, recipe: execution.recipe) do
29
- catch(SUSPEND) { run_recipe }
28
+ ActiveDurable.with_context(execution.context) do
29
+ ActiveDurable.instrument("execution", execution_id: execution.id, recipe: execution.recipe) do
30
+ catch(SUSPEND) { run_recipe }
31
+ end
30
32
  end
31
33
  rescue LeaseLost => e
32
34
  ActiveDurable.config.logger.warn("[ActiveDurable] #{e.message}")
@@ -44,6 +46,29 @@ module ActiveDurable
44
46
  throw SUSPEND, status.to_sym
45
47
  end
46
48
 
49
+ # flow.continue_as_new: in one transaction, closes this execution, creates the one that carries on and moves the
50
+ # pending signals to it. Closing it first takes its row lock, so ActiveDurable.signal (which locks the row too)
51
+ # either commits before, and its signal is moved, or waits and then follows continued_as.
52
+ def continue_as_new!(input)
53
+ unless Execution.continuations?
54
+ raise InvalidRecipe, "flow.continue_as_new needs the continued_as column. Run rails generate " \
55
+ "active_durable:upgrade and rails db:migrate."
56
+ end
57
+
58
+ successor_id = ActiveDurable.next_in_chain(execution.id)
59
+ recipe = ActiveDurable.registry.latest(execution.recipe)
60
+ Record.transaction do
61
+ lease.release!(status: "continued", continued_as: successor_id, wake_at: nil)
62
+ ActiveDurable.crash_point(:continue_as_new, successor_id)
63
+ Execution.create!(id: successor_id, recipe: recipe.name, recipe_version: recipe.version, input: input,
64
+ context: execution.context, status: "pending")
65
+ SignalRecord.pending.where(execution_id: execution.id).update_all(execution_id: successor_id)
66
+ end
67
+ ActiveDurable.instrument("continued", execution_id: execution.id, recipe: execution.recipe,
68
+ continued_as: successor_id)
69
+ throw SUSPEND, :continued
70
+ end
71
+
47
72
  private
48
73
 
49
74
  def run_recipe
@@ -2,8 +2,8 @@
2
2
 
3
3
  module ActiveDurable
4
4
  # Step results, inputs and signal payloads live in the notebook as JSON. This module turns a value
5
- # into exactly what a later replay will read back (string keys, no symbols, UTF-8 strings), so the first run
6
- # and a replay behave the same. Anything that is not plain JSON, or that some database cannot store (a NUL
5
+ # into exactly what a later replay will read back (string keys, no symbols, UTF-8 strings, keys sorted), so the
6
+ # first run and a replay behave the same. Anything that is not plain JSON, or that some database cannot store (a NUL
7
7
  # character, bytes that are not UTF-8), is rejected with a message that says where.
8
8
  #
9
9
  # @api private
@@ -27,12 +27,25 @@ module ActiveDurable
27
27
  end
28
28
 
29
29
  def normalize_hash(hash, path)
30
- hash.each_with_object({}) do |(key, item), out|
30
+ out = hash.each_with_object({}) do |(key, item), normalized|
31
31
  key = key.to_s if key.is_a?(Symbol)
32
32
  raise NotSerializable, "#{path} has a #{key.class} key; use strings or symbols" unless key.is_a?(String)
33
33
 
34
34
  key = utf8(key, "a key of #{path}")
35
- out[key] = normalize(item, "#{path}[#{key.inspect}]")
35
+ normalized[key] = normalize(item, "#{path}[#{key.inspect}]")
36
+ end
37
+ out.sort.to_h
38
+ end
39
+
40
+ # Databases do not keep the order of hash keys: PostgreSQL's jsonb and MySQL's JSON put shorter keys first.
41
+ # Sorting them, when writing and when reading back, gives a replay the very same hash as the first run, down
42
+ # to the order of its keys. That matters when a result is sent somewhere that compares bytes, such as an AI
43
+ # provider that checks the conversation it is sent again.
44
+ def canonical(value)
45
+ case value
46
+ when Hash then value.sort.to_h { |key, item| [key, canonical(item)] }
47
+ when Array then value.map { |item| canonical(item) }
48
+ else value
36
49
  end
37
50
  end
38
51
 
@@ -17,17 +17,22 @@ module ActiveDurable
17
17
  # for a later flow.wait_for) must not wake the execution up: it would only go back to waiting, forever.
18
18
  scope :awaited, lambda {
19
19
  steps = Step.arel_table
20
+ awaits = if Step.signal_names?
21
+ Arel::Nodes::NamedFunction.new("COALESCE", [steps[:signal_name], steps[:name]])
22
+ else
23
+ steps[:name]
24
+ end
20
25
  waiting = steps.project(Arel.sql("1"))
21
26
  .where(steps[:execution_id].eq(arel_table[:execution_id])
22
- .and(steps[:name].eq(arel_table[:name]))
27
+ .and(awaits.eq(arel_table[:name]))
23
28
  .and(steps[:status].eq("waiting")))
24
29
  pending.where(waiting.exists)
25
30
  }
26
31
 
27
32
  after_create_commit { ActiveDurable.enqueue(execution_id) }
28
33
 
29
- def self.next_for(execution_id, name)
30
- pending.where(execution_id: execution_id, name: name.to_s).order(:id).first
34
+ def self.pending_for(execution_id, name)
35
+ pending.where(execution_id: execution_id, name: name.to_s).order(:id)
31
36
  end
32
37
  end
33
38
  end
@@ -37,6 +37,16 @@ module ActiveDurable
37
37
  status == "waiting"
38
38
  end
39
39
 
40
+ # The name of the signal a waiting step is waiting for.
41
+ def awaits
42
+ (self.class.signal_names? && signal_name) || name
43
+ end
44
+
45
+ # @api private
46
+ def self.signal_names?
47
+ column_names.include?("signal_name")
48
+ end
49
+
40
50
  def undo?
41
51
  kind == "undo"
42
52
  end
@@ -23,16 +23,21 @@ module ActiveDurable
23
23
  end
24
24
 
25
25
  # Runs an execution synchronously until it finishes, fast-forwarding sleeps, retries and expired
26
- # leases. Signals given as { name => payload } are delivered when the saga waits for them.
27
- # Returns the execution; it stays "waiting" if it waits for a signal you did not provide.
28
- def drain(execution_id, signals: {}, max_runs: 500)
26
+ # leases, and following flow.continue_as_new to the execution that carries on. Signals given as
27
+ # { name => payload } are delivered, once each, when the saga waits for them. Returns the (last) execution; it
28
+ # stays "waiting" if it waits for a signal you did not provide.
29
+ #
30
+ # A wait with a timeout is fast-forwarded to it too, so the saga goes on as if no signal came. Pass
31
+ # `expire_waits: false` to stop at the wait instead, send signals yourself and drain again: that is how to
32
+ # test a conversation turn by turn.
33
+ def drain(execution_id, signals: {}, max_runs: 500, expire_waits: true)
29
34
  signals = signals.transform_keys(&:to_s)
30
35
  max_runs.times do
31
- execution = Execution.find(execution_id)
36
+ execution = ActiveDurable.find(execution_id, latest: true)
32
37
  return execution if execution.terminal?
33
- return execution unless ready_for_next_run?(execution, signals)
38
+ return execution unless ready_for_next_run?(execution, signals, expire_waits: expire_waits)
34
39
 
35
- Runner.run(execution_id)
40
+ Runner.run(execution.id)
36
41
  end
37
42
  raise Error, "execution #{execution_id} did not settle after #{max_runs} runs"
38
43
  end
@@ -96,12 +101,12 @@ module ActiveDurable
96
101
  # Decides what has to happen before the next run. Returns false when nothing can move the execution.
97
102
  #
98
103
  # @api private
99
- def ready_for_next_run?(execution, signals)
104
+ def ready_for_next_run?(execution, signals, expire_waits: true)
100
105
  now = ActiveDurable.now
101
106
  if execution.locked_until && execution.locked_until > now
102
107
  travel(execution.locked_until - now + 0.001) # a crashed worker still holds the lease
103
108
  elsif execution.status == "waiting"
104
- return waiting_can_move?(execution, signals, now)
109
+ return waiting_can_move?(execution, signals, now, expire_waits: expire_waits)
105
110
  elsif execution.wake_at && execution.wake_at > now
106
111
  travel(execution.wake_at - now + 0.001)
107
112
  end
@@ -109,13 +114,13 @@ module ActiveDurable
109
114
  end
110
115
 
111
116
  # @api private
112
- def waiting_can_move?(execution, signals, now)
117
+ def waiting_can_move?(execution, signals, now, expire_waits: true)
113
118
  return true if SignalRecord.awaited.exists?(execution_id: execution.id)
114
119
 
115
120
  waiting = execution.steps.find_by(status: "waiting")
116
- if waiting && signals.key?(waiting.name)
117
- ActiveDurable.signal(execution.id, waiting.name, signals[waiting.name])
118
- elsif execution.wake_at
121
+ if waiting && signals.key?(waiting.awaits)
122
+ ActiveDurable.signal(execution.id, waiting.awaits, signals.delete(waiting.awaits))
123
+ elsif execution.wake_at && expire_waits
119
124
  travel(execution.wake_at - now + 0.001) if execution.wake_at > now
120
125
  else
121
126
  return false
@@ -2,5 +2,5 @@
2
2
 
3
3
  module ActiveDurable
4
4
  # The gem's version.
5
- VERSION = "0.7.0"
5
+ VERSION = "0.8.0"
6
6
  end
@@ -124,10 +124,12 @@ module ActiveDurable
124
124
  # @raise [NotSerializable] if the input cannot be stored as JSON
125
125
  # @example
126
126
  # Durable.start(:checkout, id: "checkout-#{order.id}", order_id: order.id)
127
- def start(name, id: nil, **input)
127
+ def start(name, id: nil, context: nil, **input)
128
128
  recipe = registry.latest(name)
129
129
  attributes = { recipe: recipe.name, recipe_version: recipe.version, status: "pending",
130
130
  input: Serializer.normalize(input, "input") }
131
+ captured = capture_context(context)
132
+ attributes[:context] = captured unless captured.nil?
131
133
  return Execution.create!(attributes.merge(id: "#{recipe.name}-#{SecureRandom.uuid}")) if id.nil?
132
134
 
133
135
  id = check_id!(id)
@@ -143,6 +145,37 @@ module ActiveDurable
143
145
  raise ArgumentError, "execution #{id} already exists for recipe :#{execution.recipe}"
144
146
  end
145
147
 
148
+ # The context to store with a new execution: the one given to start, or config.context.capture.
149
+ #
150
+ # @api private
151
+ def capture_context(explicit)
152
+ value = explicit.nil? ? config.context&.capture : explicit
153
+ return nil if value.nil?
154
+
155
+ unless Execution.column_names.include?("context")
156
+ raise Error, "the durable_executions table has no context column. Run rails generate " \
157
+ "active_durable:upgrade and rails db:migrate."
158
+ end
159
+ Serializer.normalize(value, "the context")
160
+ end
161
+
162
+ # Runs the block inside config.context.wrap, with the context the execution started with.
163
+ #
164
+ # @api private
165
+ def with_context(context, &block)
166
+ wrapper = config.context
167
+ return yield if wrapper.nil?
168
+
169
+ ran = false
170
+ result = wrapper.wrap(context || {}) do
171
+ ran = true
172
+ block.call
173
+ end
174
+ raise Error, "config.context.wrap must call its block" unless ran
175
+
176
+ result
177
+ end
178
+
146
179
  # @api private
147
180
  def check_id!(id)
148
181
  id = Serializer.normalize(id.to_s, "the execution id")
@@ -167,18 +200,33 @@ module ActiveDurable
167
200
  # @example In a webhook controller
168
201
  # Durable.signal("loan-42", :kyc_done, verified: true)
169
202
  def signal(execution_id, name, payload = nil)
170
- execution = Execution.find(execution_id)
171
- raise Error, "execution #{execution_id} already finished (#{execution.status})" if execution.finished?
203
+ payload = Serializer.normalize(payload, "signal payload")
204
+ Record.transaction do
205
+ # Locked, so a flow.continue_as_new committing right now cannot strand the signal on the closed execution.
206
+ execution = Execution.lock.find(execution_id)
207
+ execution = Execution.lock.find(execution.continued_as) while execution.continued?
208
+ raise Error, "execution #{execution_id} already finished (#{execution.status})" if execution.finished?
172
209
 
173
- SignalRecord.create!(execution_id: execution.id, name: name.to_s,
174
- payload: Serializer.normalize(payload, "signal payload"))
210
+ SignalRecord.create!(execution_id: execution.id, name: name.to_s, payload: payload)
211
+ end
175
212
  end
176
213
 
177
214
  # @param execution_id [String]
215
+ # @param latest [Boolean] follow flow.continue_as_new to the execution that carries on now
178
216
  # @return [Execution]
179
217
  # @raise [ActiveRecord::RecordNotFound]
180
- def find(execution_id)
181
- Execution.find(execution_id)
218
+ def find(execution_id, latest: false)
219
+ execution = Execution.find(execution_id)
220
+ execution = Execution.find(execution.continued_as) while latest && execution.continued?
221
+ execution
222
+ end
223
+
224
+ # "order-1" continues as "order-1~2", then "order-1~3".
225
+ #
226
+ # @api private
227
+ def next_in_chain(execution_id)
228
+ base, number = execution_id.match(/\A(.+)~(\d+)\z/)&.captures
229
+ base ? "#{base}~#{number.to_i + 1}" : "#{execution_id}~2"
182
230
  end
183
231
 
184
232
  # Resumes a blocked execution where it stopped: failed steps (or failed undos, if it was undoing) get a fresh
@@ -10,18 +10,21 @@ class CreateActiveDurableTables < ActiveRecord::Migration<%= migration_version %
10
10
  t.string :status, null: false
11
11
  t.boolean :compensating, null: false, default: false
12
12
  t.column :input, json_type
13
+ t.column :context, json_type
13
14
  t.column :output, json_type
14
15
  t.column :error, json_type
15
16
  t.datetime :wake_at, precision: 6
16
17
  t.datetime :locked_until, precision: 6
17
18
  t.string :lease_token
18
19
  t.string :forked_from, **exact
20
+ t.string :continued_as, **exact
19
21
  t.timestamps precision: 6
20
22
  end
21
23
  add_index :durable_executions, %i[status wake_at]
22
24
  add_index :durable_executions, %i[status updated_at]
23
25
  add_index :durable_executions, %i[recipe status]
24
26
  add_index :durable_executions, :forked_from
27
+ add_index :durable_executions, :continued_as
25
28
 
26
29
  # The notebook: one row per step (and per undo) of an execution.
27
30
  create_table :durable_steps do |t|
@@ -31,6 +34,7 @@ class CreateActiveDurableTables < ActiveRecord::Migration<%= migration_version %
31
34
  t.integer :position
32
35
  t.string :status, null: false
33
36
  t.integer :attempts, null: false, default: 0
37
+ t.string :signal_name, **exact
34
38
  t.column :result, json_type
35
39
  t.column :error, json_type
36
40
  t.datetime :wake_at, precision: 6
@@ -0,0 +1,19 @@
1
+ # frozen_string_literal: true
2
+
3
+ # ActiveDurable 0.8: the context an execution started with (config.context), given back to it on every run.
4
+ class AddActiveDurableContext < ActiveRecord::Migration<%= migration_version %>
5
+ def up
6
+ add_column :durable_executions, :context, json_type unless column_exists?(:durable_executions, :context)
7
+ end
8
+
9
+ def down
10
+ remove_column :durable_executions, :context if column_exists?(:durable_executions, :context)
11
+ end
12
+
13
+ private
14
+
15
+ # jsonb on PostgreSQL, json elsewhere, like the other JSON columns.
16
+ def json_type
17
+ connection.adapter_name.downcase.include?("postg") ? :jsonb : :json
18
+ end
19
+ end
@@ -0,0 +1,21 @@
1
+ # frozen_string_literal: true
2
+
3
+ # ActiveDurable 0.8: flow.continue_as_new closes an execution and points it to the one that carries on.
4
+ class AddActiveDurableContinuedAs < ActiveRecord::Migration<%= migration_version %>
5
+ def up
6
+ add_column :durable_executions, :continued_as, :string, **exact unless column_exists?(:durable_executions, :continued_as)
7
+ add_index :durable_executions, :continued_as unless index_exists?(:durable_executions, :continued_as)
8
+ end
9
+
10
+ def down
11
+ remove_index :durable_executions, :continued_as if index_exists?(:durable_executions, :continued_as)
12
+ remove_column :durable_executions, :continued_as if column_exists?(:durable_executions, :continued_as)
13
+ end
14
+
15
+ private
16
+
17
+ # MySQL compares strings ignoring case and accents by default; ids must match exactly.
18
+ def exact
19
+ connection.adapter_name.match?(/mysql|trilogy/i) ? { collation: "utf8mb4_bin" } : {}
20
+ end
21
+ end
@@ -0,0 +1,22 @@
1
+ # frozen_string_literal: true
2
+
3
+ # ActiveDurable 0.8: flow.wait_for(name, signal: ...) writes down which signal a step waits for, when it is not the
4
+ # step's own name.
5
+ class AddActiveDurableSignalName < ActiveRecord::Migration<%= migration_version %>
6
+ def up
7
+ return if column_exists?(:durable_steps, :signal_name)
8
+
9
+ add_column :durable_steps, :signal_name, :string, **exact
10
+ end
11
+
12
+ def down
13
+ remove_column :durable_steps, :signal_name if column_exists?(:durable_steps, :signal_name)
14
+ end
15
+
16
+ private
17
+
18
+ # MySQL compares strings ignoring case and accents by default; signal names must match exactly.
19
+ def exact
20
+ connection.adapter_name.match?(/mysql|trilogy/i) ? { collation: "utf8mb4_bin" } : {}
21
+ end
22
+ end
@@ -16,7 +16,10 @@ module ActiveDurable
16
16
  desc "Adds the migrations that newer ActiveDurable versions need. Run it after updating the gem."
17
17
 
18
18
  # In the order they were released. Never edit one that shipped: add a new one.
19
- MIGRATIONS = %w[add_active_durable_prune_index make_active_durable_ids_case_sensitive].freeze
19
+ MIGRATIONS = %w[
20
+ add_active_durable_prune_index make_active_durable_ids_case_sensitive add_active_durable_context
21
+ add_active_durable_signal_name add_active_durable_continued_as
22
+ ].freeze
20
23
 
21
24
  # @api private
22
25
  def create_migration_files
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: active_durable
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.7.0
4
+ version: 0.8.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - William Romero
@@ -120,7 +120,10 @@ files:
120
120
  - lib/active_durable/version.rb
121
121
  - lib/generators/active_durable/install/install_generator.rb
122
122
  - lib/generators/active_durable/install/templates/create_active_durable_tables.rb.tt
123
+ - lib/generators/active_durable/upgrade/templates/add_active_durable_context.rb.tt
124
+ - lib/generators/active_durable/upgrade/templates/add_active_durable_continued_as.rb.tt
123
125
  - lib/generators/active_durable/upgrade/templates/add_active_durable_prune_index.rb.tt
126
+ - lib/generators/active_durable/upgrade/templates/add_active_durable_signal_name.rb.tt
124
127
  - lib/generators/active_durable/upgrade/templates/make_active_durable_ids_case_sensitive.rb.tt
125
128
  - lib/generators/active_durable/upgrade/upgrade_generator.rb
126
129
  - lib/tasks/active_durable.rake