active_durable 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +37 -0
- data/README.md +156 -2
- data/app/helpers/active_durable/dashboard_helper.rb +2 -2
- data/app/views/active_durable/executions/show.html.erb +3 -0
- data/lib/active_durable/configuration.rb +16 -0
- data/lib/active_durable/execution.rb +17 -4
- data/lib/active_durable/flow.rb +103 -24
- data/lib/active_durable/flow_parallel.rb +8 -4
- data/lib/active_durable/notebook.rb +4 -2
- data/lib/active_durable/operations.rb +3 -1
- data/lib/active_durable/pruner.rb +19 -2
- data/lib/active_durable/record.rb +9 -1
- data/lib/active_durable/runner.rb +28 -3
- data/lib/active_durable/serializer.rb +17 -4
- data/lib/active_durable/signal_record.rb +8 -3
- data/lib/active_durable/step.rb +10 -0
- data/lib/active_durable/testing.rb +17 -12
- data/lib/active_durable/version.rb +1 -1
- data/lib/active_durable.rb +55 -7
- data/lib/generators/active_durable/install/templates/create_active_durable_tables.rb.tt +4 -0
- data/lib/generators/active_durable/upgrade/templates/add_active_durable_context.rb.tt +19 -0
- data/lib/generators/active_durable/upgrade/templates/add_active_durable_continued_as.rb.tt +21 -0
- data/lib/generators/active_durable/upgrade/templates/add_active_durable_signal_name.rb.tt +22 -0
- data/lib/generators/active_durable/upgrade/upgrade_generator.rb +4 -1
- metadata +4 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 47ef8eb3bb5a133221ed52ce914ec952ff7f73f552b5352d4558b96e6738458c
|
|
4
|
+
data.tar.gz: af132b5a177125726a6aed8699e7cf4391c70e9e4fbac566531ab87c3204f998
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: fe0584f97ce0594ee30c2186b8a10ac91e57505172b758cf8e667b290a6cb387e0da3566db685c8070798bdab84f32f225a7f2f1787aabc9b106bbe069ceb11e
|
|
7
|
+
data.tar.gz: 863396b2acc54f007f65c80379e3bcbac0512eb05888d3a91372cddfd88333e5b3536d25798949eaf2200a85fd021ae540a65ffd4d1d1078716e0221247a609b
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,43 @@ All notable changes to this project are documented here. The format follows
|
|
|
5
5
|
[Semantic Versioning](https://semver.org/). Schema changes ship as new migrations: after updating the gem, run
|
|
6
6
|
`bin/rails generate active_durable:upgrade` and `bin/rails db:migrate`.
|
|
7
7
|
|
|
8
|
+
## [0.8.0] - 2026-10-22
|
|
9
|
+
|
|
10
|
+
For sagas that live long and talk to people, such as a WhatsApp agent that books appointments: a mailbox for
|
|
11
|
+
messages, a fresh notebook for long conversations, and the tenant in every run.
|
|
12
|
+
|
|
13
|
+
Upgrading from 0.7: `bin/rails generate active_durable:upgrade && bin/rails db:migrate`. It adds three nullable
|
|
14
|
+
columns (`durable_executions.context`, `durable_executions.continued_as` with its index, `durable_steps.signal_name`);
|
|
15
|
+
nothing is rewritten. Note the first change below: hash keys now come back sorted.
|
|
16
|
+
|
|
17
|
+
### Added
|
|
18
|
+
|
|
19
|
+
- `flow.continue_as_new(**input)` closes the execution and starts a new one of the same recipe (its latest version)
|
|
20
|
+
with that input, the same context and the signals nobody read yet, in one transaction. Ids follow the chain
|
|
21
|
+
`order-1`, `order-1~2`, `order-1~3`. The new `continued` status; `Durable.find(id, latest: true)`.
|
|
22
|
+
`Durable.signal` to any id of the chain reaches the newest execution, also during the hand-off.
|
|
23
|
+
- `flow.wait_for(name, signal:, all:)`: several waits share one signal name, like a mailbox, and `all: true` takes
|
|
24
|
+
every pending signal at once, returning their payloads in order.
|
|
25
|
+
- `config.context`: an object with `capture` (when an execution starts) and `wrap(context) { }` (around every run,
|
|
26
|
+
undo, hook and `flow.parallel` branch). `Durable.start(..., context: {...})` gives it explicitly. Reruns keep it.
|
|
27
|
+
- `ActiveDurable::Testing.drain(id, expire_waits: false)` stops at a wait instead of fast-forwarding to its
|
|
28
|
+
timeout, to test a conversation turn by turn. `drain` follows `continue_as_new`.
|
|
29
|
+
- The dashboard shows `continued` executions and links to the one that carries on.
|
|
30
|
+
|
|
31
|
+
### Changed
|
|
32
|
+
|
|
33
|
+
- Hash keys in step results, inputs, signal payloads and `flow.parallel` results are sorted, on the first run and
|
|
34
|
+
when read back. PostgreSQL's jsonb and MySQL's JSON store keys shorter-first, so a replay used to read a hash whose
|
|
35
|
+
keys were in another order than the first run returned. That broke anything that compares bytes, such as an AI
|
|
36
|
+
provider that checks the conversation it is sent again. Executions in flight during the upgrade see their keys in
|
|
37
|
+
a new order once.
|
|
38
|
+
- `Durable.signal` locks the execution row while it writes the signal.
|
|
39
|
+
- `Testing.drain` delivers each signal given in `signals:` once per call, instead of every time a step of that name
|
|
40
|
+
waits.
|
|
41
|
+
- `Durable.start` takes `context:`, so `context` can no longer be an input key.
|
|
42
|
+
- `ActiveDurable.retry`, `compensate` and `rerun` on a continued execution raise, naming the one that carries on.
|
|
43
|
+
- `ActiveDurable.prune` deletes a continued execution only together with the one that carried it on.
|
|
44
|
+
|
|
8
45
|
## [0.7.0] - 2026-10-07
|
|
9
46
|
|
|
10
47
|
Fixes from an architecture and security review. Most of them stop a saga from paying twice or undoing the wrong
|
data/README.md
CHANGED
|
@@ -213,6 +213,7 @@ ActiveDurable.configure do |config|
|
|
|
213
213
|
config.sweep_grace = 1.minute # the sweeper leaves executions this young alone
|
|
214
214
|
config.keep_finished_for = 30.days # then ActiveDurable::PruneJob deletes finished ones
|
|
215
215
|
config.code_errors += ["Payments::Misconfigured"] # your own errors that mean a bug (see "A bug is not a failure")
|
|
216
|
+
config.context = TenantContext # who the work is for, in every run (see "Context")
|
|
216
217
|
|
|
217
218
|
# Who may open the dashboard outside development and test. With Devise (HTTP basic auth: see Dashboard):
|
|
218
219
|
config.dashboard_authorize = ->(controller) { controller.request.env["warden"]&.user&.admin? }
|
|
@@ -413,7 +414,8 @@ Three rules follow from replaying the recipe:
|
|
|
413
414
|
| `flow.pivot(name) { \|ticket\| ... }` | the step after which there is no going back | retried, then the saga is undone |
|
|
414
415
|
| `flow.parallel(name) { \|branches\| ... }` | several steps at the same time | each branch like a step |
|
|
415
416
|
| `flow.sleep(name, 3.days)` | waiting without holding a worker | — |
|
|
416
|
-
| `flow.wait_for(name, timeout:)` | waiting for `Durable.signal` | a timeout fails the saga |
|
|
417
|
+
| `flow.wait_for(name, timeout:)` | waiting for `Durable.signal` (`signal:` and `all:` for a mailbox) | a timeout fails the saga |
|
|
418
|
+
| `flow.continue_as_new(**input)` | a long-lived saga, such as a chat, that starts a fresh notebook | refused inside a step or with undos pending |
|
|
417
419
|
| `flow.on(:completed) { ... }` | updating your own records when the saga ends (also `:compensated`) | blocks; a retry runs the hook again |
|
|
418
420
|
|
|
419
421
|
Steps take `undo:`, `retry:` (`3`, `false` or `{ attempts:, backoff: }`) and `undo_on_failure:`.
|
|
@@ -537,6 +539,18 @@ Durable.signal("loan-42", :kyc_done, verified: true)
|
|
|
537
539
|
|
|
538
540
|
Pass `id:` to `Durable.start` to choose the execution id (the call becomes idempotent).
|
|
539
541
|
|
|
542
|
+
**A mailbox.** By default the signal has the step's name. A conversation waits for the customer on every turn, so
|
|
543
|
+
each turn gets its own step name and all of them share one signal name with `signal:`. A message that arrives while
|
|
544
|
+
the saga is busy waits in the mailbox for the next turn. With `all: true` the wait takes every pending message at
|
|
545
|
+
once and returns them in order, so five messages in a row are one turn, not five:
|
|
546
|
+
|
|
547
|
+
```ruby
|
|
548
|
+
messages = flow.wait_for("reply.#{turn}", signal: :message, all: true, timeout: 1.day)
|
|
549
|
+
|
|
550
|
+
# in the WhatsApp webhook
|
|
551
|
+
Durable.signal(conversation.execution_id, :message, text: params[:text])
|
|
552
|
+
```
|
|
553
|
+
|
|
540
554
|
</details>
|
|
541
555
|
|
|
542
556
|
<details>
|
|
@@ -562,6 +576,57 @@ the finished branches are undone in the order they finished. Give your connectio
|
|
|
562
576
|
|
|
563
577
|
</details>
|
|
564
578
|
|
|
579
|
+
<details>
|
|
580
|
+
<summary><b>Context: the tenant in every run</b></summary>
|
|
581
|
+
|
|
582
|
+
<br>
|
|
583
|
+
|
|
584
|
+
A saga runs in a worker, days after the request that started it, so `Current.tenant` and friends are gone.
|
|
585
|
+
`config.context` carries them: `capture` runs when the execution starts and its JSON is stored with it; `wrap` runs
|
|
586
|
+
around every run of it, including undos, hooks and each `flow.parallel` branch thread.
|
|
587
|
+
|
|
588
|
+
```ruby
|
|
589
|
+
module TenantContext
|
|
590
|
+
def self.capture = { "tenant_id" => Current.tenant_id }
|
|
591
|
+
def self.wrap(context, &run) = Current.set(tenant_id: context["tenant_id"], &run)
|
|
592
|
+
end
|
|
593
|
+
|
|
594
|
+
ActiveDurable.configure { |config| config.context = TenantContext }
|
|
595
|
+
|
|
596
|
+
Durable.start(:checkout, order_id: order.id) # captures the current tenant
|
|
597
|
+
Durable.start(:welcome, tenant_id: t.id, context: { "tenant_id" => t.id }) # or gives it explicitly
|
|
598
|
+
```
|
|
599
|
+
|
|
600
|
+
`wrap` must call the block once. Executions started without a context get `{}`.
|
|
601
|
+
|
|
602
|
+
</details>
|
|
603
|
+
|
|
604
|
+
<details>
|
|
605
|
+
<summary><b>Long conversations: continue as new</b></summary>
|
|
606
|
+
|
|
607
|
+
<br>
|
|
608
|
+
|
|
609
|
+
Every run replays the recipe from the top, reading the notebook. A chat that has gone on for 200 turns reads 1,000
|
|
610
|
+
entries before it can answer. `flow.continue_as_new` closes the execution and starts a fresh one of the same recipe
|
|
611
|
+
(its latest version) with the input you give it, usually a summary, plus the same context and the messages nobody
|
|
612
|
+
read yet. All of it happens in one transaction:
|
|
613
|
+
|
|
614
|
+
```ruby
|
|
615
|
+
Durable.define(:chat) do |flow, conversation_id:, turn: 0, summary: nil|
|
|
616
|
+
# ... the turn ...
|
|
617
|
+
flow.continue_as_new(conversation_id: conversation_id, turn: turn + 1, summary: summary) if turn % 20 == 19
|
|
618
|
+
end
|
|
619
|
+
```
|
|
620
|
+
|
|
621
|
+
The new execution's id is the old one plus `~2` (then `~3`), so its steps get new tickets. `Durable.signal` to any id
|
|
622
|
+
of the chain reaches the newest execution, even one sent during the hand-off, and
|
|
623
|
+
`Durable.find(id, latest: true)` returns it. Compute the input from step results, so a replay computes the same.
|
|
624
|
+
`continue_as_new` is refused inside a step and while undos are pending (those steps could never be undone); after
|
|
625
|
+
`flow.pivot` it is allowed. The old execution's hooks do not run, and it is pruned together with the one that
|
|
626
|
+
carries on.
|
|
627
|
+
|
|
628
|
+
</details>
|
|
629
|
+
|
|
565
630
|
<details>
|
|
566
631
|
<summary><b>Changing a recipe while sagas are in flight</b></summary>
|
|
567
632
|
|
|
@@ -581,6 +646,91 @@ New executions use the highest version; each execution keeps the one it started
|
|
|
581
646
|
|
|
582
647
|
</details>
|
|
583
648
|
|
|
649
|
+
## Example: a WhatsApp assistant that books appointments
|
|
650
|
+
|
|
651
|
+
First in plain words, then the code.
|
|
652
|
+
|
|
653
|
+
**What it does.** A customer writes to a bakery on WhatsApp: "Hi, can I come on Friday to order a cake?". An
|
|
654
|
+
assistant (an AI that can see the calendar) answers, offers the free times and books the one the customer picks. The
|
|
655
|
+
conversation can take two minutes or two days. The customer may send three messages in a row, or disappear and come
|
|
656
|
+
back tomorrow. Meanwhile your servers restart, you deploy, a process dies.
|
|
657
|
+
|
|
658
|
+
**The notebook.** Think of a cook who ticks every step of a recipe in a notebook. When someone interrupts, the cook
|
|
659
|
+
does not trust their memory: they reread the notebook from the top, skip every step with a tick, and carry on from
|
|
660
|
+
the first one without it. ActiveDurable does the same with your code. Each step's result is written in your database,
|
|
661
|
+
and every time the saga wakes up (a message arrives, a timer rings) it rereads the recipe and skips what is done. That
|
|
662
|
+
is why the assistant never books twice, even if the server dies right after booking: the booking has its tick. (A
|
|
663
|
+
step that died halfway runs again with the same ticket, so the calendar and WhatsApp can recognize a repeat.)
|
|
664
|
+
|
|
665
|
+
**Turns.** A turn is one exchange: the customer writes, the assistant answers. Each turn adds three ticks: the
|
|
666
|
+
customer's messages, the AI's answer (which may book) and the WhatsApp message sent back. The AI's answer is written
|
|
667
|
+
down too, so rereading never asks the AI again: no double cost, and the very same text.
|
|
668
|
+
|
|
669
|
+
**The mailbox.** Every message goes to one mailbox, `:message`. If the customer writes while the assistant is
|
|
670
|
+
thinking, the message waits there for the next turn. If they send five in a row, the assistant takes all five at
|
|
671
|
+
once: one turn, not five.
|
|
672
|
+
|
|
673
|
+
**A fresh page.** After 20 turns the notebook has 60 ticks to reread every time something happens. Instead of making
|
|
674
|
+
the same page longer, `continue_as_new` starts a new page with a short summary ("Ana wants a cake on Friday, saw the
|
|
675
|
+
times at 10 and 3"), keeping the messages nobody read yet.
|
|
676
|
+
|
|
677
|
+
**Who it is for.** One app serves many bakeries. `config.context` writes down which one the conversation belongs to
|
|
678
|
+
when it starts, and gives it back on every run, so each step sees the right calendar.
|
|
679
|
+
|
|
680
|
+
```ruby
|
|
681
|
+
# config/initializers/active_durable.rb
|
|
682
|
+
module TenantContext
|
|
683
|
+
def self.capture = { "tenant_id" => Current.tenant_id }
|
|
684
|
+
def self.wrap(context, &run) = Current.set(tenant_id: context["tenant_id"], &run)
|
|
685
|
+
end
|
|
686
|
+
ActiveDurable.configure { |config| config.context = TenantContext }
|
|
687
|
+
|
|
688
|
+
# app/sagas/booking_assistant_saga.rb
|
|
689
|
+
BookingAssistantSaga = Durable.define(:booking_assistant) do |flow, conversation_id:, turn: 0, summary: nil|
|
|
690
|
+
conversation = Conversation.find(conversation_id)
|
|
691
|
+
history = summary ? [{ "role" => "user", "content" => "What happened so far: #{summary}" }] : []
|
|
692
|
+
booked = nil
|
|
693
|
+
|
|
694
|
+
20.times do |i|
|
|
695
|
+
n = turn + i
|
|
696
|
+
# 1. The customer's messages: every one that arrived since the last turn.
|
|
697
|
+
messages = flow.wait_for("customer.#{n}", signal: :message, all: true)
|
|
698
|
+
history << { "role" => "user", "content" => messages.map { |m| m["text"] }.join("\n") }
|
|
699
|
+
|
|
700
|
+
# 2. The AI's answer. When it decides to book, it calls Calendar.book(slot, key: ticket): booked once.
|
|
701
|
+
reply = flow.step("assistant.#{n}") { |ticket| BookingAssistant.answer(history, ticket: ticket) }
|
|
702
|
+
history << { "role" => "assistant", "content" => reply["text"] }
|
|
703
|
+
|
|
704
|
+
# 3. The answer. If the process dies right after sending it, this step runs again with the same ticket:
|
|
705
|
+
# WhatsApp.send_text keeps the ticket with the message it sent and skips sending it twice.
|
|
706
|
+
flow.step("send.#{n}") { |ticket| WhatsApp.send_text(conversation.phone, reply["text"], key: ticket) }
|
|
707
|
+
|
|
708
|
+
if reply["booked"]
|
|
709
|
+
booked = reply
|
|
710
|
+
break
|
|
711
|
+
end
|
|
712
|
+
end
|
|
713
|
+
next booked if booked
|
|
714
|
+
|
|
715
|
+
# 20 turns without a booking: a fresh page with a summary.
|
|
716
|
+
summary = flow.step("summary.#{turn}") { BookingAssistant.summarize(history) }
|
|
717
|
+
flow.continue_as_new(conversation_id: conversation_id, turn: turn + 20, summary: summary["text"])
|
|
718
|
+
end
|
|
719
|
+
|
|
720
|
+
# app/controllers/whatsapp_webhooks_controller.rb
|
|
721
|
+
def create
|
|
722
|
+
conversation = Conversation.open_for(params[:from]) # a new one once the last one ended; tenant: Current.tenant_id
|
|
723
|
+
id = "chat-#{conversation.id}"
|
|
724
|
+
Durable.start(:booking_assistant, id: id, conversation_id: conversation.id) # idempotent: starts it once
|
|
725
|
+
Durable.signal(id, :message, text: params[:text]) # reaches the newest page of the chain
|
|
726
|
+
head :ok
|
|
727
|
+
end
|
|
728
|
+
```
|
|
729
|
+
|
|
730
|
+
To close conversations where the customer went quiet, give the wait a `timeout: 1.day` and rescue
|
|
731
|
+
`ActiveDurable::StepFailed`. To test it turn by turn: `Testing.drain(id, expire_waits: false)`, then `Durable.signal`,
|
|
732
|
+
then drain again.
|
|
733
|
+
|
|
584
734
|
## Dashboard
|
|
585
735
|
|
|
586
736
|
```ruby
|
|
@@ -654,7 +804,8 @@ point, kills it right there, and finishes it with a new worker. It is the fastes
|
|
|
654
804
|
idempotent.
|
|
655
805
|
|
|
656
806
|
`ActiveDurable::Testing.drain(id, signals: { name => payload })` runs an execution synchronously, fast-forwarding
|
|
657
|
-
sleeps, retries and expired leases
|
|
807
|
+
sleeps, retries and expired leases, and follows `continue_as_new`. It fast-forwards wait timeouts too; to test a
|
|
808
|
+
conversation turn by turn, pass `expire_waits: false`, send a signal and drain again.
|
|
658
809
|
|
|
659
810
|
## Observability
|
|
660
811
|
|
|
@@ -760,6 +911,9 @@ while sagas were halfway through. All of them settled, the refunds and hooks ran
|
|
|
760
911
|
OpenTelemetry: keep secrets and personal data out of them.
|
|
761
912
|
- Ids cannot be blank or contain `/`. Ids and step names compare exactly, also on MySQL.
|
|
762
913
|
- `flow.transaction` is atomic only when the notebook lives in the same database as your data.
|
|
914
|
+
- Step results, inputs and signals come back with their hash keys sorted, on the first run and on every replay, since
|
|
915
|
+
PostgreSQL and MySQL do not keep the order you wrote them in.
|
|
916
|
+
- A replay reads the whole notebook. For sagas that grow without end, such as a chat, use `flow.continue_as_new`.
|
|
763
917
|
|
|
764
918
|
## Development
|
|
765
919
|
|
|
@@ -8,14 +8,14 @@ module ActiveDurable
|
|
|
8
8
|
|
|
9
9
|
STATUS_TONES = {
|
|
10
10
|
"completed" => "good", "compensated" => "undo", "blocked" => "bad", "superseded" => "muted",
|
|
11
|
-
"running" => "busy", "pending" => "busy", "sleeping" => "wait", "waiting" => "wait",
|
|
11
|
+
"continued" => "muted", "running" => "busy", "pending" => "busy", "sleeping" => "wait", "waiting" => "wait",
|
|
12
12
|
"retrying" => "wait", "failed" => "bad", "undone" => "undo"
|
|
13
13
|
}.freeze
|
|
14
14
|
|
|
15
15
|
STATUS_WORDS = {
|
|
16
16
|
"pending" => "Queued", "running" => "Running", "sleeping" => "Sleeping", "waiting" => "Waiting",
|
|
17
17
|
"completed" => "Completed", "compensated" => "Undone", "blocked" => "Needs a person",
|
|
18
|
-
"superseded" => "Superseded"
|
|
18
|
+
"superseded" => "Superseded", "continued" => "Continued"
|
|
19
19
|
}.freeze
|
|
20
20
|
|
|
21
21
|
def status_pill(status)
|
|
@@ -17,6 +17,9 @@
|
|
|
17
17
|
<% if @execution.forked_from %>
|
|
18
18
|
Rerun of <%= execution_link(@execution.forked_from, class: "mono") %>.
|
|
19
19
|
<% end %>
|
|
20
|
+
<% if @execution.continued? %>
|
|
21
|
+
Continued as <%= execution_link(@execution.continued_as, class: "mono") %>.
|
|
22
|
+
<% end %>
|
|
20
23
|
</p>
|
|
21
24
|
</div>
|
|
22
25
|
<span class="badge"><%= ActiveDurable::DashboardHelper::STATUS_WORDS.fetch(@execution.status, @execution.status) %><%= ", undoing" if compensating_now?(@execution) %></span>
|
|
@@ -52,6 +52,21 @@ module ActiveDurable
|
|
|
52
52
|
NoMatchingPatternError LocalJumpError RegexpError EncodingError
|
|
53
53
|
].freeze
|
|
54
54
|
|
|
55
|
+
# Who the work is for (the tenant, the user, the request id), carried from where an execution starts to every
|
|
56
|
+
# run of it. An object with two methods:
|
|
57
|
+
#
|
|
58
|
+
# module TenantContext
|
|
59
|
+
# def self.capture = { "tenant_id" => Current.tenant_id } # when the execution starts
|
|
60
|
+
# def self.wrap(context, &run) = Current.set(tenant_id: context["tenant_id"], &run) # around every run
|
|
61
|
+
# end
|
|
62
|
+
# config.context = TenantContext
|
|
63
|
+
#
|
|
64
|
+
# capture returns plain JSON (or nil), stored with the execution; ActiveDurable.start(..., context: {...})
|
|
65
|
+
# gives it explicitly instead. wrap runs around the recipe, the undos, the flow.on hooks and each
|
|
66
|
+
# flow.parallel branch thread, and must call the block exactly once. Executions started without a context
|
|
67
|
+
# get {}. Needs the context column (rails generate active_durable:upgrade).
|
|
68
|
+
attr_accessor :context
|
|
69
|
+
|
|
55
70
|
attr_writer :logger
|
|
56
71
|
|
|
57
72
|
def initialize
|
|
@@ -67,6 +82,7 @@ module ActiveDurable
|
|
|
67
82
|
@clock = -> { Time.current }
|
|
68
83
|
@dashboard_authorize = nil
|
|
69
84
|
@code_errors = DEFAULT_CODE_ERRORS.dup
|
|
85
|
+
@context = nil
|
|
70
86
|
@logger = nil
|
|
71
87
|
end
|
|
72
88
|
|
|
@@ -9,14 +9,17 @@ module ActiveDurable
|
|
|
9
9
|
# wake-up time (a sleep or a retry) or waiting for a signal.
|
|
10
10
|
ACTIVE = %w[pending running sleeping waiting].freeze
|
|
11
11
|
# Statuses where nothing happens without a person: done, undone, blocked (needs {ActiveDurable.retry},
|
|
12
|
-
# {ActiveDurable.compensate} or a fix),
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
12
|
+
# {ActiveDurable.compensate} or a fix), replaced by a rerun, or carried on by another execution
|
|
13
|
+
# (flow.continue_as_new, see {#continued_as}).
|
|
14
|
+
TERMINAL = %w[completed compensated blocked superseded continued].freeze
|
|
15
|
+
# Statuses that never change again. Signals are refused (a continued execution passes them on), and
|
|
16
|
+
# ActiveDurable.prune may delete them (a continued one only together with the execution that carried it on).
|
|
17
|
+
FINISHED = %w[completed compensated superseded continued].freeze
|
|
16
18
|
# Every status.
|
|
17
19
|
STATUSES = (ACTIVE + TERMINAL).freeze
|
|
18
20
|
|
|
19
21
|
attribute :input, JSON_TYPE, default: -> { {} }
|
|
22
|
+
attribute :context, JSON_TYPE
|
|
20
23
|
attribute :output, JSON_TYPE
|
|
21
24
|
attribute :error, JSON_TYPE
|
|
22
25
|
|
|
@@ -34,6 +37,16 @@ module ActiveDurable
|
|
|
34
37
|
# the sweeper finds the pending row and enqueues it.
|
|
35
38
|
after_create_commit { ActiveDurable.enqueue(id) }
|
|
36
39
|
|
|
40
|
+
# @return [Boolean] whether flow.continue_as_new closed it; {#continued_as} is the id that carries on
|
|
41
|
+
def continued?
|
|
42
|
+
status == "continued"
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
# @api private
|
|
46
|
+
def self.continuations?
|
|
47
|
+
column_names.include?("continued_as")
|
|
48
|
+
end
|
|
49
|
+
|
|
37
50
|
# @return [Boolean] whether it will move on by itself
|
|
38
51
|
def active?
|
|
39
52
|
ACTIVE.include?(status)
|
data/lib/active_durable/flow.rb
CHANGED
|
@@ -199,36 +199,79 @@ module ActiveDurable
|
|
|
199
199
|
# Waits, without holding a worker, for {ActiveDurable.signal}(execution_id, name, payload). A signal sent
|
|
200
200
|
# before the saga gets here is kept.
|
|
201
201
|
#
|
|
202
|
-
#
|
|
202
|
+
# By default the signal has the step's name. With `signal:`, several waits share one signal name, like a
|
|
203
|
+
# mailbox: a conversation can wait for :message on every turn, and a message that arrives while the saga is
|
|
204
|
+
# busy waits there for the next turn. With `all: true`, the wait takes every pending signal of that name at
|
|
205
|
+
# once and returns their payloads in the order they arrived, so a burst of five messages is one turn, not five.
|
|
206
|
+
#
|
|
207
|
+
# @param name [Symbol, String] the step's name
|
|
203
208
|
# @param timeout [ActiveSupport::Duration, Numeric, nil] seconds; then the step fails and the saga is undone
|
|
204
|
-
# @
|
|
209
|
+
# @param signal [Symbol, String, nil] the signal to wait for, when it is not the step's name
|
|
210
|
+
# @param all [Boolean] take every pending signal and return an Array of payloads
|
|
211
|
+
# @return [Object] the signal's payload, or an Array of payloads with `all: true`
|
|
205
212
|
# @raise [StepFailed] when the timeout passes first
|
|
206
|
-
|
|
213
|
+
# @example A chat that waits for the customer on every turn
|
|
214
|
+
# messages = flow.wait_for("reply.#{turn}", signal: :message, all: true, timeout: 1.day)
|
|
215
|
+
def wait_for(name, timeout: nil, signal: nil, all: false)
|
|
207
216
|
name, position = visit!(name, "wait")
|
|
217
|
+
signal_name = signal.nil? ? name : signal.to_s
|
|
218
|
+
check_signal_name!(name, signal_name)
|
|
208
219
|
entry = @notebook[name]
|
|
209
220
|
return entry.result.deep_dup if entry&.completed?
|
|
210
221
|
raise StepFailed.new(name, entry.error&.fetch("message", nil)) if entry&.failed?
|
|
211
222
|
raise StopForward, name if compensating?
|
|
212
223
|
|
|
213
|
-
|
|
214
|
-
return
|
|
224
|
+
signals = pending_signals(signal_name, all)
|
|
225
|
+
return consume_signals(signals, name, position, all).deep_dup if signals.any?
|
|
215
226
|
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
@notebook.wait!(name, kind: "wait", position: position, wake_at: deadline)
|
|
219
|
-
@runner.suspend!(deadline, "waiting")
|
|
220
|
-
end
|
|
227
|
+
keep_waiting(name, position, entry, timeout, signal_name)
|
|
228
|
+
end
|
|
221
229
|
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
230
|
+
# Closes this execution and starts a new one of the same recipe (its latest version) with `input`, the same
|
|
231
|
+
# context and the signals still waiting to be read. It keeps a long-lived saga, such as a conversation, from
|
|
232
|
+
# replaying an ever longer notebook: carry over a summary of what matters, not the whole history.
|
|
233
|
+
#
|
|
234
|
+
# The new execution's id is this one's plus "~2" (then "~3"...), so its steps get new tickets.
|
|
235
|
+
# {ActiveDurable.signal} sent to any id of the chain reaches the newest one; {ActiveDurable.find} with
|
|
236
|
+
# `latest: true` returns it. Call it in the recipe, not inside a step, and compute `input` from step results
|
|
237
|
+
# so a replay computes the same. Hooks of this execution do not run.
|
|
238
|
+
#
|
|
239
|
+
# @param input [Hash] the input of the new execution
|
|
240
|
+
# @raise [InvalidRecipe] inside a step or a parallel branch, or while undos are pending: those steps
|
|
241
|
+
# could never be undone. Call it before them, or after flow.pivot.
|
|
242
|
+
# @example A conversation that starts a fresh notebook every 20 turns
|
|
243
|
+
# flow.continue_as_new(summary: summary, turn: turn) if turn % 20 == 19
|
|
244
|
+
def continue_as_new(**input)
|
|
245
|
+
raise InvalidRecipe, "flow.continue_as_new must be called in the recipe, not inside a step" if inside_step?
|
|
246
|
+
raise blocked_by if blocked_by
|
|
247
|
+
raise StopForward, "continue_as_new" if compensating?
|
|
248
|
+
|
|
249
|
+
if @undo_stack.any? && !@pivoted
|
|
250
|
+
names = @undo_stack.map { |entry| ":#{entry.name}" }.join(", ")
|
|
251
|
+
raise InvalidRecipe, "flow.continue_as_new would leave #{names} without their undo. Continue before " \
|
|
252
|
+
"steps with undo:, or after flow.pivot."
|
|
227
253
|
end
|
|
228
254
|
|
|
229
|
-
@runner.
|
|
255
|
+
@runner.continue_as_new!(Serializer.normalize(input, "the input of continue_as_new"))
|
|
256
|
+
end
|
|
257
|
+
|
|
258
|
+
# @api private
|
|
259
|
+
def inside_step?
|
|
260
|
+
Thread.current[INSIDE_STEP] == true
|
|
261
|
+
end
|
|
262
|
+
|
|
263
|
+
# @api private
|
|
264
|
+
def inside_step
|
|
265
|
+
previous = Thread.current[INSIDE_STEP]
|
|
266
|
+
Thread.current[INSIDE_STEP] = true
|
|
267
|
+
yield
|
|
268
|
+
ensure
|
|
269
|
+
Thread.current[INSIDE_STEP] = previous
|
|
230
270
|
end
|
|
231
271
|
|
|
272
|
+
# @api private
|
|
273
|
+
INSIDE_STEP = :active_durable_inside_step
|
|
274
|
+
|
|
232
275
|
# Called when the recipe returns: every step the notebook knows about must have been reached.
|
|
233
276
|
#
|
|
234
277
|
# @api private
|
|
@@ -292,9 +335,11 @@ module ActiveDurable
|
|
|
292
335
|
ActiveDurable.crash_point(:before_step, name)
|
|
293
336
|
result = ActiveDurable.instrument("step", execution_id: execution_id, step: name, kind: kind) do
|
|
294
337
|
if kind == "transaction"
|
|
295
|
-
@notebook.transaction(records: name)
|
|
338
|
+
@notebook.transaction(records: name) do
|
|
339
|
+
record_result(name, kind, position, inside_step { block.call(ticket) })
|
|
340
|
+
end
|
|
296
341
|
else
|
|
297
|
-
record_result(name, kind, position, block.call(ticket))
|
|
342
|
+
record_result(name, kind, position, inside_step { block.call(ticket) })
|
|
298
343
|
end
|
|
299
344
|
end
|
|
300
345
|
ActiveDurable.crash_point(:after_record, name)
|
|
@@ -411,15 +456,49 @@ module ActiveDurable
|
|
|
411
456
|
"recipe reached :#{name} (#{kind}). #{RECIPE_CHANGED_HINT}"
|
|
412
457
|
end
|
|
413
458
|
|
|
414
|
-
def
|
|
415
|
-
|
|
459
|
+
def pending_signals(signal_name, all)
|
|
460
|
+
signals = SignalRecord.pending_for(execution_id, signal_name)
|
|
461
|
+
all ? signals.to_a : signals.limit(1).to_a
|
|
462
|
+
end
|
|
463
|
+
|
|
464
|
+
# Suspends the execution until a signal arrives, or fails the wait once its deadline has passed.
|
|
465
|
+
def keep_waiting(name, position, entry, timeout, signal_name)
|
|
466
|
+
unless entry&.waiting? # first time here, or retried after a timeout: wait again, with a new deadline
|
|
467
|
+
deadline = timeout && (now + timeout)
|
|
468
|
+
@notebook.wait!(name, kind: "wait", position: position, wake_at: deadline,
|
|
469
|
+
signal_name: (signal_name unless signal_name == name))
|
|
470
|
+
@runner.suspend!(deadline, "waiting")
|
|
471
|
+
end
|
|
472
|
+
|
|
473
|
+
if entry.wake_at && now >= entry.wake_at
|
|
474
|
+
error = WaitTimeout.new("no :#{signal_name} signal arrived before #{entry.wake_at.utc.iso8601}")
|
|
475
|
+
@notebook.fail!(name, kind: "wait", position: position, attempts: 1,
|
|
476
|
+
error: ActiveDurable.dump_error(error, step: name))
|
|
477
|
+
raise StepFailed.new(name, error)
|
|
478
|
+
end
|
|
479
|
+
|
|
480
|
+
@runner.suspend!(entry.wake_at, "waiting")
|
|
481
|
+
end
|
|
482
|
+
|
|
483
|
+
# Takes the signals and completes the wait in one transaction: either both happen or neither does.
|
|
484
|
+
def consume_signals(signals, name, position, all)
|
|
485
|
+
result = all ? signals.map(&:payload) : signals.first.payload
|
|
486
|
+
ids = signals.map(&:id)
|
|
416
487
|
@notebook.transaction do
|
|
417
|
-
taken = SignalRecord.where(id:
|
|
418
|
-
raise LeaseLost, "
|
|
488
|
+
taken = SignalRecord.where(id: ids, consumed_at: nil).update_all(consumed_at: now)
|
|
489
|
+
raise LeaseLost, "a signal for #{execution_id} was already consumed" if taken != ids.size
|
|
419
490
|
|
|
420
|
-
@notebook.complete!(name, kind: "wait", position: position, result:
|
|
491
|
+
@notebook.complete!(name, kind: "wait", position: position, result: result)
|
|
421
492
|
end
|
|
422
|
-
|
|
493
|
+
result
|
|
494
|
+
end
|
|
495
|
+
|
|
496
|
+
def check_signal_name!(name, signal_name)
|
|
497
|
+
raise InvalidRecipe, "flow.wait_for :#{name} needs a signal name that is not blank" if signal_name.empty?
|
|
498
|
+
return if signal_name == name || Step.signal_names?
|
|
499
|
+
|
|
500
|
+
raise InvalidRecipe, "flow.wait_for :#{name}, signal: :#{signal_name} needs the signal_name column. Run " \
|
|
501
|
+
"rails generate active_durable:upgrade and rails db:migrate."
|
|
423
502
|
end
|
|
424
503
|
end
|
|
425
504
|
end
|
|
@@ -99,7 +99,8 @@ module ActiveDurable
|
|
|
99
99
|
wakes = outcomes.values.filter_map { |state, value| value if state == :retry }
|
|
100
100
|
@runner.suspend!(wakes.min, "sleeping") if wakes.any?
|
|
101
101
|
|
|
102
|
-
|
|
102
|
+
# By branch name in sorted order, which is also how a replay reads it back from the notebook.
|
|
103
|
+
results = Serializer.canonical(branches.to_h { |branch| [branch.name, outcomes.fetch(branch.full_name).last] })
|
|
103
104
|
@notebook.complete!(name, kind: "parallel", position: position, result: results)
|
|
104
105
|
remember_branches(name, branches)
|
|
105
106
|
results.deep_dup
|
|
@@ -155,9 +156,12 @@ module ActiveDurable
|
|
|
155
156
|
end
|
|
156
157
|
|
|
157
158
|
def in_branch_context(wrappers, &block)
|
|
159
|
+
context = @runner.execution.context
|
|
158
160
|
run = lambda do
|
|
159
161
|
Record.connection_pool.with_connection do
|
|
160
|
-
|
|
162
|
+
ActiveDurable.with_context(context) do
|
|
163
|
+
wrappers.reverse.inject(block) { |inner, (wrapper, captured)| -> { wrapper.wrap(captured, &inner) } }.call
|
|
164
|
+
end
|
|
161
165
|
end
|
|
162
166
|
end
|
|
163
167
|
if defined?(Rails) && Rails.respond_to?(:application) && Rails.application
|
|
@@ -175,10 +179,10 @@ module ActiveDurable
|
|
|
175
179
|
kind: branch.kind) do
|
|
176
180
|
if branch.kind == "transaction"
|
|
177
181
|
@notebook.transaction(records: branch.full_name) do
|
|
178
|
-
record_result(branch.full_name, branch.kind, nil, branch.block.call(ticket))
|
|
182
|
+
record_result(branch.full_name, branch.kind, nil, inside_step { branch.block.call(ticket) })
|
|
179
183
|
end
|
|
180
184
|
else
|
|
181
|
-
record_result(branch.full_name, branch.kind, nil, branch.block.call(ticket))
|
|
185
|
+
record_result(branch.full_name, branch.kind, nil, inside_step { branch.block.call(ticket) })
|
|
182
186
|
end
|
|
183
187
|
end
|
|
184
188
|
ActiveDurable.crash_point(:after_record, branch.full_name)
|
|
@@ -53,8 +53,10 @@ module ActiveDurable
|
|
|
53
53
|
write!(name, kind: kind, position: position, status: "blocked", attempts: attempts, error: error, wake_at: nil)
|
|
54
54
|
end
|
|
55
55
|
|
|
56
|
-
def wait!(name, kind:, position:, wake_at:)
|
|
57
|
-
|
|
56
|
+
def wait!(name, kind:, position:, wake_at:, signal_name: nil)
|
|
57
|
+
attributes = { kind: kind, position: position, status: "waiting", wake_at: wake_at }
|
|
58
|
+
attributes[:signal_name] = signal_name if Step.signal_names?
|
|
59
|
+
write!(name, **attributes)
|
|
58
60
|
end
|
|
59
61
|
|
|
60
62
|
# Runs the block in a database transaction and remembers the notebook writes made inside it only once it
|
|
@@ -58,7 +58,7 @@ module ActiveDurable
|
|
|
58
58
|
start = rerun_start(original, from)
|
|
59
59
|
execution = Execution.create!(id: rerun_id(original), recipe: original.recipe,
|
|
60
60
|
recipe_version: original.recipe_version, input: original.input,
|
|
61
|
-
status: "pending", forked_from: original.id)
|
|
61
|
+
context: original.context, status: "pending", forked_from: original.id)
|
|
62
62
|
copy_steps(original, execution, before: start.position)
|
|
63
63
|
original.update_columns(status: "superseded", updated_at: ActiveDurable.now) if original.status == "blocked"
|
|
64
64
|
end
|
|
@@ -78,6 +78,7 @@ module ActiveDurable
|
|
|
78
78
|
def with_idle_execution(execution_id, allowed:)
|
|
79
79
|
execution = Record.transaction do
|
|
80
80
|
record = Execution.lock.find(execution_id)
|
|
81
|
+
raise Error, "#{record.id} continued as #{record.continued_as}; use that one" if record.continued?
|
|
81
82
|
unless allowed.include?(record.status)
|
|
82
83
|
raise Error, "#{record.id} is #{record.status}; this works on #{allowed.join(", ")} executions"
|
|
83
84
|
end
|
|
@@ -98,6 +99,7 @@ module ActiveDurable
|
|
|
98
99
|
end
|
|
99
100
|
|
|
100
101
|
def check_rerunnable!(original)
|
|
102
|
+
raise Error, "#{original.id} continued as #{original.continued_as}; rerun that one" if original.continued?
|
|
101
103
|
return if original.status == "completed"
|
|
102
104
|
return if original.status == "blocked" && !original.compensating
|
|
103
105
|
|
|
@@ -2,7 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
module ActiveDurable
|
|
4
4
|
# Deletes finished executions (completed, compensated, superseded) older than a cutoff, with their notebook and
|
|
5
|
-
# their signals. Active and blocked executions are never touched: they still need a worker or a person.
|
|
5
|
+
# their signals. Active and blocked executions are never touched: they still need a worker or a person. A
|
|
6
|
+
# continued execution goes together with the one that carried it on, never before: its id may still be in use.
|
|
6
7
|
#
|
|
7
8
|
# @api private
|
|
8
9
|
module Pruner
|
|
@@ -16,9 +17,12 @@ module ActiveDurable
|
|
|
16
17
|
cutoff = now - older_than.to_f
|
|
17
18
|
deleted = 0
|
|
18
19
|
loop do
|
|
19
|
-
ids = Execution.where(status: Execution::FINISHED).where("updated_at < ?", cutoff)
|
|
20
|
+
ids = Execution.where(status: Execution::FINISHED - ["continued"]).where("updated_at < ?", cutoff)
|
|
21
|
+
.limit(batch_size).pluck(:id)
|
|
20
22
|
break if ids.empty?
|
|
21
23
|
|
|
24
|
+
ids += predecessors(ids)
|
|
25
|
+
|
|
22
26
|
Record.transaction do
|
|
23
27
|
Step.where(execution_id: ids).delete_all
|
|
24
28
|
SignalRecord.where(execution_id: ids).delete_all
|
|
@@ -28,5 +32,18 @@ module ActiveDurable
|
|
|
28
32
|
end
|
|
29
33
|
deleted
|
|
30
34
|
end
|
|
35
|
+
|
|
36
|
+
# The continued executions that led to these ones, all the way back.
|
|
37
|
+
def predecessors(ids)
|
|
38
|
+
return [] unless Execution.continuations?
|
|
39
|
+
|
|
40
|
+
found = []
|
|
41
|
+
layer = ids
|
|
42
|
+
until layer.empty?
|
|
43
|
+
layer = Execution.where(continued_as: layer).pluck(:id)
|
|
44
|
+
found.concat(layer)
|
|
45
|
+
end
|
|
46
|
+
found
|
|
47
|
+
end
|
|
31
48
|
end
|
|
32
49
|
end
|
|
@@ -8,8 +8,16 @@ module ActiveDurable
|
|
|
8
8
|
class Record < ActiveRecord::Base
|
|
9
9
|
self.abstract_class = true
|
|
10
10
|
|
|
11
|
+
# JSON that reads back with its hash keys sorted, whatever order the database stored them in (see
|
|
12
|
+
# Serializer.canonical).
|
|
13
|
+
class CanonicalJson < ActiveRecord::Type::Json
|
|
14
|
+
def deserialize(value)
|
|
15
|
+
Serializer.canonical(super)
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
|
|
11
19
|
# A JSON type object (not the :json symbol): resolving a symbol needs a database connection,
|
|
12
20
|
# and these classes may load before the app has configured one.
|
|
13
|
-
JSON_TYPE =
|
|
21
|
+
JSON_TYPE = CanonicalJson.new
|
|
14
22
|
end
|
|
15
23
|
end
|
|
@@ -8,7 +8,7 @@ module ActiveDurable
|
|
|
8
8
|
class Runner
|
|
9
9
|
SUSPEND = :active_durable_suspend
|
|
10
10
|
|
|
11
|
-
# Returns :completed, :compensated, :blocked, :sleeping, :waiting, :busy or :lease_lost.
|
|
11
|
+
# Returns :completed, :compensated, :continued, :blocked, :sleeping, :waiting, :busy or :lease_lost.
|
|
12
12
|
def self.run(execution_id)
|
|
13
13
|
lease = Lease.claim(execution_id)
|
|
14
14
|
return :busy unless lease
|
|
@@ -25,8 +25,10 @@ module ActiveDurable
|
|
|
25
25
|
end
|
|
26
26
|
|
|
27
27
|
def run
|
|
28
|
-
ActiveDurable.
|
|
29
|
-
|
|
28
|
+
ActiveDurable.with_context(execution.context) do
|
|
29
|
+
ActiveDurable.instrument("execution", execution_id: execution.id, recipe: execution.recipe) do
|
|
30
|
+
catch(SUSPEND) { run_recipe }
|
|
31
|
+
end
|
|
30
32
|
end
|
|
31
33
|
rescue LeaseLost => e
|
|
32
34
|
ActiveDurable.config.logger.warn("[ActiveDurable] #{e.message}")
|
|
@@ -44,6 +46,29 @@ module ActiveDurable
|
|
|
44
46
|
throw SUSPEND, status.to_sym
|
|
45
47
|
end
|
|
46
48
|
|
|
49
|
+
# flow.continue_as_new: in one transaction, closes this execution, creates the one that carries on and moves the
|
|
50
|
+
# pending signals to it. Closing it first takes its row lock, so ActiveDurable.signal (which locks the row too)
|
|
51
|
+
# either commits before, and its signal is moved, or waits and then follows continued_as.
|
|
52
|
+
def continue_as_new!(input)
|
|
53
|
+
unless Execution.continuations?
|
|
54
|
+
raise InvalidRecipe, "flow.continue_as_new needs the continued_as column. Run rails generate " \
|
|
55
|
+
"active_durable:upgrade and rails db:migrate."
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
successor_id = ActiveDurable.next_in_chain(execution.id)
|
|
59
|
+
recipe = ActiveDurable.registry.latest(execution.recipe)
|
|
60
|
+
Record.transaction do
|
|
61
|
+
lease.release!(status: "continued", continued_as: successor_id, wake_at: nil)
|
|
62
|
+
ActiveDurable.crash_point(:continue_as_new, successor_id)
|
|
63
|
+
Execution.create!(id: successor_id, recipe: recipe.name, recipe_version: recipe.version, input: input,
|
|
64
|
+
context: execution.context, status: "pending")
|
|
65
|
+
SignalRecord.pending.where(execution_id: execution.id).update_all(execution_id: successor_id)
|
|
66
|
+
end
|
|
67
|
+
ActiveDurable.instrument("continued", execution_id: execution.id, recipe: execution.recipe,
|
|
68
|
+
continued_as: successor_id)
|
|
69
|
+
throw SUSPEND, :continued
|
|
70
|
+
end
|
|
71
|
+
|
|
47
72
|
private
|
|
48
73
|
|
|
49
74
|
def run_recipe
|
|
@@ -2,8 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
module ActiveDurable
|
|
4
4
|
# Step results, inputs and signal payloads live in the notebook as JSON. This module turns a value
|
|
5
|
-
# into exactly what a later replay will read back (string keys, no symbols, UTF-8 strings), so the
|
|
6
|
-
# and a replay behave the same. Anything that is not plain JSON, or that some database cannot store (a NUL
|
|
5
|
+
# into exactly what a later replay will read back (string keys, no symbols, UTF-8 strings, keys sorted), so the
|
|
6
|
+
# first run and a replay behave the same. Anything that is not plain JSON, or that some database cannot store (a NUL
|
|
7
7
|
# character, bytes that are not UTF-8), is rejected with a message that says where.
|
|
8
8
|
#
|
|
9
9
|
# @api private
|
|
@@ -27,12 +27,25 @@ module ActiveDurable
|
|
|
27
27
|
end
|
|
28
28
|
|
|
29
29
|
def normalize_hash(hash, path)
|
|
30
|
-
hash.each_with_object({}) do |(key, item),
|
|
30
|
+
out = hash.each_with_object({}) do |(key, item), normalized|
|
|
31
31
|
key = key.to_s if key.is_a?(Symbol)
|
|
32
32
|
raise NotSerializable, "#{path} has a #{key.class} key; use strings or symbols" unless key.is_a?(String)
|
|
33
33
|
|
|
34
34
|
key = utf8(key, "a key of #{path}")
|
|
35
|
-
|
|
35
|
+
normalized[key] = normalize(item, "#{path}[#{key.inspect}]")
|
|
36
|
+
end
|
|
37
|
+
out.sort.to_h
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
# Databases do not keep the order of hash keys: PostgreSQL's jsonb and MySQL's JSON put shorter keys first.
|
|
41
|
+
# Sorting them, when writing and when reading back, gives a replay the very same hash as the first run, down
|
|
42
|
+
# to the order of its keys. That matters when a result is sent somewhere that compares bytes, such as an AI
|
|
43
|
+
# provider that checks the conversation it is sent again.
|
|
44
|
+
def canonical(value)
|
|
45
|
+
case value
|
|
46
|
+
when Hash then value.sort.to_h { |key, item| [key, canonical(item)] }
|
|
47
|
+
when Array then value.map { |item| canonical(item) }
|
|
48
|
+
else value
|
|
36
49
|
end
|
|
37
50
|
end
|
|
38
51
|
|
|
@@ -17,17 +17,22 @@ module ActiveDurable
|
|
|
17
17
|
# for a later flow.wait_for) must not wake the execution up: it would only go back to waiting, forever.
|
|
18
18
|
scope :awaited, lambda {
|
|
19
19
|
steps = Step.arel_table
|
|
20
|
+
awaits = if Step.signal_names?
|
|
21
|
+
Arel::Nodes::NamedFunction.new("COALESCE", [steps[:signal_name], steps[:name]])
|
|
22
|
+
else
|
|
23
|
+
steps[:name]
|
|
24
|
+
end
|
|
20
25
|
waiting = steps.project(Arel.sql("1"))
|
|
21
26
|
.where(steps[:execution_id].eq(arel_table[:execution_id])
|
|
22
|
-
.and(
|
|
27
|
+
.and(awaits.eq(arel_table[:name]))
|
|
23
28
|
.and(steps[:status].eq("waiting")))
|
|
24
29
|
pending.where(waiting.exists)
|
|
25
30
|
}
|
|
26
31
|
|
|
27
32
|
after_create_commit { ActiveDurable.enqueue(execution_id) }
|
|
28
33
|
|
|
29
|
-
def self.
|
|
30
|
-
pending.where(execution_id: execution_id, name: name.to_s).order(:id)
|
|
34
|
+
def self.pending_for(execution_id, name)
|
|
35
|
+
pending.where(execution_id: execution_id, name: name.to_s).order(:id)
|
|
31
36
|
end
|
|
32
37
|
end
|
|
33
38
|
end
|
data/lib/active_durable/step.rb
CHANGED
|
@@ -37,6 +37,16 @@ module ActiveDurable
|
|
|
37
37
|
status == "waiting"
|
|
38
38
|
end
|
|
39
39
|
|
|
40
|
+
# The name of the signal a waiting step is waiting for.
|
|
41
|
+
def awaits
|
|
42
|
+
(self.class.signal_names? && signal_name) || name
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
# @api private
|
|
46
|
+
def self.signal_names?
|
|
47
|
+
column_names.include?("signal_name")
|
|
48
|
+
end
|
|
49
|
+
|
|
40
50
|
def undo?
|
|
41
51
|
kind == "undo"
|
|
42
52
|
end
|
|
@@ -23,16 +23,21 @@ module ActiveDurable
|
|
|
23
23
|
end
|
|
24
24
|
|
|
25
25
|
# Runs an execution synchronously until it finishes, fast-forwarding sleeps, retries and expired
|
|
26
|
-
# leases
|
|
27
|
-
#
|
|
28
|
-
|
|
26
|
+
# leases, and following flow.continue_as_new to the execution that carries on. Signals given as
|
|
27
|
+
# { name => payload } are delivered, once each, when the saga waits for them. Returns the (last) execution; it
|
|
28
|
+
# stays "waiting" if it waits for a signal you did not provide.
|
|
29
|
+
#
|
|
30
|
+
# A wait with a timeout is fast-forwarded to it too, so the saga goes on as if no signal came. Pass
|
|
31
|
+
# `expire_waits: false` to stop at the wait instead, send signals yourself and drain again: that is how to
|
|
32
|
+
# test a conversation turn by turn.
|
|
33
|
+
def drain(execution_id, signals: {}, max_runs: 500, expire_waits: true)
|
|
29
34
|
signals = signals.transform_keys(&:to_s)
|
|
30
35
|
max_runs.times do
|
|
31
|
-
execution =
|
|
36
|
+
execution = ActiveDurable.find(execution_id, latest: true)
|
|
32
37
|
return execution if execution.terminal?
|
|
33
|
-
return execution unless ready_for_next_run?(execution, signals)
|
|
38
|
+
return execution unless ready_for_next_run?(execution, signals, expire_waits: expire_waits)
|
|
34
39
|
|
|
35
|
-
Runner.run(
|
|
40
|
+
Runner.run(execution.id)
|
|
36
41
|
end
|
|
37
42
|
raise Error, "execution #{execution_id} did not settle after #{max_runs} runs"
|
|
38
43
|
end
|
|
@@ -96,12 +101,12 @@ module ActiveDurable
|
|
|
96
101
|
# Decides what has to happen before the next run. Returns false when nothing can move the execution.
|
|
97
102
|
#
|
|
98
103
|
# @api private
|
|
99
|
-
def ready_for_next_run?(execution, signals)
|
|
104
|
+
def ready_for_next_run?(execution, signals, expire_waits: true)
|
|
100
105
|
now = ActiveDurable.now
|
|
101
106
|
if execution.locked_until && execution.locked_until > now
|
|
102
107
|
travel(execution.locked_until - now + 0.001) # a crashed worker still holds the lease
|
|
103
108
|
elsif execution.status == "waiting"
|
|
104
|
-
return waiting_can_move?(execution, signals, now)
|
|
109
|
+
return waiting_can_move?(execution, signals, now, expire_waits: expire_waits)
|
|
105
110
|
elsif execution.wake_at && execution.wake_at > now
|
|
106
111
|
travel(execution.wake_at - now + 0.001)
|
|
107
112
|
end
|
|
@@ -109,13 +114,13 @@ module ActiveDurable
|
|
|
109
114
|
end
|
|
110
115
|
|
|
111
116
|
# @api private
|
|
112
|
-
def waiting_can_move?(execution, signals, now)
|
|
117
|
+
def waiting_can_move?(execution, signals, now, expire_waits: true)
|
|
113
118
|
return true if SignalRecord.awaited.exists?(execution_id: execution.id)
|
|
114
119
|
|
|
115
120
|
waiting = execution.steps.find_by(status: "waiting")
|
|
116
|
-
if waiting && signals.key?(waiting.
|
|
117
|
-
ActiveDurable.signal(execution.id, waiting.
|
|
118
|
-
elsif execution.wake_at
|
|
121
|
+
if waiting && signals.key?(waiting.awaits)
|
|
122
|
+
ActiveDurable.signal(execution.id, waiting.awaits, signals.delete(waiting.awaits))
|
|
123
|
+
elsif execution.wake_at && expire_waits
|
|
119
124
|
travel(execution.wake_at - now + 0.001) if execution.wake_at > now
|
|
120
125
|
else
|
|
121
126
|
return false
|
data/lib/active_durable.rb
CHANGED
|
@@ -124,10 +124,12 @@ module ActiveDurable
|
|
|
124
124
|
# @raise [NotSerializable] if the input cannot be stored as JSON
|
|
125
125
|
# @example
|
|
126
126
|
# Durable.start(:checkout, id: "checkout-#{order.id}", order_id: order.id)
|
|
127
|
-
def start(name, id: nil, **input)
|
|
127
|
+
def start(name, id: nil, context: nil, **input)
|
|
128
128
|
recipe = registry.latest(name)
|
|
129
129
|
attributes = { recipe: recipe.name, recipe_version: recipe.version, status: "pending",
|
|
130
130
|
input: Serializer.normalize(input, "input") }
|
|
131
|
+
captured = capture_context(context)
|
|
132
|
+
attributes[:context] = captured unless captured.nil?
|
|
131
133
|
return Execution.create!(attributes.merge(id: "#{recipe.name}-#{SecureRandom.uuid}")) if id.nil?
|
|
132
134
|
|
|
133
135
|
id = check_id!(id)
|
|
@@ -143,6 +145,37 @@ module ActiveDurable
|
|
|
143
145
|
raise ArgumentError, "execution #{id} already exists for recipe :#{execution.recipe}"
|
|
144
146
|
end
|
|
145
147
|
|
|
148
|
+
# The context to store with a new execution: the one given to start, or config.context.capture.
|
|
149
|
+
#
|
|
150
|
+
# @api private
|
|
151
|
+
def capture_context(explicit)
|
|
152
|
+
value = explicit.nil? ? config.context&.capture : explicit
|
|
153
|
+
return nil if value.nil?
|
|
154
|
+
|
|
155
|
+
unless Execution.column_names.include?("context")
|
|
156
|
+
raise Error, "the durable_executions table has no context column. Run rails generate " \
|
|
157
|
+
"active_durable:upgrade and rails db:migrate."
|
|
158
|
+
end
|
|
159
|
+
Serializer.normalize(value, "the context")
|
|
160
|
+
end
|
|
161
|
+
|
|
162
|
+
# Runs the block inside config.context.wrap, with the context the execution started with.
|
|
163
|
+
#
|
|
164
|
+
# @api private
|
|
165
|
+
def with_context(context, &block)
|
|
166
|
+
wrapper = config.context
|
|
167
|
+
return yield if wrapper.nil?
|
|
168
|
+
|
|
169
|
+
ran = false
|
|
170
|
+
result = wrapper.wrap(context || {}) do
|
|
171
|
+
ran = true
|
|
172
|
+
block.call
|
|
173
|
+
end
|
|
174
|
+
raise Error, "config.context.wrap must call its block" unless ran
|
|
175
|
+
|
|
176
|
+
result
|
|
177
|
+
end
|
|
178
|
+
|
|
146
179
|
# @api private
|
|
147
180
|
def check_id!(id)
|
|
148
181
|
id = Serializer.normalize(id.to_s, "the execution id")
|
|
@@ -167,18 +200,33 @@ module ActiveDurable
|
|
|
167
200
|
# @example In a webhook controller
|
|
168
201
|
# Durable.signal("loan-42", :kyc_done, verified: true)
|
|
169
202
|
def signal(execution_id, name, payload = nil)
|
|
170
|
-
|
|
171
|
-
|
|
203
|
+
payload = Serializer.normalize(payload, "signal payload")
|
|
204
|
+
Record.transaction do
|
|
205
|
+
# Locked, so a flow.continue_as_new committing right now cannot strand the signal on the closed execution.
|
|
206
|
+
execution = Execution.lock.find(execution_id)
|
|
207
|
+
execution = Execution.lock.find(execution.continued_as) while execution.continued?
|
|
208
|
+
raise Error, "execution #{execution_id} already finished (#{execution.status})" if execution.finished?
|
|
172
209
|
|
|
173
|
-
|
|
174
|
-
|
|
210
|
+
SignalRecord.create!(execution_id: execution.id, name: name.to_s, payload: payload)
|
|
211
|
+
end
|
|
175
212
|
end
|
|
176
213
|
|
|
177
214
|
# @param execution_id [String]
|
|
215
|
+
# @param latest [Boolean] follow flow.continue_as_new to the execution that carries on now
|
|
178
216
|
# @return [Execution]
|
|
179
217
|
# @raise [ActiveRecord::RecordNotFound]
|
|
180
|
-
def find(execution_id)
|
|
181
|
-
Execution.find(execution_id)
|
|
218
|
+
def find(execution_id, latest: false)
|
|
219
|
+
execution = Execution.find(execution_id)
|
|
220
|
+
execution = Execution.find(execution.continued_as) while latest && execution.continued?
|
|
221
|
+
execution
|
|
222
|
+
end
|
|
223
|
+
|
|
224
|
+
# "order-1" continues as "order-1~2", then "order-1~3".
|
|
225
|
+
#
|
|
226
|
+
# @api private
|
|
227
|
+
def next_in_chain(execution_id)
|
|
228
|
+
base, number = execution_id.match(/\A(.+)~(\d+)\z/)&.captures
|
|
229
|
+
base ? "#{base}~#{number.to_i + 1}" : "#{execution_id}~2"
|
|
182
230
|
end
|
|
183
231
|
|
|
184
232
|
# Resumes a blocked execution where it stopped: failed steps (or failed undos, if it was undoing) get a fresh
|
|
@@ -10,18 +10,21 @@ class CreateActiveDurableTables < ActiveRecord::Migration<%= migration_version %
|
|
|
10
10
|
t.string :status, null: false
|
|
11
11
|
t.boolean :compensating, null: false, default: false
|
|
12
12
|
t.column :input, json_type
|
|
13
|
+
t.column :context, json_type
|
|
13
14
|
t.column :output, json_type
|
|
14
15
|
t.column :error, json_type
|
|
15
16
|
t.datetime :wake_at, precision: 6
|
|
16
17
|
t.datetime :locked_until, precision: 6
|
|
17
18
|
t.string :lease_token
|
|
18
19
|
t.string :forked_from, **exact
|
|
20
|
+
t.string :continued_as, **exact
|
|
19
21
|
t.timestamps precision: 6
|
|
20
22
|
end
|
|
21
23
|
add_index :durable_executions, %i[status wake_at]
|
|
22
24
|
add_index :durable_executions, %i[status updated_at]
|
|
23
25
|
add_index :durable_executions, %i[recipe status]
|
|
24
26
|
add_index :durable_executions, :forked_from
|
|
27
|
+
add_index :durable_executions, :continued_as
|
|
25
28
|
|
|
26
29
|
# The notebook: one row per step (and per undo) of an execution.
|
|
27
30
|
create_table :durable_steps do |t|
|
|
@@ -31,6 +34,7 @@ class CreateActiveDurableTables < ActiveRecord::Migration<%= migration_version %
|
|
|
31
34
|
t.integer :position
|
|
32
35
|
t.string :status, null: false
|
|
33
36
|
t.integer :attempts, null: false, default: 0
|
|
37
|
+
t.string :signal_name, **exact
|
|
34
38
|
t.column :result, json_type
|
|
35
39
|
t.column :error, json_type
|
|
36
40
|
t.datetime :wake_at, precision: 6
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# ActiveDurable 0.8: the context an execution started with (config.context), given back to it on every run.
|
|
4
|
+
class AddActiveDurableContext < ActiveRecord::Migration<%= migration_version %>
|
|
5
|
+
def up
|
|
6
|
+
add_column :durable_executions, :context, json_type unless column_exists?(:durable_executions, :context)
|
|
7
|
+
end
|
|
8
|
+
|
|
9
|
+
def down
|
|
10
|
+
remove_column :durable_executions, :context if column_exists?(:durable_executions, :context)
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
private
|
|
14
|
+
|
|
15
|
+
# jsonb on PostgreSQL, json elsewhere, like the other JSON columns.
|
|
16
|
+
def json_type
|
|
17
|
+
connection.adapter_name.downcase.include?("postg") ? :jsonb : :json
|
|
18
|
+
end
|
|
19
|
+
end
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# ActiveDurable 0.8: flow.continue_as_new closes an execution and points it to the one that carries on.
|
|
4
|
+
class AddActiveDurableContinuedAs < ActiveRecord::Migration<%= migration_version %>
|
|
5
|
+
def up
|
|
6
|
+
add_column :durable_executions, :continued_as, :string, **exact unless column_exists?(:durable_executions, :continued_as)
|
|
7
|
+
add_index :durable_executions, :continued_as unless index_exists?(:durable_executions, :continued_as)
|
|
8
|
+
end
|
|
9
|
+
|
|
10
|
+
def down
|
|
11
|
+
remove_index :durable_executions, :continued_as if index_exists?(:durable_executions, :continued_as)
|
|
12
|
+
remove_column :durable_executions, :continued_as if column_exists?(:durable_executions, :continued_as)
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
private
|
|
16
|
+
|
|
17
|
+
# MySQL compares strings ignoring case and accents by default; ids must match exactly.
|
|
18
|
+
def exact
|
|
19
|
+
connection.adapter_name.match?(/mysql|trilogy/i) ? { collation: "utf8mb4_bin" } : {}
|
|
20
|
+
end
|
|
21
|
+
end
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# ActiveDurable 0.8: flow.wait_for(name, signal: ...) writes down which signal a step waits for, when it is not the
|
|
4
|
+
# step's own name.
|
|
5
|
+
class AddActiveDurableSignalName < ActiveRecord::Migration<%= migration_version %>
|
|
6
|
+
def up
|
|
7
|
+
return if column_exists?(:durable_steps, :signal_name)
|
|
8
|
+
|
|
9
|
+
add_column :durable_steps, :signal_name, :string, **exact
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
def down
|
|
13
|
+
remove_column :durable_steps, :signal_name if column_exists?(:durable_steps, :signal_name)
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
private
|
|
17
|
+
|
|
18
|
+
# MySQL compares strings ignoring case and accents by default; signal names must match exactly.
|
|
19
|
+
def exact
|
|
20
|
+
connection.adapter_name.match?(/mysql|trilogy/i) ? { collation: "utf8mb4_bin" } : {}
|
|
21
|
+
end
|
|
22
|
+
end
|
|
@@ -16,7 +16,10 @@ module ActiveDurable
|
|
|
16
16
|
desc "Adds the migrations that newer ActiveDurable versions need. Run it after updating the gem."
|
|
17
17
|
|
|
18
18
|
# In the order they were released. Never edit one that shipped: add a new one.
|
|
19
|
-
MIGRATIONS = %w[
|
|
19
|
+
MIGRATIONS = %w[
|
|
20
|
+
add_active_durable_prune_index make_active_durable_ids_case_sensitive add_active_durable_context
|
|
21
|
+
add_active_durable_signal_name add_active_durable_continued_as
|
|
22
|
+
].freeze
|
|
20
23
|
|
|
21
24
|
# @api private
|
|
22
25
|
def create_migration_files
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: active_durable
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.8.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- William Romero
|
|
@@ -120,7 +120,10 @@ files:
|
|
|
120
120
|
- lib/active_durable/version.rb
|
|
121
121
|
- lib/generators/active_durable/install/install_generator.rb
|
|
122
122
|
- lib/generators/active_durable/install/templates/create_active_durable_tables.rb.tt
|
|
123
|
+
- lib/generators/active_durable/upgrade/templates/add_active_durable_context.rb.tt
|
|
124
|
+
- lib/generators/active_durable/upgrade/templates/add_active_durable_continued_as.rb.tt
|
|
123
125
|
- lib/generators/active_durable/upgrade/templates/add_active_durable_prune_index.rb.tt
|
|
126
|
+
- lib/generators/active_durable/upgrade/templates/add_active_durable_signal_name.rb.tt
|
|
124
127
|
- lib/generators/active_durable/upgrade/templates/make_active_durable_ids_case_sensitive.rb.tt
|
|
125
128
|
- lib/generators/active_durable/upgrade/upgrade_generator.rb
|
|
126
129
|
- lib/tasks/active_durable.rake
|