change_requests 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +4 -0
- data/config/locales/en.yml +7 -0
- data/docs/05_execution_and_idempotency.md +129 -0
- data/docs/adr/0015-one-guard-object-per-transition.md +25 -6
- data/docs/adr/0016-commands-are-the-only-writers.md +28 -7
- data/docs/adr/0022-execution-in-three-transactions.md +77 -0
- data/docs/adr/0023-one-declaration-surface.md +68 -0
- data/docs/adr/0024-idempotence-is-required-not-declared.md +57 -0
- data/docs/adr/0025-verification-from-one-set-of-checks.md +62 -0
- data/docs/adr/0026-distinct-intent-distinct-command.md +59 -0
- data/docs/adr/0027-activejob-is-optional.md +55 -0
- data/docs/adr/0028-sweeps-are-rake-tasks.md +67 -0
- data/docs/adr/README.md +7 -0
- data/lib/change_requests/commands/cancel.rb +11 -1
- data/lib/change_requests/commands/cancel_undeclared.rb +46 -0
- data/lib/change_requests/commands/reap.rb +37 -0
- data/lib/change_requests/commands/settle_execution.rb +27 -3
- data/lib/change_requests/configuration.rb +35 -0
- data/lib/change_requests/engine.rb +7 -0
- data/lib/change_requests/execution/job.rb +27 -0
- data/lib/change_requests/execution/runner.rb +31 -2
- data/lib/change_requests/guards/base.rb +2 -0
- data/lib/change_requests/guards/cancel.rb +9 -1
- data/lib/change_requests/guards/reap.rb +59 -0
- data/lib/change_requests/maintenance.rb +48 -0
- data/lib/change_requests/models/request.rb +18 -0
- data/lib/change_requests/version.rb +1 -1
- data/lib/change_requests.rb +37 -1
- data/lib/tasks/change_requests.rake +39 -1
- metadata +14 -1
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
# ADR-0028: Sweep with rake tasks, and keep the destructive one off the schedule
|
|
2
|
+
|
|
3
|
+
- **Status:** Accepted
|
|
4
|
+
- **Date:** 2026-09-12
|
|
5
|
+
|
|
6
|
+
## Context
|
|
7
|
+
|
|
8
|
+
Three kinds of row do not resolve themselves. A request nobody acted on sits `pending` past its
|
|
9
|
+
deadline. A request claimed by a process that then died sits `executing`, which no command will move
|
|
10
|
+
([ADR-0022](0022-execution-in-three-transactions.md)). A request whose operation is no longer declared
|
|
11
|
+
can never run at all.
|
|
12
|
+
|
|
13
|
+
Each needs sweeping. The question is what does the sweeping, and how often — which the plan asserted
|
|
14
|
+
in three places without ever saying.
|
|
15
|
+
|
|
16
|
+
## Decision
|
|
17
|
+
|
|
18
|
+
`ChangeRequests::Maintenance` holds one method per sweep, each a query plus the command that owns its
|
|
19
|
+
transition — never a write of its own. The commands hold the locks and emit the events; the sweeps
|
|
20
|
+
decide which rows to hand them ([ADR-0016](0016-commands-are-the-only-writers.md)). Each returns the
|
|
21
|
+
number of rows it moved.
|
|
22
|
+
|
|
23
|
+
`lib/tasks/change_requests.rake` exposes one task per sweep, loaded by the engine's own `lib/tasks`
|
|
24
|
+
path so a host requires nothing. Nothing rescues: rake exits non-zero when a task raises, so cron's
|
|
25
|
+
mail-on-failure is the alarm, and a sweep that moved nothing is not an error.
|
|
26
|
+
|
|
27
|
+
**Two are documented as cron. The third is not.**
|
|
28
|
+
|
|
29
|
+
```cron
|
|
30
|
+
5 * * * * cd /app && bin/rails change_requests:expire_stale
|
|
31
|
+
6 * * * * cd /app && bin/rails change_requests:reap_stuck_executions
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
`change_requests:cancel_undeclared` is run by an operator who has looked. A missing declaration is as
|
|
35
|
+
likely to be a deploy accident — an initializer that did not load, a file renamed — as a deliberate
|
|
36
|
+
removal, and `canceled` is final. The gem's other responses to an undeclared operation are immediate
|
|
37
|
+
**and reversible**: every guard but `Comment`, `Cancel` and `Reap` refuses it, and it leaves inboxes
|
|
38
|
+
and badges at once. A scheduled sweep would turn a bad deploy into a table of permanently cancelled
|
|
39
|
+
requests within the hour, when reverting the deploy would have cost nothing.
|
|
40
|
+
|
|
41
|
+
Cron rather than a recurring job, because these are time-of-day work with no per-request trigger, and
|
|
42
|
+
because a host running headless has no job backend to schedule into. The interval is a recommendation:
|
|
43
|
+
`expires_at` and the reaper's `older_than` are the real deadlines, and a sweeper running late moves the
|
|
44
|
+
same rows, just later.
|
|
45
|
+
|
|
46
|
+
## Consequences
|
|
47
|
+
|
|
48
|
+
### Positive
|
|
49
|
+
|
|
50
|
+
- The sweeps are ordinary commands, so every row they move gets its lock, its event and its `System`
|
|
51
|
+
actor for free, and the reaper's transition is guarded like every other one.
|
|
52
|
+
- A host with no job backend can still run all three.
|
|
53
|
+
- The destructive one cannot fire on its own. Recovering from a failed initializer is reverting the
|
|
54
|
+
deploy, not restoring rows.
|
|
55
|
+
|
|
56
|
+
### Negative
|
|
57
|
+
|
|
58
|
+
- **Nothing runs by default.** A host that installs the gem and never reads `docs/05` accumulates
|
|
59
|
+
expired-but-`pending` requests and stuck `executing` rows indefinitely, and nothing warns them.
|
|
60
|
+
- Three tasks to schedule is three chances to schedule none.
|
|
61
|
+
- `cancel_undeclared` being manual means a stranded request is cleared by whoever notices, which in
|
|
62
|
+
practice is nobody until someone reads a list. §5.11 mitigates this by making such requests
|
|
63
|
+
cancelable by *any* actor, so clearing one does not wait on the rake task — but the bulk path is
|
|
64
|
+
still a decision somebody has to make.
|
|
65
|
+
- The reaper's default threshold is a guess about the host's slowest target, and the failure mode is
|
|
66
|
+
silent: an execution written off while it is still working leaves a `failed` row and a side effect
|
|
67
|
+
that lands afterwards.
|
data/docs/adr/README.md
CHANGED
|
@@ -33,3 +33,10 @@ than an edit.
|
|
|
33
33
|
| [0019](0019-separation-of-duties.md) | Refuse the requester by identity, and make only execution configurable | Accepted |
|
|
34
34
|
| [0020](0020-refusal-vocabulary-and-fallback.md) | Ship refusal reasons as a closed vocabulary that degrades to the symbol | Accepted |
|
|
35
35
|
| [0021](0021-serialise-commands-let-the-index-arbitrate.md) | Serialise commands with a row lock and let the unique index arbitrate | Accepted |
|
|
36
|
+
| [0022](0022-execution-in-three-transactions.md) | Split execution into three transactions, and commit the claim before the side effect | Accepted |
|
|
37
|
+
| [0023](0023-one-declaration-surface.md) | Declare a workflow one way | Accepted |
|
|
38
|
+
| [0024](0024-idempotence-is-required-not-declared.md) | Require idempotence of every target rather than declaring it per operation | Accepted |
|
|
39
|
+
| [0025](0025-verification-from-one-set-of-checks.md) | Verify declarations at boot, from the same checks the runtime reads | Accepted |
|
|
40
|
+
| [0026](0026-distinct-intent-distinct-command.md) | Give a distinct intent a distinct command class, not a flag | Accepted |
|
|
41
|
+
| [0027](0027-activejob-is-optional.md) | Make background execution a setting and ActiveJob an optional dependency | Accepted |
|
|
42
|
+
| [0028](0028-sweeps-are-rake-tasks.md) | Sweep with rake tasks, and keep the destructive one off the schedule | Accepted |
|
|
@@ -19,7 +19,7 @@ module ChangeRequests
|
|
|
19
19
|
|
|
20
20
|
# Emitted before the status changes, so the trail records the request as it was cancelled
|
|
21
21
|
# rather than as it ended up.
|
|
22
|
-
emit(
|
|
22
|
+
emit(event_kind, body: reason, metadata: metadata)
|
|
23
23
|
request.update!(status: "canceled")
|
|
24
24
|
|
|
25
25
|
request
|
|
@@ -27,6 +27,16 @@ module ChangeRequests
|
|
|
27
27
|
|
|
28
28
|
private
|
|
29
29
|
|
|
30
|
+
# The two seams CancelUndeclared overrides, and the whole of the difference between them:
|
|
31
|
+
# who cancelled decides which fact the timeline records (Q11, §5.11).
|
|
32
|
+
def event_kind
|
|
33
|
+
:canceled
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def metadata
|
|
37
|
+
{ status: request.status }
|
|
38
|
+
end
|
|
39
|
+
|
|
30
40
|
def reason
|
|
31
41
|
options[:reason]
|
|
32
42
|
end
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module ChangeRequests
|
|
4
|
+
module Commands
|
|
5
|
+
# The sweeper closing out a request whose declaration vanished (§5.11).
|
|
6
|
+
#
|
|
7
|
+
# Commands::CancelUndeclared.call(request:)
|
|
8
|
+
#
|
|
9
|
+
# `Cancel` with two things changed: the event kind, and the metadata. **Who cancelled decides
|
|
10
|
+
# which event is emitted** (Q11) - a person cancelling a stranded request emits `canceled` with
|
|
11
|
+
# their own reason, because they cancelled it; this emits `operation_undeclared`, because
|
|
12
|
+
# "nobody decided this, its declaration vanished" is a different fact and a timeline should not
|
|
13
|
+
# have to infer it from the actor column.
|
|
14
|
+
#
|
|
15
|
+
# It supplies its own reason, so `Cancel`'s mandatory-reason rule holds unchanged rather than
|
|
16
|
+
# growing an exception. The text is translatable, not a hardcoded English sentence (§5.11).
|
|
17
|
+
class CancelUndeclared < Cancel
|
|
18
|
+
REASON_KEY = "change_requests.events.operation_undeclared"
|
|
19
|
+
|
|
20
|
+
def self.call(request:)
|
|
21
|
+
new(request: request, actor: nil, reason: reason).call
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
def self.reason
|
|
25
|
+
Translation.translate(REASON_KEY,
|
|
26
|
+
default: "This operation is no longer declared, so the request " \
|
|
27
|
+
"could never run.")
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
private
|
|
31
|
+
|
|
32
|
+
def event_kind
|
|
33
|
+
:operation_undeclared
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
# Alongside Cancel's `status`: what it was cancelled out of is still worth recording. The
|
|
37
|
+
# version is the request's creation-time one - `Base#emit` falls back to it for the column
|
|
38
|
+
# too, there being no live declaration to read - and it is the version whose disappearance
|
|
39
|
+
# this event reports (§5.5, §5.11).
|
|
40
|
+
def metadata
|
|
41
|
+
super.merge(operation_key: request.operation_key,
|
|
42
|
+
operation_version: request.operation_version)
|
|
43
|
+
end
|
|
44
|
+
end
|
|
45
|
+
end
|
|
46
|
+
end
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module ChangeRequests
|
|
4
|
+
module Commands
|
|
5
|
+
# The clock writing off an execution nobody finished (§8).
|
|
6
|
+
#
|
|
7
|
+
# Commands::Reap.call(request:)
|
|
8
|
+
#
|
|
9
|
+
# T1 claimed the request and committed; then the process died, the box was replaced, or the job
|
|
10
|
+
# backend lost the work. Nothing will ever settle that attempt, so the row would sit `executing`
|
|
11
|
+
# forever - and `executing` is the one status no other command will move (Q28).
|
|
12
|
+
#
|
|
13
|
+
# No actor: `emit` stamps `SYSTEM_ACTOR`, so "who did this" is answerable for every row (§5.5).
|
|
14
|
+
# `Maintenance.reap_stuck_executions!` is the sweep around this transition.
|
|
15
|
+
class Reap < Base
|
|
16
|
+
def self.call(request:, older_than: Guards::Reap::DEFAULT_STUCK_AFTER)
|
|
17
|
+
new(request: request, actor: nil, older_than: older_than).call
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def perform
|
|
21
|
+
guard = Guards::Reap.new(request: request, actor: actor, older_than: options[:older_than])
|
|
22
|
+
guard.check!
|
|
23
|
+
|
|
24
|
+
attempt = guard.stuck_attempt
|
|
25
|
+
|
|
26
|
+
# Emitted before the status changes, as Expire does: the trail records what was reaped
|
|
27
|
+
# rather than what it became, which is `failed` for every one of these rows.
|
|
28
|
+
emit(:reaped, metadata: { attempt: attempt.number, stuck_for: guard.stuck_for })
|
|
29
|
+
|
|
30
|
+
attempt.update!(outcome: "abandoned", finished_at: Time.current)
|
|
31
|
+
request.update!(status: "failed")
|
|
32
|
+
|
|
33
|
+
request
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
end
|
|
@@ -6,13 +6,17 @@ module ChangeRequests
|
|
|
6
6
|
# target that raises leaves no business change and still leaves a durable record of the failure.
|
|
7
7
|
#
|
|
8
8
|
# Internal, like ClaimExecution. `error:` nil is the success branch.
|
|
9
|
+
#
|
|
10
|
+
# It takes no actor. T1 recorded the executer's triple on the attempt, and this reads it back -
|
|
11
|
+
# so background mode can settle in a process that never saw the actor object, and the event
|
|
12
|
+
# names whoever actually claimed the run (§5.6, §8).
|
|
9
13
|
class SettleExecution < Base
|
|
10
14
|
# Enough to locate the failure, bounded so a deep stack cannot write a megabyte per attempt
|
|
11
15
|
# into a host's database. The column is text; nothing else truncates it.
|
|
12
16
|
BACKTRACE_FRAMES = 20
|
|
13
17
|
|
|
14
|
-
def self.call(request:,
|
|
15
|
-
new(request: request,
|
|
18
|
+
def self.call(request:, attempt:, error: nil)
|
|
19
|
+
new(request: request, attempt: attempt, error: error).call
|
|
16
20
|
end
|
|
17
21
|
|
|
18
22
|
def perform
|
|
@@ -32,7 +36,7 @@ module ChangeRequests
|
|
|
32
36
|
end
|
|
33
37
|
|
|
34
38
|
def record_success
|
|
35
|
-
request.update!(status: "successful", executed_at: Time.current,
|
|
39
|
+
request.update!(status: "successful", executed_at: Time.current, **executer_columns)
|
|
36
40
|
attempt.update!(outcome: "succeeded", finished_at: Time.current)
|
|
37
41
|
emit(:executed, metadata: { attempt: attempt.number })
|
|
38
42
|
end
|
|
@@ -47,6 +51,26 @@ module ChangeRequests
|
|
|
47
51
|
metadata: { attempt: attempt.number, error_class: error.class.name })
|
|
48
52
|
end
|
|
49
53
|
|
|
54
|
+
# Copied up from the attempt rather than resolved from an actor: §5.7's triple is already
|
|
55
|
+
# snapshotted there, and the request's executer is whoever claimed the attempt that ran.
|
|
56
|
+
def executer_columns
|
|
57
|
+
{
|
|
58
|
+
executer_type: attempt.executer_type,
|
|
59
|
+
executer_id: attempt.executer_id,
|
|
60
|
+
executer_label: attempt.executer_label,
|
|
61
|
+
}
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
# Base stamps the acting actor, or the System sentinel when there is none. Here there is an
|
|
65
|
+
# actor - the one on the attempt - and no object to resolve it from.
|
|
66
|
+
def event_actor
|
|
67
|
+
{
|
|
68
|
+
actor_type: attempt.executer_type,
|
|
69
|
+
actor_id: attempt.executer_id,
|
|
70
|
+
actor_label: attempt.executer_label,
|
|
71
|
+
}
|
|
72
|
+
end
|
|
73
|
+
|
|
50
74
|
def bounded_backtrace
|
|
51
75
|
Array(error.backtrace).first(BACKTRACE_FRAMES).join("\n").presence
|
|
52
76
|
end
|
|
@@ -6,6 +6,7 @@ module ChangeRequests
|
|
|
6
6
|
class Configuration
|
|
7
7
|
LABEL_STRATEGIES = %i(live snapshot).freeze
|
|
8
8
|
PERMISSION_MATCHES = %i(any all).freeze
|
|
9
|
+
EXECUTION_MODES = %i(inline background).freeze
|
|
9
10
|
|
|
10
11
|
# Identity (§9.1)
|
|
11
12
|
attr_reader :actor_types, :tenant_types
|
|
@@ -22,6 +23,10 @@ module ChangeRequests
|
|
|
22
23
|
# Workflow and execution defaults (§7.1, §8)
|
|
23
24
|
attr_accessor :only_record_rejections, :default_max_attempts, :default_expires_in
|
|
24
25
|
|
|
26
|
+
# Where T2 and T3 run (§8, §10). `job_class` is a string so the gem never holds a class
|
|
27
|
+
# reference across a reload, and so a headless host can name one it has not loaded.
|
|
28
|
+
attr_accessor :execution_mode, :job_class, :job_queue
|
|
29
|
+
|
|
25
30
|
def initialize
|
|
26
31
|
@actor_types = {}
|
|
27
32
|
@tenant_types = {}
|
|
@@ -39,6 +44,10 @@ module ChangeRequests
|
|
|
39
44
|
@only_record_rejections = false
|
|
40
45
|
@default_max_attempts = 1
|
|
41
46
|
@default_expires_in = nil
|
|
47
|
+
|
|
48
|
+
@execution_mode = :inline
|
|
49
|
+
@job_class = "ChangeRequests::Execution::Job"
|
|
50
|
+
@job_queue = :default
|
|
42
51
|
end
|
|
43
52
|
|
|
44
53
|
# §9.2 tells hosts to assign a bare lambda; the gem needs one object answering `allows?`.
|
|
@@ -78,6 +87,9 @@ module ChangeRequests
|
|
|
78
87
|
actor_identity_problem,
|
|
79
88
|
authorization_problem,
|
|
80
89
|
max_attempts_problem,
|
|
90
|
+
execution_mode_problem,
|
|
91
|
+
job_class_problem,
|
|
92
|
+
job_queue_problem,
|
|
81
93
|
*actor_types.values.flat_map(&:problems),
|
|
82
94
|
*tenant_types.values.flat_map(&:problems),
|
|
83
95
|
].compact
|
|
@@ -123,6 +135,29 @@ module ChangeRequests
|
|
|
123
135
|
"Expected #{PERMISSION_MATCHES.map(&:inspect).join(" or ")} (§5.3)."
|
|
124
136
|
end
|
|
125
137
|
|
|
138
|
+
def execution_mode_problem
|
|
139
|
+
return if EXECUTION_MODES.include?(execution_mode)
|
|
140
|
+
|
|
141
|
+
"config.execution_mode is #{execution_mode.inspect}. " \
|
|
142
|
+
"Expected #{EXECUTION_MODES.map(&:inspect).join(" or ")} (§8)."
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
# Not checked for resolvability here: a headless process may legitimately have `:background`
|
|
146
|
+
# configured and no ActiveJob at all, and refusing that would fail a boot that works. The
|
|
147
|
+
# enqueue reports it instead - see ChangeRequests.background_job! (§8).
|
|
148
|
+
def job_class_problem
|
|
149
|
+
return if job_class.respond_to?(:to_str) && !job_class.to_str.strip.empty?
|
|
150
|
+
|
|
151
|
+
"config.job_class is #{job_class.inspect}. Expected the name of an ActiveJob class, " \
|
|
152
|
+
"for example \"ChangeRequests::Execution::Job\" (§10)."
|
|
153
|
+
end
|
|
154
|
+
|
|
155
|
+
def job_queue_problem
|
|
156
|
+
return unless job_queue.nil? || job_queue.to_s.strip.empty?
|
|
157
|
+
|
|
158
|
+
"config.job_queue is #{job_queue.inspect}. Expected a queue name (§10)."
|
|
159
|
+
end
|
|
160
|
+
|
|
126
161
|
def actor_identity_problem
|
|
127
162
|
return if actor_identity.nil? || actor_identity.respond_to?(:call)
|
|
128
163
|
|
|
@@ -15,6 +15,13 @@ module ChangeRequests
|
|
|
15
15
|
g.test_framework :rspec
|
|
16
16
|
end
|
|
17
17
|
|
|
18
|
+
# So a host never calls `load_execution_job!` itself: ActiveJob is usually loaded before
|
|
19
|
+
# Bundler reaches this gem, but `rails/all` is not the only way to boot and `require` decides
|
|
20
|
+
# only once (§8).
|
|
21
|
+
initializer "change_requests.execution_job" do
|
|
22
|
+
ActiveSupport.on_load(:active_job) { ChangeRequests.load_execution_job! }
|
|
23
|
+
end
|
|
24
|
+
|
|
18
25
|
# Fail the boot, not the first request that touches the gem.
|
|
19
26
|
config.after_initialize do
|
|
20
27
|
ChangeRequests.config.validate!
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# §8's background mode. Zeitwerk ignores this file and `ChangeRequests.load_execution_job!`
|
|
4
|
+
# requires it, because the gem has no ActiveJob dependency: `activejob` is a development
|
|
5
|
+
# dependency for the dummy application, and the domain core must load in a process that has
|
|
6
|
+
# never heard of it. Requiring this file without ActiveJob defines nothing at all.
|
|
7
|
+
#
|
|
8
|
+
# archspec:disable-next-line constants.forbid -- §8: the job exists only where the host has ActiveJob
|
|
9
|
+
if defined?(ActiveJob::Base)
|
|
10
|
+
module ChangeRequests
|
|
11
|
+
module Execution
|
|
12
|
+
# T1 has already committed in the caller's process, so the request is visibly `executing`
|
|
13
|
+
# and the attempt row is claimed. This runs T2 and T3, and nothing else.
|
|
14
|
+
#
|
|
15
|
+
# archspec:disable-next-line constants.forbid -- ditto, and the class cannot exist without it
|
|
16
|
+
class Job < ::ActiveJob::Base
|
|
17
|
+
# The two ids rather than the objects: a job argument has to survive serialisation, and the
|
|
18
|
+
# attempt already carries the executer's triple, so T3 needs no actor object (§5.6).
|
|
19
|
+
def perform(change_request_id, attempt_id)
|
|
20
|
+
request = Request.find(change_request_id)
|
|
21
|
+
|
|
22
|
+
Runner.finish(request: request, attempt: request.attempts.find(attempt_id))
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
end
|
|
@@ -18,7 +18,12 @@ module ChangeRequests
|
|
|
18
18
|
new(request: request, actor: actor, override: override, reason: reason).call
|
|
19
19
|
end
|
|
20
20
|
|
|
21
|
-
|
|
21
|
+
# T2 and T3 alone: T1 committed in another process, which is what Execution::Job resumes.
|
|
22
|
+
def self.finish(request:, attempt:)
|
|
23
|
+
new(request: request).finish(attempt)
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def initialize(request:, actor: nil, override: false, reason: nil)
|
|
22
27
|
@request = request
|
|
23
28
|
@actor = actor
|
|
24
29
|
@override = override
|
|
@@ -29,6 +34,14 @@ module ChangeRequests
|
|
|
29
34
|
attempt = Commands::ClaimExecution.call(request: request, actor: actor,
|
|
30
35
|
override: override, reason: reason)
|
|
31
36
|
|
|
37
|
+
# T1 commits either way, so the request is visibly `executing` the moment this returns -
|
|
38
|
+
# which is the whole reason background mode does not enqueue the claim as well (§8).
|
|
39
|
+
return enqueue(attempt) if background?
|
|
40
|
+
|
|
41
|
+
finish(attempt)
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
def finish(attempt)
|
|
32
45
|
begin
|
|
33
46
|
invoke
|
|
34
47
|
rescue StandardError => e
|
|
@@ -47,6 +60,20 @@ module ChangeRequests
|
|
|
47
60
|
|
|
48
61
|
attr_reader :request, :actor, :override, :reason
|
|
49
62
|
|
|
63
|
+
def background?
|
|
64
|
+
ChangeRequests.config.execution_mode == :background
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
# The ids, not the objects: §8's job takes what survives serialisation. Returns the claimed
|
|
68
|
+
# request, so a caller sees what inline mode gives it - a row, already `executing`.
|
|
69
|
+
def enqueue(attempt)
|
|
70
|
+
ChangeRequests.background_job!
|
|
71
|
+
.set(queue: ChangeRequests.config.job_queue)
|
|
72
|
+
.perform_later(request.id, attempt.id)
|
|
73
|
+
|
|
74
|
+
request.reload
|
|
75
|
+
end
|
|
76
|
+
|
|
50
77
|
def invoke
|
|
51
78
|
Dispatcher.call(operation_key: request.operation_key, payload: request.payload,
|
|
52
79
|
change_request_id: request.id)
|
|
@@ -62,8 +89,10 @@ module ChangeRequests
|
|
|
62
89
|
"#{operation.service}.#{operation.method_name}"
|
|
63
90
|
end
|
|
64
91
|
|
|
92
|
+
# No actor: the attempt carries the executer's triple from T1, which is what T3 records.
|
|
93
|
+
# That is what lets the job settle a claim it did not make (§5.6).
|
|
65
94
|
def settle(attempt, error)
|
|
66
|
-
Commands::SettleExecution.call(request: request,
|
|
95
|
+
Commands::SettleExecution.call(request: request, attempt: attempt, error: error)
|
|
67
96
|
end
|
|
68
97
|
end
|
|
69
98
|
end
|
|
@@ -9,14 +9,22 @@ module ChangeRequests
|
|
|
9
9
|
# about the step it happens to be sitting on.
|
|
10
10
|
#
|
|
11
11
|
# The reason is mandatory and `Commands::Cancel` enforces it, for the reason Reject does (Q25).
|
|
12
|
+
#
|
|
13
|
+
# Once the operation is undeclared, **anyone** may cancel (§5.11, Q9). A request that can never
|
|
14
|
+
# run is not worth adjudicating who may tidy it away, and leaving it clearable only by a rake
|
|
15
|
+
# task means a stranded row sits in the table until an operator notices.
|
|
12
16
|
class Cancel < Base
|
|
13
17
|
refuses_with NotCancelable
|
|
18
|
+
exempt_from_undeclared_operation!
|
|
14
19
|
|
|
15
20
|
def refusal
|
|
16
21
|
return :already_finalized if request.final?
|
|
17
22
|
# The target is mid-flight. A status change cannot recall it, and setting a terminal status
|
|
18
|
-
# would leave the execution unable to record its own outcome (Q28, §8).
|
|
23
|
+
# would leave the execution unable to record its own outcome (Q28, §8). Ahead of the
|
|
24
|
+
# undeclared branch: being undeclared does not make a running request recallable.
|
|
19
25
|
return :executing if request.executing?
|
|
26
|
+
# §5.11: the requester-or-approver rule is dropped along with the refusal, not before it.
|
|
27
|
+
return nil if operation.nil?
|
|
20
28
|
return :not_permitted unless requester? || eligible_approver?
|
|
21
29
|
|
|
22
30
|
nil
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module ChangeRequests
|
|
4
|
+
module Guards
|
|
5
|
+
# May this request's abandoned execution be written off (§8)?
|
|
6
|
+
#
|
|
7
|
+
# System only, for the reason `Expire` is: nobody reaps a request on purpose, the clock does,
|
|
8
|
+
# so an actor being supplied at all is the refusal. Every refusal is `NotAuthorized` rather
|
|
9
|
+
# than a TransitionError apart from the shared `:already_finalized` mapping - the only caller
|
|
10
|
+
# is `Maintenance.reap_stuck_executions!`, whose query already filters on status and age, so
|
|
11
|
+
# these branches are a floor beneath it and never a user-facing flash (Q32).
|
|
12
|
+
# Exempt from the undeclared-operation refusal, joining Comment and Cancel (§5.11). Whether
|
|
13
|
+
# the declaration still exists has no bearing on "this attempt died and nobody will settle it" -
|
|
14
|
+
# and Cancel refuses an `executing` request (Q28), so without this a row claimed just as its
|
|
15
|
+
# declaration vanished could be cleared by nothing at all.
|
|
16
|
+
class Reap < Base
|
|
17
|
+
# §8's documented default, in seconds rather than `1.hour`: the domain core loads against a
|
|
18
|
+
# bare ActiveRecord, which does not bring active_support/core_ext/numeric/time with it. A
|
|
19
|
+
# host passing `1.hour` still works - a Duration subtracts from a Time exactly the same.
|
|
20
|
+
DEFAULT_STUCK_AFTER = 3600
|
|
21
|
+
|
|
22
|
+
refuses_with NotAuthorized
|
|
23
|
+
exempt_from_undeclared_operation!
|
|
24
|
+
|
|
25
|
+
def refusal
|
|
26
|
+
return :not_system unless actor.nil?
|
|
27
|
+
return :already_finalized if request.final?
|
|
28
|
+
return :not_executing unless request.executing?
|
|
29
|
+
return :not_stuck if stuck_attempt.nil?
|
|
30
|
+
|
|
31
|
+
nil
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
# The attempt the reaper writes off, exposed so the command writes the same row the guard
|
|
35
|
+
# judged - one guard object, built once (I7).
|
|
36
|
+
def stuck_attempt
|
|
37
|
+
return @stuck_attempt if defined?(@stuck_attempt)
|
|
38
|
+
|
|
39
|
+
@stuck_attempt = request.attempts.in_flight.where(started_at: ..cutoff).order(:number).last
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def stuck_for(now = Time.current)
|
|
43
|
+
return nil if stuck_attempt&.started_at.nil?
|
|
44
|
+
|
|
45
|
+
(now - stuck_attempt.started_at).round
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
private
|
|
49
|
+
|
|
50
|
+
def cutoff
|
|
51
|
+
Time.current - older_than
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
def older_than
|
|
55
|
+
options.fetch(:older_than, DEFAULT_STUCK_AFTER)
|
|
56
|
+
end
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
end
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module ChangeRequests
|
|
4
|
+
# The scheduled sweeps (§8, §5.11). Each one is a query plus the command that owns its
|
|
5
|
+
# transition, never a write of its own: the commands hold the locks and emit the events, and
|
|
6
|
+
# these decide which rows to hand them (ADR 0016).
|
|
7
|
+
#
|
|
8
|
+
# Every event they cause carries the `System` sentinel, because `Commands::Base#emit` stamps it
|
|
9
|
+
# when `actor` is nil - closing these rows out is the gem's own act, not anyone's decision.
|
|
10
|
+
#
|
|
11
|
+
# Each returns the number of rows it moved, which is what M3b-3's rake tasks report. Rows are
|
|
12
|
+
# read into an array first: a sweep whose scope stops matching as it works should still visit
|
|
13
|
+
# everything it set out to.
|
|
14
|
+
#
|
|
15
|
+
# `close_due_stages!` is **M9b** - a cooldown window has to exist before a stage can be due.
|
|
16
|
+
module Maintenance
|
|
17
|
+
module_function
|
|
18
|
+
|
|
19
|
+
# `pending` / `approved` past `expires_at` become `expired`. The query is the same
|
|
20
|
+
# `Request.expired_candidates` that `Guards::Expire` agrees with status for status (§8).
|
|
21
|
+
def expire_stale!(now: Time.current)
|
|
22
|
+
sweep(Request.expired_candidates(now)) { |request| Commands::Expire.call(request: request) }
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
# `executing` rows whose attempt nobody ever settled. T1 committed the claim and then the
|
|
26
|
+
# process died; `executing` is the one status no other command will move, so without this the
|
|
27
|
+
# row sits there forever (§8).
|
|
28
|
+
def reap_stuck_executions!(older_than: Guards::Reap::DEFAULT_STUCK_AFTER)
|
|
29
|
+
sweep(Request.stuck_executions(older_than)) do |request|
|
|
30
|
+
Commands::Reap.call(request: request, older_than: older_than)
|
|
31
|
+
end
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
# Non-final requests whose `operation_key` is no longer declared (§5.11). Deliberately not
|
|
35
|
+
# automatic: a missing declaration is as likely to be a deploy accident as a deliberate
|
|
36
|
+
# removal, and `canceled` is final - so this runs when an operator asks for it.
|
|
37
|
+
def cancel_undeclared!
|
|
38
|
+
sweep(Request.undeclared) { |request| Commands::CancelUndeclared.call(request: request) }
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
def sweep(scope, &)
|
|
42
|
+
rows = scope.to_a
|
|
43
|
+
rows.each(&)
|
|
44
|
+
|
|
45
|
+
rows.size
|
|
46
|
+
end
|
|
47
|
+
end
|
|
48
|
+
end
|
|
@@ -53,6 +53,24 @@ module ChangeRequests
|
|
|
53
53
|
where(status: %w(pending approved)).where(expires_at: ...now)
|
|
54
54
|
}
|
|
55
55
|
|
|
56
|
+
# What Maintenance.reap_stuck_executions! sweeps (§8): claimed, and nothing ever settled the
|
|
57
|
+
# attempt. Inclusive at the cutoff, as Guards::Reap is - a spec asserts the two agree.
|
|
58
|
+
scope :stuck_executions, lambda { |older_than = Guards::Reap::DEFAULT_STUCK_AFTER|
|
|
59
|
+
claimed = Attempt.in_flight.where(started_at: ..(Time.current - older_than))
|
|
60
|
+
|
|
61
|
+
where(status: "executing").where(id: claimed.select(:change_request_id))
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
# What Maintenance.cancel_undeclared! sweeps (§5.11). `executing` is excluded deliberately:
|
|
65
|
+
# Guards::Cancel refuses a request mid-flight, and being undeclared does not make it
|
|
66
|
+
# recallable - the reaper is what clears those.
|
|
67
|
+
scope :undeclared, lambda {
|
|
68
|
+
declared = ChangeRequests.operations.keys
|
|
69
|
+
candidates = where(status: OPEN_STATUSES - %w(executing))
|
|
70
|
+
|
|
71
|
+
declared.empty? ? candidates : candidates.where.not(operation_key: declared)
|
|
72
|
+
}
|
|
73
|
+
|
|
56
74
|
def current_stage
|
|
57
75
|
stages.find_by(position: current_stage_position)
|
|
58
76
|
end
|
data/lib/change_requests.rb
CHANGED
|
@@ -57,6 +57,39 @@ module ChangeRequests
|
|
|
57
57
|
payload: payload, tenant: tenant)
|
|
58
58
|
end
|
|
59
59
|
|
|
60
|
+
# §8's background mode, on the same terms as the engine: idempotent, public, and guarded so
|
|
61
|
+
# requiring the gem in a process without ActiveJob defines no job at all. A host that loads
|
|
62
|
+
# ActiveJob after this gem calls it again - the engine does that automatically on_load.
|
|
63
|
+
def load_execution_job!
|
|
64
|
+
# archspec:disable-next-line dependencies.forbid -- the loader must name what it loads (§1)
|
|
65
|
+
return false if defined?(ChangeRequests::Execution::Job)
|
|
66
|
+
# archspec:disable-next-line constants.forbid -- the guard that keeps ActiveJob optional (§8)
|
|
67
|
+
return false unless defined?(::ActiveJob::Base)
|
|
68
|
+
|
|
69
|
+
require_relative "change_requests/execution/job"
|
|
70
|
+
|
|
71
|
+
true
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
# What a headless process asks before trusting `execution_mode = :background`.
|
|
75
|
+
def background_available?
|
|
76
|
+
# archspec:disable-next-line dependencies.forbid -- the same reference, as a question (§8)
|
|
77
|
+
defined?(ChangeRequests::Execution::Job) ? true : false
|
|
78
|
+
end
|
|
79
|
+
|
|
80
|
+
# The configured job class, resolved at enqueue time and never held: a reloading application
|
|
81
|
+
# redefines it, and §10 makes the setting a string for that reason.
|
|
82
|
+
def background_job!
|
|
83
|
+
job = config.job_class.to_s.safe_constantize
|
|
84
|
+
|
|
85
|
+
return job unless job.nil?
|
|
86
|
+
|
|
87
|
+
fail ConfigurationError,
|
|
88
|
+
"config.execution_mode is :background but config.job_class " \
|
|
89
|
+
"(#{config.job_class.inspect}) does not resolve. ChangeRequests::Execution::Job is " \
|
|
90
|
+
"defined only where ActiveJob is loaded, which this process has not done (§8)."
|
|
91
|
+
end
|
|
92
|
+
|
|
60
93
|
def configure
|
|
61
94
|
yield(config)
|
|
62
95
|
|
|
@@ -97,8 +130,10 @@ module ChangeRequests
|
|
|
97
130
|
loader.ignore("#{__dir__}/change_requests/version.rb")
|
|
98
131
|
loader.ignore("#{__dir__}/change_requests/errors.rb")
|
|
99
132
|
|
|
100
|
-
# Autoloading
|
|
133
|
+
# Autoloading either would let an eager load pull in something optional: Rails for the
|
|
134
|
+
# engine, ActiveJob for the job - which defines nothing at all when ActiveJob is absent.
|
|
101
135
|
loader.ignore("#{__dir__}/change_requests/engine.rb")
|
|
136
|
+
loader.ignore("#{__dir__}/change_requests/execution/job.rb")
|
|
102
137
|
|
|
103
138
|
loader.setup
|
|
104
139
|
end
|
|
@@ -108,3 +143,4 @@ end
|
|
|
108
143
|
|
|
109
144
|
ChangeRequests.setup_loader
|
|
110
145
|
ChangeRequests.load_engine!
|
|
146
|
+
ChangeRequests.load_execution_job!
|