shifty 0.4.2 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,127 @@
1
+ # Testing Workers
2
+
3
+ Shifty 0.6.0 ships an opt-in test harness (`require "shifty/testing"`) and RSpec sugar (`require "shifty/rspec"`) so your unit tests exercise workers under the **same handoff policy the production pipeline will** (see [[Handoff-Policies]]). Neither is loaded by `require "shifty"` — they're deliberately opt-in. This page covers the parity problem they solve, `Shifty::Testing.run`, the mutation detector, the RSpec matcher and shared example, and the "test at the ceiling" guidance.
4
+
5
+ ## The parity problem
6
+
7
+ A pipeline runs `:frozen`; a unit test exercises the worker's raw proc with an ordinary mutable object:
8
+
9
+ ```ruby
10
+ worker.task.call(input) # bypasses the framework — and the policy
11
+ ```
12
+
13
+ The test passes; the pipeline raises. The gap closes from two directions:
14
+
15
+ 1. **Worker-level policy declarations win** over pipeline defaults, so the policy travels *with* the worker. Any test that runs the worker through the framework automatically exercises production semantics.
16
+ 2. **The framework path is the convenient path** — that's `Shifty::Testing.run`.
17
+
18
+ With `:frozen` as the global default, the common case requires no test configuration at all.
19
+
20
+ ## `Shifty::Testing.run`
21
+
22
+ ```ruby
23
+ require "shifty/testing"
24
+
25
+ Shifty::Testing.run(worker, inputs:, policy: nil, max_shifts: 10_000)
26
+ ```
27
+
28
+ Feeds the inputs to the worker via a synthetic source and collects its outputs until the end-of-stream sentinel (`nil`). The worker's declared/effective policy governs each handoff, exactly as in production; the harness restores the worker's policy, supply, and Fiber afterward, so it never leaves a mark on the worker under test.
29
+
30
+ ```ruby
31
+ worker = relay_worker { |v| v.upcase }
32
+ Shifty::Testing.run(worker, inputs: ["a", "b", "c"]) #=> ["A", "B", "C"]
33
+
34
+ # Parity with production — a mutating task raises under the default policy:
35
+ mutator = Shifty::Worker.new { |v| v && v << :x }
36
+ Shifty::Testing.run(mutator, inputs: [[:a]]) # raises PolicyViolation
37
+
38
+ # A worker that declared :isolated runs under :isolated:
39
+ scratch = Shifty::Worker.new(policy: :isolated) { |v| v && v << :x }
40
+ Shifty::Testing.run(scratch, inputs: [[:a]]) #=> [[:a, :x]]
41
+
42
+ # Explicit policy: override beats the worker's declaration (policy-matrix testing);
43
+ # the worker's real declaration is untouched afterward:
44
+ loose = Shifty::Worker.new(policy: :shared) { |v| v && v << :x }
45
+ Shifty::Testing.run(loose, inputs: [[:a]], policy: :frozen) # raises PolicyViolation
46
+ loose.effective_policy #=> :shared
47
+
48
+ # Non-1:1 output streams collect naturally:
49
+ evens = filter_worker { |v| v.even? }
50
+ Shifty::Testing.run(evens, inputs: [1, 2, 3, 4]) #=> [2, 4]
51
+ ```
52
+
53
+ ### Sentinel semantics
54
+
55
+ `nil` is Shifty's end-of-stream sentinel; collection stops at the first `nil` output. **`false` is a legitimate payload**, not end-of-stream — it flows through and gets collected:
56
+
57
+ ```ruby
58
+ flipper = relay_worker { |v| !v }
59
+ Shifty::Testing.run(flipper, inputs: [true, false, true])
60
+ #=> [false, false, false]
61
+ # (relay_worker's `value &&` guard passes the false input through untouched.)
62
+ ```
63
+
64
+ If a worker's task converts `nil` into something non-nil, the run would never terminate — so after `max_shifts` (default 10,000) the harness raises a diagnostic `Shifty::Error` telling you to let `nil` pass through (e.g. `value && ...`) or raise `max_shifts:`.
65
+
66
+ ## The mutation detector: `mutates_input?`
67
+
68
+ ```ruby
69
+ Shifty::Testing.mutates_input?(worker, input) #=> true / false
70
+ ```
71
+
72
+ Hands the task a private mutable deep copy of `input`, runs it through the framework under `:shared`, and compares the copy before and after (via `Marshal.dump`). It surfaces mutation **even when the worker's current policy permits or hides it**:
73
+
74
+ ```ruby
75
+ sneaky = side_worker(policy: :isolated) { |v| v << :boo }
76
+ Shifty::Testing.mutates_input?(sneaky, [:a]) #=> true — the :isolated copy hid it in production
77
+
78
+ clean = relay_worker { |v| v + [:x] }
79
+ Shifty::Testing.mutates_input?(clean, [:a]) #=> false
80
+ ```
81
+
82
+ This is exactly the information you need **before** loosening a policy — moving to `:shared` never raises, so the detector is your only warning (the silent-loosening asymmetry; see [[Handoff-Policies]]). It's also the pre-upgrade inventory tool for [[Migration-Guide-0.6]]. The input must be Marshal-dumpable; anything else raises a descriptive `Shifty::Error`.
83
+
84
+ ## RSpec sugar
85
+
86
+ ```ruby
87
+ # spec_helper.rb
88
+ require "shifty/rspec" # pulls in shifty/testing; assumes RSpec is loaded
89
+ ```
90
+
91
+ ### The `mutate_input` matcher
92
+
93
+ Takes the input as an argument and delegates to the mutation detector:
94
+
95
+ ```ruby
96
+ expect(worker).not_to mutate_input([:a])
97
+ ```
98
+
99
+ The negated failure message spells out the consequences: a mutating task is only correct under `:isolated` (mutation stays local) or `:shared` (mutation is intentional); it will raise under `:frozen`.
100
+
101
+ ### The `"a policy-safe worker"` shared example
102
+
103
+ Expects `worker` and `safe_input` to be defined with `let`:
104
+
105
+ ```ruby
106
+ RSpec.describe "my enricher" do
107
+ let(:worker) { relay_worker { |v| v.merge(enriched: true) } }
108
+ let(:safe_input) { {id: 1} }
109
+
110
+ it_behaves_like "a policy-safe worker"
111
+ end
112
+ ```
113
+
114
+ It runs two checks: the worker processes a **deeply frozen** input through the framework without raising (`Testing.run(..., policy: :frozen)`), and it does not mutate its input.
115
+
116
+ ## Test at the ceiling
117
+
118
+ Policy safety is (almost) a one-way ratchet: a task correct on deeply frozen input is correct under every policy; the reverse does not hold. So the guidance is to **test under `:frozen`** — the ceiling — unless the worker explicitly declares mutation as part of its design. Consequences:
119
+
120
+ - Loosening a pipeline's policy can never break a ceiling-tested worker.
121
+ - Tightening is the only breaking direction, and tightening fails loudly with `PolicyViolation` diagnostics.
122
+
123
+ **The "almost":** `:isolated` hands the task a *copy*, so a worker depending on object **identity** (rare, but conceivable with identity-keyed caches) behaves differently there. This is the one spot where strict-passing does not imply lax-passing. If your worker cares about `equal?`, test it under `:isolated` explicitly:
124
+
125
+ ```ruby
126
+ Shifty::Testing.run(worker, inputs: [thing], policy: :isolated)
127
+ ```
@@ -0,0 +1,171 @@
1
+ # Worker Types
2
+
3
+ Shifty's DSL (`include Shifty::DSL`) provides shorthand constructors for the common worker shapes; underneath them all is `Shifty::Worker`, and above them sits `Shifty::Gang` for treating a chain as one unit. This page is the per-type reference for 0.6.0, including how each type behaves under the default `:frozen` handoff policy (see [[Handoff-Policies]]). All DSL constructors except `source_worker` and `trailing_worker` accept an options hash that passes through to `Worker.new` — so `policy:`, `name:`, `tags:` work everywhere.
4
+
5
+ ## `source_worker`
6
+
7
+ At the headwater of every pipeline: a worker that generates its own work items. Its block takes no arguments; `nil` signals end-of-stream (and once a source returns `nil`, it returns `nil` henceforth).
8
+
9
+ ```ruby
10
+ worker = source_worker { "Number 9" } # generates forever
11
+
12
+ counter = 0
13
+ finite = source_worker do # generates until it returns nil
14
+ if counter < 3
15
+ counter += 1
16
+ counter + 1000
17
+ end
18
+ end
19
+
20
+ w1 = source_worker [0, 1, 2] # or hand it a series
21
+ w2 = source_worker (0..2) # ranges work too
22
+ ```
23
+
24
+ A series-based source yields each element and then `nil` forever. Strings are split into characters; a bare scalar becomes a one-element series. You can also combine a series with a block, which acts as a transform: `source_worker([1,2]) { |v| v * 10 }`.
25
+
26
+ Sources are where "mutable within, immutable between" starts: build values freely in closure scope, and hand off snapshots (see [[Coding-Idioms-Under-Frozen]]).
27
+
28
+ ## `relay_worker`
29
+
30
+ The most "normal" worker: accepts a value, returns a transformation of it. `nil` passes through untouched (the end-of-stream sentinel survives).
31
+
32
+ ```ruby
33
+ squarer = relay_worker { |number| number ** 2 }
34
+ pipeline = source_worker(0..3) | squarer
35
+ pipeline.shift #=> 0, 1, 4, 9, nil
36
+ ```
37
+
38
+ Under `:frozen`, relay tasks must be non-destructive — `v.merge(...)`, `v + [...]`, `v.with(...)`. A `v <<` raises `PolicyViolation` at this worker, by name if you gave it one.
39
+
40
+ ## `side_worker`
41
+
42
+ Passes through the value it received while performing a side effect — logging, metrics, stashing. Its purpose is *intentionality*: side effects live in named, removable workers, so pulling one out of the pipeline never changes the pipeline's output.
43
+
44
+ ```ruby
45
+ evens = []
46
+ even_stasher = side_worker { |value| evens << value if value.even? }
47
+ ```
48
+
49
+ Policy behavior is where 0.6.0 makes the side worker's contract real:
50
+
51
+ - **Under `:frozen` (default):** a block that mutates the passed value raises `PolicyViolation`. Observation is enforced.
52
+ - **Under `:isolated`:** the block observes a private scratch copy; since a side worker's return value is discarded, its mutations simply **evaporate** and the untouched value flows on:
53
+
54
+ ```ruby
55
+ source = source_worker [[:foo], [:bar]]
56
+ unsafe = side_worker(policy: :isolated) { |v| v << :boo }
57
+ pipeline = source | unsafe
58
+ pipeline.shift #=> [:foo] — shenanigans contained
59
+ pipeline.shift #=> [:bar]
60
+ ```
61
+
62
+ - **`mode:` is deprecated:** `mode: :hardened` maps to `policy: :isolated` with a warning; any other `mode:` value warns and is ignored. Removed in 1.0.0 — see [[Migration-Guide-0.6]].
63
+
64
+ ## `filter_worker`
65
+
66
+ Passes through only values for which the block is truthy; falsy values are discarded (the worker pulls from its supply until something passes).
67
+
68
+ ```ruby
69
+ evens_only = filter_worker { |value| value % 2 == 0 }
70
+ pipeline = source_worker(0..5) | evens_only
71
+ pipeline.shift #=> 0, 2, 4, nil
72
+ ```
73
+
74
+ Every value the filter pulls mid-task crosses the boundary under the worker's policy — including the ones it discards.
75
+
76
+ ## `batch_worker`
77
+
78
+ Gathers values into batches. Either a fixed size:
79
+
80
+ ```ruby
81
+ batch = batch_worker gathering: 3
82
+ pipeline = source_worker(0..7) | batch
83
+ pipeline.shift #=> [0, 1, 2], then [3, 4, 5], then [6, 7], then nil
84
+ ```
85
+
86
+ (the final batch may be short), or a condition — batch until the block is truthy:
87
+
88
+ ```ruby
89
+ line_reader = batch_worker { |value| value.end_with?("\n") }
90
+ ```
91
+
92
+ ## `splitter_worker`
93
+
94
+ Accepts one value, produces an array from it, and hands off each element successively before asking its supply for more.
95
+
96
+ ```ruby
97
+ splitter = splitter_worker { |value| value.split(" ") }
98
+ pipeline = source_worker(["A bold", "move westward"]) | splitter
99
+ pipeline.shift #=> "A", "bold", "move", "westward", nil
100
+ ```
101
+
102
+ Every part a splitter yields is policy-governed as it crosses into the next worker — under `:frozen`, each part arrives frozen downstream.
103
+
104
+ ## `trailing_worker`
105
+
106
+ Returns an array of the last *n* values — useful for rolling averages. Nothing is returned until *n* values have accumulated; new values are unshifted into position zero.
107
+
108
+ ```ruby
109
+ trailer = trailing_worker 4
110
+ pipeline = source_worker(0..5) | trailer
111
+ pipeline.shift #=> [3, 2, 1, 0], then [4, 3, 2, 1], then [5, 4, 3, 2], then nil
112
+ ```
113
+
114
+ New in 0.6.0: the trailing worker **hands off a snapshot** (`trail.dup`) rather than its live closure array. It keeps mutating that array across calls, and a downstream `:frozen` intake would otherwise freeze the live array in place. This is the canonical example of the builder snapshot rule — see [[Coding-Idioms-Under-Frozen]].
115
+
116
+ (Signature note: `trailing_worker(trail_length = 2)` takes the length positionally and no options hash.)
117
+
118
+ ## Raw `Shifty::Worker.new`
119
+
120
+ When the DSL shapes don't fit, build a worker directly:
121
+
122
+ ```ruby
123
+ worker = Shifty::Worker.new(
124
+ supply: upstream, # or wire later: worker.supply = upstream
125
+ policy: :isolated, # this worker's contract; validated eagerly
126
+ name: "enricher", # used in PolicyViolation / UnshareableValue messages
127
+ tags: [:etl], # also reported in diagnostics
128
+ criteria: ->(w) { ... }, # when falsy, the task is bypassed (value still policy-governed)
129
+ task: some_callable # or pass a block
130
+ ) { |value, supply, context| ... }
131
+ ```
132
+
133
+ The task's arity matters:
134
+
135
+ - **Arity 0** — a source; it cannot accept a supply (`supply=` raises). Use `handoff(value)` (which is `Fiber.yield`) to emit from inside loops.
136
+ - **Arity 1+** — `|value|` receives the policy-governed intake value.
137
+ - **Second argument** — a policy-governed supply proxy responding only to `#shift`, for tasks that pull additional values themselves (this is how filter/batch/trailing work). It is *not* the raw upstream worker.
138
+ - **Third argument** — the worker's `context`, an `OpenStruct` by default or whatever you pass as `context:`; per-worker mutable state that survives across shifts (and survives a rescued `PolicyViolation`).
139
+
140
+ A worker with no task gets a default pass-through task. `worker.effective_policy` reports the resolved policy (own declaration, else pipeline default, else global default).
141
+
142
+ ## Composition: `|`, `with_policy`, `freeze!`
143
+
144
+ ```ruby
145
+ pipeline = source | filter | transform | sink # tail-returned; call shift on it
146
+
147
+ pipeline = (source | filter | transform | sink)
148
+ .with_policy(:frozen) # pipeline default for workers with no declaration
149
+ .freeze! # lock the topology; upstream walk from the tail
150
+ ```
151
+
152
+ Both `with_policy` and `freeze!` walk **upstream** through the supply chain from their receiver and return it, so they chain — and both should be called on the pipeline's tail. `freeze!` locks topology only (supply wiring, Gang rosters); closure and context state stay mutable. Use `#freeze!`, never bare `#freeze`. Details in [[Handoff-Policies]].
153
+
154
+ ## `Shifty::Gang`
155
+
156
+ A Gang wraps an ordered roster of workers so a whole chain acts like one worker: it has a `supply`, a `shift`, and composes with `|` like anything else.
157
+
158
+ ```ruby
159
+ gang = Shifty::Gang[parser, enricher] # or Gang.new([parser, enricher])
160
+ pipeline = reader | gang | writer
161
+ ```
162
+
163
+ Policy features:
164
+
165
+ - **Construction fanout:** `Gang.new(workers, policy: :isolated)` sets the pipeline default on every roster member.
166
+ - **`with_policy` fanout:** `gang.with_policy(:shared)` does the same, chainably.
167
+ - **Append inheritance:** the gang persists its declared policy, so workers appended *after* the declaration inherit it too.
168
+ - **Member contracts win:** a roster member's own `policy:` declaration beats the gang's, per the usual precedence.
169
+ - **Chains walk through:** `(upstream | gang | tail).with_policy(:isolated)` reaches the gang's roster *and* workers upstream of it; `freeze!` walks the same path.
170
+ - **Freezing:** a frozen gang still runs, but its roster membership is locked — `append` raises `FrozenError`.
171
+ - **Criteria bypass is still governed:** when a gang's `criteria` bypasses its workers, the value crossing the gang boundary is still policy-governed at the entry worker.
@@ -0,0 +1,13 @@
1
+ **[[Home]]**
2
+
3
+ *Concepts*
4
+ - [[Handoff-Policies]]
5
+ - [[Coding-Idioms-Under-Frozen]]
6
+ - [[Worker-Types]]
7
+
8
+ *Doing things*
9
+ - [[Migration-Guide-0.6]]
10
+ - [[Testing-Workers]]
11
+
12
+ *Reference*
13
+ - [[Performance]]
@@ -0,0 +1,27 @@
1
+ module Shifty
2
+ class Configuration
3
+ attr_reader :default_policy
4
+
5
+ def initialize
6
+ @default_policy = :frozen
7
+ end
8
+
9
+ def default_policy=(policy_name)
10
+ @default_policy = Policy.validate!(policy_name)
11
+ end
12
+ end
13
+
14
+ class << self
15
+ def config
16
+ @config ||= Configuration.new
17
+ end
18
+
19
+ def configure
20
+ yield config
21
+ end
22
+
23
+ def reset_configuration!
24
+ @config = Configuration.new
25
+ end
26
+ end
27
+ end
data/lib/shifty/dsl.rb CHANGED
@@ -1,6 +1,4 @@
1
1
  module Shifty
2
- class WorkerInitializationError < StandardError; end
3
-
4
2
  module DSL
5
3
  def source_worker(argument = nil, &block)
6
4
  ensure_correct_arity_for!(argument, block)
@@ -19,36 +17,46 @@ module Shifty
19
17
  end
20
18
  end
21
19
 
22
- def relay_worker(&block)
23
- ensure_regular_arity(block)
20
+ def relay_worker(options = {}, &block)
21
+ options[:tags] ||= []
22
+ options[:tags] << :relay
23
+ ensure_regular_arity!(block)
24
24
 
25
- Worker.new(tags: [:relay]) do |value|
25
+ Worker.new(options) do |value|
26
26
  value && block.call(value)
27
27
  end
28
28
  end
29
29
 
30
- def side_worker(mode = :normal, &block)
31
- ensure_regular_arity(block)
30
+ def side_worker(options = {}, &block)
31
+ options[:tags] ||= []
32
+ options[:tags] << :side_effect
33
+ deprecate_mode_option!(options)
34
+ ensure_regular_arity!(block)
32
35
 
33
- Worker.new(tags: [:side_effect]) do |value|
36
+ # The block must ask the worker for its policy at shift time, so the
37
+ # local is captured by the closure before it is assigned. Not redundant.
38
+ worker = nil
39
+ worker = Worker.new(options) do |value| # standard:disable Style/RedundantAssignment
34
40
  value.tap do |v|
35
- used_value = (mode == :hardened) ?
36
- Marshal.load(Marshal.dump(v)) : v
41
+ next unless v
42
+ # Under :isolated the block observes a private scratch copy, so
43
+ # its mutations evaporate and the untouched value flows on.
44
+ used_value = (worker.effective_policy == :isolated) ?
45
+ Policy::Isolated.call(v, worker: worker) : v
37
46
 
38
- v && block.call(used_value)
47
+ block.call(used_value)
39
48
  end
40
49
  end
50
+ worker
41
51
  end
42
52
 
43
- def filter_worker(argument = nil, &block)
44
- if block && argument.respond_to?(:call)
45
- throw_with "You cannot supply two callables"
46
- end
47
- callable = argument.respond_to?(:call) ? argument : block
48
- ensure_callable(callable)
53
+ def filter_worker(options = {}, &block)
54
+ options[:tags] ||= []
55
+ options[:tags] << :filter
56
+ ensure_callable!(block)
49
57
 
50
- Worker.new(tags: [:filter]) do |value, supply|
51
- while value && !callable.call(value)
58
+ Worker.new(options) do |value, supply|
59
+ while value && !block.call(value)
52
60
  value = supply.shift
53
61
  end
54
62
  value
@@ -62,14 +70,18 @@ module Shifty
62
70
  end
63
71
  end
64
72
 
65
- def batch_worker(options = {gathering: 1}, &block)
66
- ensure_regular_arity(block) if block
73
+ def batch_worker(options = {}, &block)
74
+ options[:tags] ||= []
75
+ options[:tags] << :batch
76
+ options[:gathering] ||= 1
77
+
78
+ ensure_regular_arity!(block) if block
67
79
  batch_full = block ||
68
80
  proc { |_, batch| batch.size >= options[:gathering] }
69
81
 
70
- batch_context = BatchContext.new({batch_full: batch_full})
82
+ options[:context] = BatchContext.new({batch_full: batch_full})
71
83
 
72
- Worker.new(tags: [:batch], context: batch_context) do |value, supply, context|
84
+ Worker.new(options) do |value, supply, context|
73
85
  if value
74
86
  context.collection = [value]
75
87
  until context.batch_complete?(
@@ -83,10 +95,12 @@ module Shifty
83
95
  end
84
96
  end
85
97
 
86
- def splitter_worker(&block)
87
- ensure_regular_arity(block)
98
+ def splitter_worker(options = {}, &block)
99
+ options[:tags] ||= []
100
+ options[:tags] << :splitter
101
+ ensure_regular_arity!(block)
88
102
 
89
- Worker.new(tags: [:splitter]) do |value|
103
+ Worker.new(options) do |value|
90
104
  if value.nil?
91
105
  value
92
106
  else
@@ -99,9 +113,11 @@ module Shifty
99
113
  end
100
114
  end
101
115
 
116
+ # don't like that this is a second exception to accepting options..
102
117
  def trailing_worker(trail_length = 2)
118
+ options = {tags: [:trailing]}
103
119
  trail = []
104
- Worker.new(tags: [:trailing]) do |value, supply|
120
+ Worker.new(options) do |value, supply|
105
121
  if value
106
122
  trail.unshift value
107
123
  if trail.size >= trail_length
@@ -111,9 +127,12 @@ module Shifty
111
127
  trail.unshift supply.shift
112
128
  end
113
129
 
114
- trail
130
+ # Hand off a snapshot: the builder keeps mutating `trail` across
131
+ # calls, and a downstream :frozen intake would freeze the live
132
+ # closure array in place.
133
+ trail.dup
115
134
  else
116
- value
135
+ value # hint: it's nil!
117
136
  end
118
137
  end
119
138
  end
@@ -124,17 +143,30 @@ module Shifty
124
143
 
125
144
  private
126
145
 
146
+ def deprecate_mode_option!(options)
147
+ return unless options.key?(:mode)
148
+ mode = options.delete(:mode)
149
+ if mode == :hardened
150
+ warn "[shifty] side_worker mode: :hardened is deprecated and will be " \
151
+ "removed in 1.0.0; use policy: :isolated instead."
152
+ options[:policy] ||= :isolated
153
+ else
154
+ warn "[shifty] side_worker's mode: option is deprecated and ignored " \
155
+ "(received mode: #{mode.inspect}); declare a policy: instead."
156
+ end
157
+ end
158
+
127
159
  def throw_with(*msg)
128
160
  raise WorkerInitializationError.new([msg].flatten.join(" "))
129
161
  end
130
162
 
131
- def ensure_callable(callable)
163
+ def ensure_callable!(callable)
132
164
  unless callable&.respond_to?(:call)
133
165
  throw_with "You must supply a callable"
134
166
  end
135
167
  end
136
168
 
137
- def ensure_regular_arity(block)
169
+ def ensure_regular_arity!(block)
138
170
  if block.arity != 1
139
171
  throw_with \
140
172
  "Worker must accept exactly one argument (arity == 1)"
@@ -145,7 +177,7 @@ module Shifty
145
177
  def ensure_correct_arity_for!(argument, block)
146
178
  return unless block
147
179
  if argument
148
- ensure_regular_arity(block)
180
+ ensure_regular_arity!(block)
149
181
  elsif block.arity > 0
150
182
  throw_with \
151
183
  "Source worker cannot accept any arguments (arity == 0)"
@@ -0,0 +1,121 @@
1
+ module Shifty
2
+ class Error < StandardError; end
3
+
4
+ class WorkerError < Error; end
5
+
6
+ class WorkerInitializationError < Error; end
7
+
8
+ class PolicyError < Error
9
+ attr_reader :worker, :policy, :value
10
+
11
+ def cause
12
+ @wrapped_cause || super
13
+ end
14
+
15
+ private
16
+
17
+ def worker_label
18
+ label = (worker.respond_to?(:name) && worker.name) ? "`#{worker.name}`" : "(unnamed)"
19
+ tags = worker.respond_to?(:tags) ? worker.tags : []
20
+ tags&.any? ? "#{label} (tags: #{tags.inspect})" : label
21
+ end
22
+ end
23
+
24
+ # Raised when a task mutates a value it received under a policy that
25
+ # forbids mutation. Wraps the original FrozenError (never masks it) and
26
+ # names the worker, the effective policy, and the object the task tried
27
+ # to mutate, with a heuristic locating that object relative to the
28
+ # handed-off value.
29
+ class PolicyViolation < PolicyError
30
+ attr_reader :receiver
31
+
32
+ def initialize(worker:, policy:, receiver:, value:, cause:)
33
+ @worker = worker
34
+ @policy = policy
35
+ @receiver = receiver
36
+ @value = value
37
+ @wrapped_cause = cause
38
+ super(build_message)
39
+ end
40
+
41
+ private
42
+
43
+ def build_message
44
+ <<~MSG
45
+ Worker #{worker_label} received its value under the #{policy.inspect} handoff \
46
+ policy, and its task attempted to mutate #{receiver_description}.
47
+
48
+ Either make the task non-destructive — e.g. `map` instead of `map!`, \
49
+ `value.with(...)`, `arr + [x]`, `hash.merge(...)` — or declare a different \
50
+ policy on this worker: `policy: :isolated` (task works on a private scratch \
51
+ copy) or `policy: :shared` (raw reference; no protection). \
52
+ More: https://github.com/joelhelbling/shifty/wiki/Handoff-Policies
53
+ MSG
54
+ end
55
+
56
+ def receiver_description
57
+ if receiver.equal?(value)
58
+ "an instance of #{receiver.class} — the handed-off value itself"
59
+ elsif reachable_from_value?
60
+ "an instance of #{receiver.class} reachable from the handed-off value"
61
+ else
62
+ "an instance of #{receiver.class} (#{receiver.inspect[0, 80]}), which may be " \
63
+ "unrelated to the handed-off value (an instance of #{value.class}); " \
64
+ "inspect both to judge"
65
+ end
66
+ end
67
+
68
+ # Bounds the diagnostic graph walk; past this the heuristic gives up
69
+ # and reports the honest "inspect both to judge" fallback rather than
70
+ # risking a SystemStackError that would mask the violation itself.
71
+ MAX_REACHABILITY_NODES = 50_000
72
+
73
+ def reachable_from_value?
74
+ seen = {}.compare_by_identity
75
+ stack = [value]
76
+ until stack.empty?
77
+ node = stack.pop
78
+ next if node.nil? || seen[node]
79
+ return false if seen.size >= MAX_REACHABILITY_NODES
80
+ seen[node] = true
81
+ return true if node.equal?(receiver)
82
+ stack.concat(children_of(node))
83
+ end
84
+ false
85
+ end
86
+
87
+ def children_of(node)
88
+ case node
89
+ when Array then node
90
+ when Hash then node.keys + node.values
91
+ when Struct then node.to_a
92
+ else
93
+ members = (node.respond_to?(:to_h) && node.is_a?(Data)) ? node.to_h.values : []
94
+ members + node.instance_variables.map { |iv| node.instance_variable_get(iv) }
95
+ end
96
+ end
97
+ end
98
+
99
+ # Raised at the handoff itself when a value cannot cross the boundary
100
+ # under the effective policy (cannot be frozen or deep-copied).
101
+ class UnshareableValue < PolicyError
102
+ def initialize(worker:, policy:, value:, cause: nil)
103
+ @worker = worker
104
+ @policy = policy
105
+ @value = value
106
+ @wrapped_cause = cause
107
+ super(build_message)
108
+ end
109
+
110
+ private
111
+
112
+ def build_message
113
+ "Worker #{worker_label} received a value (an instance of #{value.class}) " \
114
+ "that cannot cross the handoff boundary under the #{policy.inspect} policy: " \
115
+ "it cannot be #{(policy == :isolated) ? "deep-copied" : "frozen"}. " \
116
+ "Declare `policy: :shared` on this worker (raw pass-by-reference), " \
117
+ "or restructure the value. " \
118
+ "More: https://github.com/joelhelbling/shifty/wiki/Handoff-Policies"
119
+ end
120
+ end
121
+ end