shifty 0.5.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,127 @@
1
+ # Testing Workers
2
+
3
+ Shifty 0.6.0 ships an opt-in test harness (`require "shifty/testing"`) and RSpec sugar (`require "shifty/rspec"`) so your unit tests exercise workers under the **same handoff policy the production pipeline will** (see [[Handoff-Policies]]). Neither is loaded by `require "shifty"` — they're deliberately opt-in. This page covers the parity problem they solve, `Shifty::Testing.run`, the mutation detector, the RSpec matcher and shared example, and the "test at the ceiling" guidance.
4
+
5
+ ## The parity problem
6
+
7
+ A pipeline runs `:frozen`; a unit test exercises the worker's raw proc with an ordinary mutable object:
8
+
9
+ ```ruby
10
+ worker.task.call(input) # bypasses the framework — and the policy
11
+ ```
12
+
13
+ The test passes; the pipeline raises. The gap closes from two directions:
14
+
15
+ 1. **Worker-level policy declarations win** over pipeline defaults, so the policy travels *with* the worker. Any test that runs the worker through the framework automatically exercises production semantics.
16
+ 2. **The framework path is the convenient path** — that's `Shifty::Testing.run`.
17
+
18
+ With `:frozen` as the global default, the common case requires no test configuration at all.
19
+
20
+ ## `Shifty::Testing.run`
21
+
22
+ ```ruby
23
+ require "shifty/testing"
24
+
25
+ Shifty::Testing.run(worker, inputs:, policy: nil, max_shifts: 10_000)
26
+ ```
27
+
28
+ Feeds the inputs to the worker via a synthetic source and collects its outputs until the end-of-stream sentinel (`nil`). The worker's declared/effective policy governs each handoff, exactly as in production; the harness restores the worker's policy, supply, and Fiber afterward, so it never leaves a mark on the worker under test.
29
+
30
+ ```ruby
31
+ worker = relay_worker { |v| v.upcase }
32
+ Shifty::Testing.run(worker, inputs: ["a", "b", "c"]) #=> ["A", "B", "C"]
33
+
34
+ # Parity with production — a mutating task raises under the default policy:
35
+ mutator = Shifty::Worker.new { |v| v && v << :x }
36
+ Shifty::Testing.run(mutator, inputs: [[:a]]) # raises PolicyViolation
37
+
38
+ # A worker that declared :isolated runs under :isolated:
39
+ scratch = Shifty::Worker.new(policy: :isolated) { |v| v && v << :x }
40
+ Shifty::Testing.run(scratch, inputs: [[:a]]) #=> [[:a, :x]]
41
+
42
+ # Explicit policy: override beats the worker's declaration (policy-matrix testing);
43
+ # the worker's real declaration is untouched afterward:
44
+ loose = Shifty::Worker.new(policy: :shared) { |v| v && v << :x }
45
+ Shifty::Testing.run(loose, inputs: [[:a]], policy: :frozen) # raises PolicyViolation
46
+ loose.effective_policy #=> :shared
47
+
48
+ # Non-1:1 output streams collect naturally:
49
+ evens = filter_worker { |v| v.even? }
50
+ Shifty::Testing.run(evens, inputs: [1, 2, 3, 4]) #=> [2, 4]
51
+ ```
52
+
53
+ ### Sentinel semantics
54
+
55
+ `nil` is Shifty's end-of-stream sentinel; collection stops at the first `nil` output. **`false` is a legitimate payload**, not end-of-stream — it flows through and gets collected:
56
+
57
+ ```ruby
58
+ flipper = relay_worker { |v| !v }
59
+ Shifty::Testing.run(flipper, inputs: [true, false, true])
60
+ #=> [false, false, false]
61
+ # (relay_worker's `value &&` guard passes the false input through untouched.)
62
+ ```
63
+
64
+ If a worker's task converts `nil` into something non-nil, the run would never terminate — so after `max_shifts` (default 10,000) the harness raises a diagnostic `Shifty::Error` telling you to let `nil` pass through (e.g. `value && ...`) or raise `max_shifts:`.
65
+
66
+ ## The mutation detector: `mutates_input?`
67
+
68
+ ```ruby
69
+ Shifty::Testing.mutates_input?(worker, input) #=> true / false
70
+ ```
71
+
72
+ Hands the task a private mutable deep copy of `input`, runs it through the framework under `:shared`, and compares the copy before and after (via `Marshal.dump`). It surfaces mutation **even when the worker's current policy permits or hides it**:
73
+
74
+ ```ruby
75
+ sneaky = side_worker(policy: :isolated) { |v| v << :boo }
76
+ Shifty::Testing.mutates_input?(sneaky, [:a]) #=> true — the :isolated copy hid it in production
77
+
78
+ clean = relay_worker { |v| v + [:x] }
79
+ Shifty::Testing.mutates_input?(clean, [:a]) #=> false
80
+ ```
81
+
82
+ This is exactly the information you need **before** loosening a policy — moving to `:shared` never raises, so the detector is your only warning (the silent-loosening asymmetry; see [[Handoff-Policies]]). It's also the pre-upgrade inventory tool for [[Migration-Guide-0.6]]. The input must be Marshal-dumpable; anything else raises a descriptive `Shifty::Error`.
83
+
84
+ ## RSpec sugar
85
+
86
+ ```ruby
87
+ # spec_helper.rb
88
+ require "shifty/rspec" # pulls in shifty/testing; assumes RSpec is loaded
89
+ ```
90
+
91
+ ### The `mutate_input` matcher
92
+
93
+ Takes the input as an argument and delegates to the mutation detector:
94
+
95
+ ```ruby
96
+ expect(worker).not_to mutate_input([:a])
97
+ ```
98
+
99
+ The negated failure message spells out the consequences: a mutating task is only correct under `:isolated` (mutation stays local) or `:shared` (mutation is intentional); it will raise under `:frozen`.
100
+
101
+ ### The `"a policy-safe worker"` shared example
102
+
103
+ Expects `worker` and `safe_input` to be defined with `let`:
104
+
105
+ ```ruby
106
+ RSpec.describe "my enricher" do
107
+ let(:worker) { relay_worker { |v| v.merge(enriched: true) } }
108
+ let(:safe_input) { {id: 1} }
109
+
110
+ it_behaves_like "a policy-safe worker"
111
+ end
112
+ ```
113
+
114
+ It runs two checks: the worker processes a **deeply frozen** input through the framework without raising (`Testing.run(..., policy: :frozen)`), and it does not mutate its input.
115
+
116
+ ## Test at the ceiling
117
+
118
+ Policy safety is (almost) a one-way ratchet: a task correct on deeply frozen input is correct under every policy; the reverse does not hold. So the guidance is to **test under `:frozen`** — the ceiling — unless the worker explicitly declares mutation as part of its design. Consequences:
119
+
120
+ - Loosening a pipeline's policy can never break a ceiling-tested worker.
121
+ - Tightening is the only breaking direction, and tightening fails loudly with `PolicyViolation` diagnostics.
122
+
123
+ **The "almost":** `:isolated` hands the task a *copy*, so a worker depending on object **identity** (rare, but conceivable with identity-keyed caches) behaves differently there. This is the one spot where strict-passing does not imply lax-passing. If your worker cares about `equal?`, test it under `:isolated` explicitly:
124
+
125
+ ```ruby
126
+ Shifty::Testing.run(worker, inputs: [thing], policy: :isolated)
127
+ ```
@@ -0,0 +1,171 @@
1
+ # Worker Types
2
+
3
+ Shifty's DSL (`include Shifty::DSL`) provides shorthand constructors for the common worker shapes; underneath them all is `Shifty::Worker`, and above them sits `Shifty::Gang` for treating a chain as one unit. This page is the per-type reference for 0.6.0, including how each type behaves under the default `:frozen` handoff policy (see [[Handoff-Policies]]). All DSL constructors except `source_worker` and `trailing_worker` accept an options hash that passes through to `Worker.new` — so `policy:`, `name:`, `tags:` work everywhere.
4
+
5
+ ## `source_worker`
6
+
7
+ At the headwater of every pipeline: a worker that generates its own work items. Its block takes no arguments; `nil` signals end-of-stream (and once a source returns `nil`, it returns `nil` henceforth).
8
+
9
+ ```ruby
10
+ worker = source_worker { "Number 9" } # generates forever
11
+
12
+ counter = 0
13
+ finite = source_worker do # generates until it returns nil
14
+ if counter < 3
15
+ counter += 1
16
+ counter + 1000
17
+ end
18
+ end
19
+
20
+ w1 = source_worker [0, 1, 2] # or hand it a series
21
+ w2 = source_worker (0..2) # ranges work too
22
+ ```
23
+
24
+ A series-based source yields each element and then `nil` forever. Strings are split into characters; a bare scalar becomes a one-element series. You can also combine a series with a block, which acts as a transform: `source_worker([1,2]) { |v| v * 10 }`.
25
+
26
+ Sources are where "mutable within, immutable between" starts: build values freely in closure scope, and hand off snapshots (see [[Coding-Idioms-Under-Frozen]]).
27
+
28
+ ## `relay_worker`
29
+
30
+ The most "normal" worker: accepts a value, returns a transformation of it. `nil` passes through untouched (the end-of-stream sentinel survives).
31
+
32
+ ```ruby
33
+ squarer = relay_worker { |number| number ** 2 }
34
+ pipeline = source_worker(0..3) | squarer
35
+ pipeline.shift #=> 0, 1, 4, 9, nil
36
+ ```
37
+
38
+ Under `:frozen`, relay tasks must be non-destructive — `v.merge(...)`, `v + [...]`, `v.with(...)`. A `v <<` raises `PolicyViolation` at this worker, by name if you gave it one.
39
+
40
+ ## `side_worker`
41
+
42
+ Passes through the value it received while performing a side effect — logging, metrics, stashing. Its purpose is *intentionality*: side effects live in named, removable workers, so pulling one out of the pipeline never changes the pipeline's output.
43
+
44
+ ```ruby
45
+ evens = []
46
+ even_stasher = side_worker { |value| evens << value if value.even? }
47
+ ```
48
+
49
+ Policy behavior is where 0.6.0 makes the side worker's contract real:
50
+
51
+ - **Under `:frozen` (default):** a block that mutates the passed value raises `PolicyViolation`. Observation is enforced.
52
+ - **Under `:isolated`:** the block observes a private scratch copy; since a side worker's return value is discarded, its mutations simply **evaporate** and the untouched value flows on:
53
+
54
+ ```ruby
55
+ source = source_worker [[:foo], [:bar]]
56
+ unsafe = side_worker(policy: :isolated) { |v| v << :boo }
57
+ pipeline = source | unsafe
58
+ pipeline.shift #=> [:foo] — shenanigans contained
59
+ pipeline.shift #=> [:bar]
60
+ ```
61
+
62
+ - **`mode:` is deprecated:** `mode: :hardened` maps to `policy: :isolated` with a warning; any other `mode:` value warns and is ignored. Removed in 1.0.0 — see [[Migration-Guide-0.6]].
63
+
64
+ ## `filter_worker`
65
+
66
+ Passes through only values for which the block is truthy; falsy values are discarded (the worker pulls from its supply until something passes).
67
+
68
+ ```ruby
69
+ evens_only = filter_worker { |value| value % 2 == 0 }
70
+ pipeline = source_worker(0..5) | evens_only
71
+ pipeline.shift #=> 0, 2, 4, nil
72
+ ```
73
+
74
+ Every value the filter pulls mid-task crosses the boundary under the worker's policy — including the ones it discards.
75
+
76
+ ## `batch_worker`
77
+
78
+ Gathers values into batches. Either a fixed size:
79
+
80
+ ```ruby
81
+ batch = batch_worker gathering: 3
82
+ pipeline = source_worker(0..7) | batch
83
+ pipeline.shift #=> [0, 1, 2], then [3, 4, 5], then [6, 7], then nil
84
+ ```
85
+
86
+ (the final batch may be short), or a condition — batch until the block is truthy:
87
+
88
+ ```ruby
89
+ line_reader = batch_worker { |value| value.end_with?("\n") }
90
+ ```
91
+
92
+ ## `splitter_worker`
93
+
94
+ Accepts one value, produces an array from it, and hands off each element successively before asking its supply for more.
95
+
96
+ ```ruby
97
+ splitter = splitter_worker { |value| value.split(" ") }
98
+ pipeline = source_worker(["A bold", "move westward"]) | splitter
99
+ pipeline.shift #=> "A", "bold", "move", "westward", nil
100
+ ```
101
+
102
+ Every part a splitter yields is policy-governed as it crosses into the next worker — under `:frozen`, each part arrives frozen downstream.
103
+
104
+ ## `trailing_worker`
105
+
106
+ Returns an array of the last *n* values — useful for rolling averages. Nothing is returned until *n* values have accumulated; new values are unshifted into position zero.
107
+
108
+ ```ruby
109
+ trailer = trailing_worker 4
110
+ pipeline = source_worker(0..5) | trailer
111
+ pipeline.shift #=> [3, 2, 1, 0], then [4, 3, 2, 1], then [5, 4, 3, 2], then nil
112
+ ```
113
+
114
+ New in 0.6.0: the trailing worker **hands off a snapshot** (`trail.dup`) rather than its live closure array. It keeps mutating that array across calls, and a downstream `:frozen` intake would otherwise freeze the live array in place. This is the canonical example of the builder snapshot rule — see [[Coding-Idioms-Under-Frozen]].
115
+
116
+ (Signature note: `trailing_worker(trail_length = 2)` takes the length positionally and no options hash.)
117
+
118
+ ## Raw `Shifty::Worker.new`
119
+
120
+ When the DSL shapes don't fit, build a worker directly:
121
+
122
+ ```ruby
123
+ worker = Shifty::Worker.new(
124
+ supply: upstream, # or wire later: worker.supply = upstream
125
+ policy: :isolated, # this worker's contract; validated eagerly
126
+ name: "enricher", # used in PolicyViolation / UnshareableValue messages
127
+ tags: [:etl], # also reported in diagnostics
128
+ criteria: ->(w) { ... }, # when falsy, the task is bypassed (value still policy-governed)
129
+ task: some_callable # or pass a block
130
+ ) { |value, supply, context| ... }
131
+ ```
132
+
133
+ The task's arity matters:
134
+
135
+ - **Arity 0** — a source; it cannot accept a supply (`supply=` raises). Use `handoff(value)` (which is `Fiber.yield`) to emit from inside loops.
136
+ - **Arity 1+** — `|value|` receives the policy-governed intake value.
137
+ - **Second argument** — a policy-governed supply proxy responding only to `#shift`, for tasks that pull additional values themselves (this is how filter/batch/trailing work). It is *not* the raw upstream worker.
138
+ - **Third argument** — the worker's `context`, an `OpenStruct` by default or whatever you pass as `context:`; per-worker mutable state that survives across shifts (and survives a rescued `PolicyViolation`).
139
+
140
+ A worker with no task gets a default pass-through task. `worker.effective_policy` reports the resolved policy (own declaration, else pipeline default, else global default).
141
+
142
+ ## Composition: `|`, `with_policy`, `freeze!`
143
+
144
+ ```ruby
145
+ pipeline = source | filter | transform | sink # tail-returned; call shift on it
146
+
147
+ pipeline = (source | filter | transform | sink)
148
+ .with_policy(:frozen) # pipeline default for workers with no declaration
149
+ .freeze! # lock the topology; upstream walk from the tail
150
+ ```
151
+
152
+ Both `with_policy` and `freeze!` walk **upstream** through the supply chain from their receiver and return it, so they chain — and both should be called on the pipeline's tail. `freeze!` locks topology only (supply wiring, Gang rosters); closure and context state stay mutable. Use `#freeze!`, never bare `#freeze`. Details in [[Handoff-Policies]].
153
+
154
+ ## `Shifty::Gang`
155
+
156
+ A Gang wraps an ordered roster of workers so a whole chain acts like one worker: it has a `supply`, a `shift`, and composes with `|` like anything else.
157
+
158
+ ```ruby
159
+ gang = Shifty::Gang[parser, enricher] # or Gang.new([parser, enricher])
160
+ pipeline = reader | gang | writer
161
+ ```
162
+
163
+ Policy features:
164
+
165
+ - **Construction fanout:** `Gang.new(workers, policy: :isolated)` sets the pipeline default on every roster member.
166
+ - **`with_policy` fanout:** `gang.with_policy(:shared)` does the same, chainably.
167
+ - **Append inheritance:** the gang persists its declared policy, so workers appended *after* the declaration inherit it too.
168
+ - **Member contracts win:** a roster member's own `policy:` declaration beats the gang's, per the usual precedence.
169
+ - **Chains walk through:** `(upstream | gang | tail).with_policy(:isolated)` reaches the gang's roster *and* workers upstream of it; `freeze!` walks the same path.
170
+ - **Freezing:** a frozen gang still runs, but its roster membership is locked — `append` raises `FrozenError`.
171
+ - **Criteria bypass is still governed:** when a gang's `criteria` bypasses its workers, the value crossing the gang boundary is still policy-governed at the entry worker.
@@ -0,0 +1,13 @@
1
+ **[[Home]]**
2
+
3
+ *Concepts*
4
+ - [[Handoff-Policies]]
5
+ - [[Coding-Idioms-Under-Frozen]]
6
+ - [[Worker-Types]]
7
+
8
+ *Doing things*
9
+ - [[Migration-Guide-0.6]]
10
+ - [[Testing-Workers]]
11
+
12
+ *Reference*
13
+ - [[Performance]]
@@ -0,0 +1,27 @@
1
+ module Shifty
2
+ class Configuration
3
+ attr_reader :default_policy
4
+
5
+ def initialize
6
+ @default_policy = :frozen
7
+ end
8
+
9
+ def default_policy=(policy_name)
10
+ @default_policy = Policy.validate!(policy_name)
11
+ end
12
+ end
13
+
14
+ class << self
15
+ def config
16
+ @config ||= Configuration.new
17
+ end
18
+
19
+ def configure
20
+ yield config
21
+ end
22
+
23
+ def reset_configuration!
24
+ @config = Configuration.new
25
+ end
26
+ end
27
+ end
data/lib/shifty/dsl.rb CHANGED
@@ -1,6 +1,4 @@
1
1
  module Shifty
2
- class WorkerInitializationError < StandardError; end
3
-
4
2
  module DSL
5
3
  def source_worker(argument = nil, &block)
6
4
  ensure_correct_arity_for!(argument, block)
@@ -22,7 +20,7 @@ module Shifty
22
20
  def relay_worker(options = {}, &block)
23
21
  options[:tags] ||= []
24
22
  options[:tags] << :relay
25
- ensure_regular_arity(block)
23
+ ensure_regular_arity!(block)
26
24
 
27
25
  Worker.new(options) do |value|
28
26
  value && block.call(value)
@@ -32,17 +30,24 @@ module Shifty
32
30
  def side_worker(options = {}, &block)
33
31
  options[:tags] ||= []
34
32
  options[:tags] << :side_effect
35
- mode = options[:mode] || :normal
36
- ensure_regular_arity(block)
33
+ deprecate_mode_option!(options)
34
+ ensure_regular_arity!(block)
37
35
 
38
- Worker.new(options) do |value|
36
+ # The block must ask the worker for its policy at shift time, so the
37
+ # local is captured by the closure before it is assigned. Not redundant.
38
+ worker = nil
39
+ worker = Worker.new(options) do |value| # standard:disable Style/RedundantAssignment
39
40
  value.tap do |v|
40
- used_value = (mode == :hardened) ?
41
- Marshal.load(Marshal.dump(v)) : v
41
+ next unless v
42
+ # Under :isolated the block observes a private scratch copy, so
43
+ # its mutations evaporate and the untouched value flows on.
44
+ used_value = (worker.effective_policy == :isolated) ?
45
+ Policy::Isolated.call(v, worker: worker) : v
42
46
 
43
- v && block.call(used_value)
47
+ block.call(used_value)
44
48
  end
45
49
  end
50
+ worker
46
51
  end
47
52
 
48
53
  def filter_worker(options = {}, &block)
@@ -70,7 +75,7 @@ module Shifty
70
75
  options[:tags] << :batch
71
76
  options[:gathering] ||= 1
72
77
 
73
- ensure_regular_arity(block) if block
78
+ ensure_regular_arity!(block) if block
74
79
  batch_full = block ||
75
80
  proc { |_, batch| batch.size >= options[:gathering] }
76
81
 
@@ -93,7 +98,7 @@ module Shifty
93
98
  def splitter_worker(options = {}, &block)
94
99
  options[:tags] ||= []
95
100
  options[:tags] << :splitter
96
- ensure_regular_arity(block)
101
+ ensure_regular_arity!(block)
97
102
 
98
103
  Worker.new(options) do |value|
99
104
  if value.nil?
@@ -122,7 +127,10 @@ module Shifty
122
127
  trail.unshift supply.shift
123
128
  end
124
129
 
125
- trail
130
+ # Hand off a snapshot: the builder keeps mutating `trail` across
131
+ # calls, and a downstream :frozen intake would freeze the live
132
+ # closure array in place.
133
+ trail.dup
126
134
  else
127
135
  value # hint: it's nil!
128
136
  end
@@ -135,6 +143,19 @@ module Shifty
135
143
 
136
144
  private
137
145
 
146
+ def deprecate_mode_option!(options)
147
+ return unless options.key?(:mode)
148
+ mode = options.delete(:mode)
149
+ if mode == :hardened
150
+ warn "[shifty] side_worker mode: :hardened is deprecated and will be " \
151
+ "removed in 1.0.0; use policy: :isolated instead."
152
+ options[:policy] ||= :isolated
153
+ else
154
+ warn "[shifty] side_worker's mode: option is deprecated and ignored " \
155
+ "(received mode: #{mode.inspect}); declare a policy: instead."
156
+ end
157
+ end
158
+
138
159
  def throw_with(*msg)
139
160
  raise WorkerInitializationError.new([msg].flatten.join(" "))
140
161
  end
@@ -145,7 +166,7 @@ module Shifty
145
166
  end
146
167
  end
147
168
 
148
- def ensure_regular_arity(block)
169
+ def ensure_regular_arity!(block)
149
170
  if block.arity != 1
150
171
  throw_with \
151
172
  "Worker must accept exactly one argument (arity == 1)"
@@ -156,7 +177,7 @@ module Shifty
156
177
  def ensure_correct_arity_for!(argument, block)
157
178
  return unless block
158
179
  if argument
159
- ensure_regular_arity(block)
180
+ ensure_regular_arity!(block)
160
181
  elsif block.arity > 0
161
182
  throw_with \
162
183
  "Source worker cannot accept any arguments (arity == 0)"
@@ -0,0 +1,121 @@
1
+ module Shifty
2
+ class Error < StandardError; end
3
+
4
+ class WorkerError < Error; end
5
+
6
+ class WorkerInitializationError < Error; end
7
+
8
+ class PolicyError < Error
9
+ attr_reader :worker, :policy, :value
10
+
11
+ def cause
12
+ @wrapped_cause || super
13
+ end
14
+
15
+ private
16
+
17
+ def worker_label
18
+ label = (worker.respond_to?(:name) && worker.name) ? "`#{worker.name}`" : "(unnamed)"
19
+ tags = worker.respond_to?(:tags) ? worker.tags : []
20
+ tags&.any? ? "#{label} (tags: #{tags.inspect})" : label
21
+ end
22
+ end
23
+
24
+ # Raised when a task mutates a value it received under a policy that
25
+ # forbids mutation. Wraps the original FrozenError (never masks it) and
26
+ # names the worker, the effective policy, and the object the task tried
27
+ # to mutate, with a heuristic locating that object relative to the
28
+ # handed-off value.
29
+ class PolicyViolation < PolicyError
30
+ attr_reader :receiver
31
+
32
+ def initialize(worker:, policy:, receiver:, value:, cause:)
33
+ @worker = worker
34
+ @policy = policy
35
+ @receiver = receiver
36
+ @value = value
37
+ @wrapped_cause = cause
38
+ super(build_message)
39
+ end
40
+
41
+ private
42
+
43
+ def build_message
44
+ <<~MSG
45
+ Worker #{worker_label} received its value under the #{policy.inspect} handoff \
46
+ policy, and its task attempted to mutate #{receiver_description}.
47
+
48
+ Either make the task non-destructive — e.g. `map` instead of `map!`, \
49
+ `value.with(...)`, `arr + [x]`, `hash.merge(...)` — or declare a different \
50
+ policy on this worker: `policy: :isolated` (task works on a private scratch \
51
+ copy) or `policy: :shared` (raw reference; no protection). \
52
+ More: https://github.com/joelhelbling/shifty/wiki/Handoff-Policies
53
+ MSG
54
+ end
55
+
56
+ def receiver_description
57
+ if receiver.equal?(value)
58
+ "an instance of #{receiver.class} — the handed-off value itself"
59
+ elsif reachable_from_value?
60
+ "an instance of #{receiver.class} reachable from the handed-off value"
61
+ else
62
+ "an instance of #{receiver.class} (#{receiver.inspect[0, 80]}), which may be " \
63
+ "unrelated to the handed-off value (an instance of #{value.class}); " \
64
+ "inspect both to judge"
65
+ end
66
+ end
67
+
68
+ # Bounds the diagnostic graph walk; past this the heuristic gives up
69
+ # and reports the honest "inspect both to judge" fallback rather than
70
+ # risking a SystemStackError that would mask the violation itself.
71
+ MAX_REACHABILITY_NODES = 50_000
72
+
73
+ def reachable_from_value?
74
+ seen = {}.compare_by_identity
75
+ stack = [value]
76
+ until stack.empty?
77
+ node = stack.pop
78
+ next if node.nil? || seen[node]
79
+ return false if seen.size >= MAX_REACHABILITY_NODES
80
+ seen[node] = true
81
+ return true if node.equal?(receiver)
82
+ stack.concat(children_of(node))
83
+ end
84
+ false
85
+ end
86
+
87
+ def children_of(node)
88
+ case node
89
+ when Array then node
90
+ when Hash then node.keys + node.values
91
+ when Struct then node.to_a
92
+ else
93
+ members = (node.respond_to?(:to_h) && node.is_a?(Data)) ? node.to_h.values : []
94
+ members + node.instance_variables.map { |iv| node.instance_variable_get(iv) }
95
+ end
96
+ end
97
+ end
98
+
99
+ # Raised at the handoff itself when a value cannot cross the boundary
100
+ # under the effective policy (cannot be frozen or deep-copied).
101
+ class UnshareableValue < PolicyError
102
+ def initialize(worker:, policy:, value:, cause: nil)
103
+ @worker = worker
104
+ @policy = policy
105
+ @value = value
106
+ @wrapped_cause = cause
107
+ super(build_message)
108
+ end
109
+
110
+ private
111
+
112
+ def build_message
113
+ "Worker #{worker_label} received a value (an instance of #{value.class}) " \
114
+ "that cannot cross the handoff boundary under the #{policy.inspect} policy: " \
115
+ "it cannot be #{(policy == :isolated) ? "deep-copied" : "frozen"}. " \
116
+ "Declare `policy: :shared` on this worker (raw pass-by-reference), " \
117
+ "or restructure the value. " \
118
+ "More: https://github.com/joelhelbling/shifty/wiki/Handoff-Policies"
119
+ end
120
+ end
121
+ end
data/lib/shifty/gang.rb CHANGED
@@ -6,18 +6,31 @@ module Shifty
6
6
  attr_reader :roster, :tags
7
7
 
8
8
  include Taggable
9
+ include PolicyDeclarable
9
10
 
10
11
  def initialize(workers = [], p = {})
11
12
  @roster = Roster.new(workers)
12
13
  self.criteria = p[:criteria]
13
14
  self.tags = p[:tags]
15
+ self.pipeline_policy = Policy.validate!(p[:policy]) if p[:policy]
14
16
  end
15
17
 
18
+ def pipeline_policy=(policy_name)
19
+ # Persisted so workers appended after the declaration inherit it.
20
+ @pipeline_policy = policy_name
21
+ roster.workers.each { |w| w.pipeline_policy = policy_name }
22
+ end
23
+
24
+ attr_reader :pipeline_policy
25
+
16
26
  def shift
17
27
  if criteria_passes?
18
28
  roster.last.shift
19
29
  else
20
- roster.first.supply.shift
30
+ # Even when the gang's criteria bypasses its workers, the value
31
+ # still crosses the gang's boundary — govern it with the entry
32
+ # worker's policy, matching Worker#shift's bypass behavior.
33
+ roster.first.intake(roster.first.supply.shift)
21
34
  end
22
35
  end
23
36
 
@@ -26,7 +39,7 @@ module Shifty
26
39
  end
27
40
 
28
41
  def supply
29
- roster.first.supply
42
+ roster.first&.supply
30
43
  end
31
44
 
32
45
  def supply=(supplier)
@@ -41,6 +54,16 @@ module Shifty
41
54
 
42
55
  def append(worker)
43
56
  roster << worker
57
+ worker.pipeline_policy = pipeline_policy if pipeline_policy
58
+ end
59
+
60
+ # Freezes every roster member plus the roster's own membership, so
61
+ # append/push/pop/shift/unshift all raise FrozenError afterward.
62
+ def freeze_topology_node!
63
+ roster.workers.each(&:freeze_topology_node!)
64
+ roster.workers.freeze
65
+ roster.freeze
66
+ freeze
44
67
  end
45
68
 
46
69
  class << self