shifty 0.5.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: b2d09c087eca2d3c2790efab38c0e12230edd9738daf7e43c6788a8671c64356
4
- data.tar.gz: 98fe18da84e2889e1b0c1bb90ea41ffcb6928e50868033fa33627bce58482052
3
+ metadata.gz: 8e7ad29ca6b77f1216ed40460e252d48743f6b6786a8adbd1e277fc673115829
4
+ data.tar.gz: ab8d5fced324f9b7d1834ac9c1741428122de18acd03f02d1e7d550d8f190879
5
5
  SHA512:
6
- metadata.gz: 0fb0046526de78394da2a742f9c9d41d07d58717592f0ce40f13d32e6cdf8e2c0c42d30fffe2bc53e164398422639c5f42943c0e7edb064eb0ec8c21960fe833
7
- data.tar.gz: 45792b0187073a88781217d1559afc185471bbdbd849c233af81055a9dc5d71ebf8bbd018a3224baa4d0a32795109802eadfcfe9dba754f964907154dabc0e4c
6
+ metadata.gz: f7884214ed282f0d200d18391989c04af01493f2f17477e2b0f4fe69ebe782e1401fcaed5dcd40561f23f4e05831b79dd09b50e8b09d422262de7af1310e5ec3
7
+ data.tar.gz: 67c9354307d51baf7628b4079cf0f86dd9d80e7444d6c286fc65d6dd3df6316c9f24d62190281ba127700d2f64562d716cedec5fa5f605d337d81639e1cd37f8
data/.gitignore CHANGED
@@ -16,3 +16,4 @@
16
16
  Gemfile.lock
17
17
 
18
18
  shifty-*.gem
19
+ .secrets
data/CHANGELOG.md ADDED
@@ -0,0 +1,79 @@
1
+ # Changelog
2
+
3
+ ## 0.6.0
4
+
5
+ The handoff immutability release. Values crossing worker boundaries are now
6
+ governed by a **handoff policy**, and the default changed from raw shared
7
+ references to deeply frozen values. This is a breaking change — see the
8
+ [Migration Guide](https://github.com/joelhelbling/shifty/wiki/Migration-Guide-0.6).
9
+
10
+ ### Breaking changes
11
+
12
+ - **`:frozen` is the new default handoff policy.** Every value a worker
13
+ receives is deeply frozen (`Ractor.make_shareable`) at intake. A task that
14
+ mutates its input now raises `Shifty::PolicyViolation` at the offending
15
+ worker — immediately and with diagnostics — instead of silently corrupting
16
+ what downstream workers observe. Restore the old behavior per worker with
17
+ `policy: :shared`, per pipeline with `.with_policy(:shared)`, or globally
18
+ with `Shifty.configure { |c| c.default_policy = :shared }`.
19
+ - **Unshareable values can no longer cross a default-policy boundary.**
20
+ IO handles, procs, lazy enumerators, and (under `:isolated`) anything
21
+ Marshal rejects raise `Shifty::UnshareableValue` with guidance. Declare
22
+ `policy: :shared` on workers that pass such values. Note: an IO *nested
23
+ inside* a container is not detected — declare `:shared` for those workers.
24
+ - **The task's second argument is now a policy-governed supply proxy** that
25
+ responds to `#shift` only, no longer the raw upstream worker object.
26
+ - **Ruby floor is 3.2** (`required_ruby_version >= 3.2`). Older Rubies stay
27
+ on shifty 0.5.0.
28
+ - `Shifty::WorkerError` and `Shifty::WorkerInitializationError` are
29
+ reparented under a new `Shifty::Error` base (still `StandardError`s).
30
+
31
+ ### Deprecations
32
+
33
+ - **`side_worker mode: :hardened` is deprecated** (removed in 1.0.0); it maps
34
+ to `policy: :isolated` with a warning. Semantics preserved — both use a
35
+ Marshal deep copy. Any other `mode:` value now warns and is ignored.
36
+
37
+ ### Added
38
+
39
+ - **Handoff policies**: `:frozen` (default), `:isolated` (private mutable
40
+ deep copy; task mutations stay local), `:shared` (raw reference — the
41
+ escape hatch). Declarable per worker (`policy:` kwarg), per pipeline
42
+ (`.with_policy(...)` on a chain or Gang, `policy:` kwarg on Gang), and
43
+ globally. Precedence: worker > pipeline > global.
44
+ - **Rich diagnostics**: `Shifty::PolicyViolation` (worker, policy, receiver,
45
+ value, cause; a heuristic reports whether the mutated object is the
46
+ handed-off value, reachable from it, or possibly unrelated) and
47
+ `Shifty::UnshareableValue`. A rescued `PolicyViolation` does not kill the
48
+ pipeline — the next `shift` continues with the next value.
49
+ - **Worker `name:` kwarg** for diagnostics.
50
+ - **`Shifty::Testing`** (opt-in `require "shifty/testing"`):
51
+ `Testing.run(worker, inputs:, policy: nil)` runs a worker through the
52
+ framework under its production policy; `Testing.mutates_input?` is the
53
+ mutation detector — the pre-upgrade migration inventory tool.
54
+ - **RSpec sugar** (opt-in `require "shifty/rspec"`): the `mutate_input`
55
+ matcher and the `"a policy-safe worker"` shared example.
56
+ - **`#freeze!`** locks an assembled pipeline's topology (call it on the
57
+ tail). Rewiring — `supply=`, Gang `append`, roster mutation — raises
58
+ `FrozenError`. Worker closure/context state stays mutable.
59
+ - **Benchmark suite** (`benchmark/handoff_policies.rb`, manual-run) with
60
+ results in `benchmark/RESULTS.md` and on the wiki: steady-state `:frozen`
61
+ costs ~71ns per handoff regardless of value size, with zero allocations.
62
+ - **In-depth documentation** on the
63
+ [GitHub wiki](https://github.com/joelhelbling/shifty/wiki): policies,
64
+ coding idioms under `:frozen`, migration, testing, worker types,
65
+ performance.
66
+
67
+ ### Fixed
68
+
69
+ - `trailing_worker` handed off its live internal array; under `:frozen` this
70
+ is the exact aliasing bug the policies exist to catch. It now hands off a
71
+ snapshot.
72
+ - `ostruct` is declared as a runtime dependency (no longer a default gem as
73
+ of Ruby 4).
74
+ - CI now tests Ruby 3.2/3.3/3.4 (was 2.6/2.7/3.0); dropped the abandoned
75
+ codeclimate-test-reporter (which pinned simplecov to a 2016 release).
76
+
77
+ ## 0.5.0 and earlier
78
+
79
+ See the [release history](https://github.com/joelhelbling/shifty/releases).
data/README.md CHANGED
@@ -1,12 +1,65 @@
1
1
  [![Gem Version](https://badge.fury.io/rb/shifty.svg)](https://badge.fury.io/rb/shifty)
2
2
  [![Tests](https://github.com/joelhelbling/shifty/actions/workflows/ruby.yml/badge.svg)](https://github.com/joelhelbling/shifty/actions/workflows/ruby.yml)
3
- [![Maintainability](https://api.codeclimate.com/v1/badges/950ded888350c1124348/maintainability)](https://codeclimate.com/github/joelhelbling/shifty/maintainability)
4
- [![Test Coverage](https://api.codeclimate.com/v1/badges/950ded888350c1124348/test_coverage)](https://codeclimate.com/github/joelhelbling/shifty/test_coverage)
5
3
 
6
4
  # The Shifty Framework
7
5
 
8
6
  _"How many Ruby fibers does it take to screw in a lightbulb?"_
9
7
 
8
+ Shifty requires **Ruby 3.2 or newer** as of version 0.6.0. On older Rubies,
9
+ use shifty 0.5.0 (which also retains the classic mutable-handoff behavior).
10
+
11
+ ## Concurrency Model
12
+
13
+ Shifty utilizes Ruby Fibers for cooperative multitasking. This means that all tasks (or "workers") run within a single operating system thread and explicitly yield control to one another. This model is intentionally chosen for its simplicity, which makes it easier to reason about and build sequential data processing pipelines.
14
+
15
+ Single-threading frees you from *preemptive* concurrency hazards (races, mutexes) within a pipeline. It does not, by itself, make the data passing between workers safe — a worker that mutates a value it was handed could corrupt what later workers see. As of 0.6.0, those handoffs are governed by [handoff immutability policies](#handoff-policies-new-in-060). The Fiber model is also deliberately compatible with a future Ractor-backed worker type, rather than a rejection of parallelism — and frozen, shareable handoff values are exactly what Ractor boundaries want.
16
+
17
+ For a more detailed explanation of Shifty's design and typical use cases, please see the [Use Cases Document](docs/use_cases.md).
18
+
19
+ ## Handoff Policies (new in 0.6.0)
20
+
21
+ Shifty workers are easy to reason about because each one is isolated; the
22
+ values flowing *between* them were, until now, the un-governed part of the
23
+ system. As of 0.6.0, values are **deeply frozen at every handoff by
24
+ default**. Workers that need a private scratch copy can declare
25
+ `policy: :isolated`; workers that genuinely need shared mutable references
26
+ can declare `policy: :shared`. Mutation bugs that used to surface as
27
+ mysterious downstream corruption now either cannot happen or raise
28
+ immediately at the worker responsible — with an error message that tells
29
+ you exactly what to do about it.
30
+
31
+ ```ruby
32
+ # Per worker — part of the worker's contract:
33
+ scratch = Shifty::Worker.new(policy: :isolated) { |v| v << transform(v) }
34
+
35
+ # Per pipeline — the default for workers that don't declare their own:
36
+ pipeline = (source | parser | sink).with_policy(:shared)
37
+
38
+ # Globally (the built-in default is :frozen):
39
+ Shifty.configure { |c| c.default_policy = :shared }
40
+ ```
41
+
42
+ | | `:frozen` (default) | `:isolated` | `:shared` |
43
+ |---|---|---|---|
44
+ | The task receives | frozen reference | private mutable deep copy | the raw reference |
45
+ | Mutation in the task | raises `Shifty::PolicyViolation` | works, stays local | works, leaks |
46
+ | Copying per handoff | none | full graph | none |
47
+
48
+ ### Upgrading from 0.5.0?
49
+
50
+ Start with the [Migration Guide](https://github.com/joelhelbling/shifty/wiki/Migration-Guide-0.6) —
51
+ it names everything that breaks and gives four migration paths in order of
52
+ preference. `Shifty::Testing.mutates_input?` will inventory which of your
53
+ workers mutate their input *before* you flip the switch.
54
+
55
+ There's much more in the wiki: [Handoff Policies](https://github.com/joelhelbling/shifty/wiki/Handoff-Policies) in depth,
56
+ [coding idioms under :frozen](https://github.com/joelhelbling/shifty/wiki/Coding-Idioms-Under-Frozen),
57
+ the [0.6 migration guide](https://github.com/joelhelbling/shifty/wiki/Migration-Guide-0.6),
58
+ [testing workers](https://github.com/joelhelbling/shifty/wiki/Testing-Workers), and
59
+ [performance numbers](https://github.com/joelhelbling/shifty/wiki/Performance)
60
+ (spoiler: at steady state, `:frozen` costs about 71 nanoseconds per handoff
61
+ no matter how big the value is).
62
+
10
63
  ## Quick Start
11
64
 
12
65
  Add this line to your application's Gemfile:
@@ -134,7 +187,7 @@ work.
134
187
  source = source_worker (0..3)
135
188
 
136
189
  evens = []
137
- even_stasher = side_effect do |value|
190
+ even_stasher = side_worker do |value|
138
191
  if value % 2 == 0
139
192
  evens << value
140
193
  end
@@ -163,10 +216,10 @@ it is, there really isn't a way to prevent the implementation of any
163
216
  worker from creating side effects.
164
217
 
165
218
  _And still wait,_ you'll also be wanting to say, _isn't it possible
166
- that a side worker could mutate the value as it's passed through?_ And
167
- again, yes. It would be very difficult\* in the Ruby language (where
168
- so many things are passed by reference) to perfectly prevent a side
169
- worker from mutating the value. But please don't.
219
+ that a side worker could mutate the value as it's passed through?_
220
+ As of 0.6.0: no, not by accident. A side worker's contract is
221
+ "observe, don't modify," and the default `:frozen` handoff policy
222
+ finally enforces it.
170
223
 
171
224
  The side effect worker's purpose is to provide _intentionality_ and
172
225
  clarity. When you're creating a side effect, let it be very clear.
@@ -178,55 +231,40 @@ same token, if there is a problem with a side effect, troubleshooting
178
231
  it will be much simpler if the side effects are already isolated and
179
232
  named.
180
233
 
181
- Another measure for preventing side workers from creating unwanted
182
- side-effects is to use `:hardened` mode. That's right, `side_worker`
183
- has a mode (which defaults to `:normal`). When set to `:hardened`,
184
- each value is `Marshal.dump`ed and `Marshal.load`ed before being
185
- passed to the side worker's callable. This helps to ensure that
186
- the original value will be passed through to the next worker in the
187
- queue in a pristine state (unmodified), and that no future or
188
- subsequent modification can take place.
189
-
190
234
  Given a source...
191
235
 
192
236
  ```ruby
193
237
  source = source_worker [[:foo], [:bar]]
194
238
  ```
195
239
 
196
- Consider this unsafe, unhardened side worker:
240
+ ...consider this mutating side worker:
197
241
 
198
242
  ```ruby
199
243
  unsafe = side_worker { |v| v << :boo }
200
- ```
201
-
202
- This produces unwanted mutations, because of the `<<`.
203
244
 
204
- ```ruby
205
245
  pipeline = source | unsafe
206
246
 
207
- pipeline.shift #=> [:foo, :boo] <-- Oh noes!
208
- pipeline.shift #=> [:bar, :boo] <-- Disaster!
209
- pipeline.shift #=> nil
247
+ pipeline.shift #=> raises Shifty::PolicyViolation, naming this worker
210
248
  ```
211
249
 
212
- Let's harden it:
250
+ Under the default `:frozen` policy that mutation raises immediately,
251
+ at the offending worker. If the side worker's mutations are harmless
252
+ scratch work you'd rather keep, give it a private copy instead:
213
253
 
214
254
  ```ruby
215
- unsafe = side_worker(:hardened) { |v| v << :boo }
216
- ```
217
-
218
- The `:hardened` side worker doesn't have access to the original
219
- value, and therefore its shenanigans are moot with respect to
220
- the main workflow:
255
+ unsafe = side_worker(policy: :isolated) { |v| v << :boo }
221
256
 
222
- ```ruby
223
257
  pipeline = source | unsafe
224
258
 
225
- pipeline.shift #=> [:foo] <-- No changes
226
- pipeline.shift #=> [:bar] <-- Much better
259
+ pipeline.shift #=> [:foo] <-- mutations evaporate
260
+ pipeline.shift #=> [:bar] <-- the original flows on, pristine
227
261
  pipeline.shift #=> nil
228
262
  ```
229
263
 
264
+ (Old-timers: `side_worker mode: :hardened` still works, mapped to
265
+ `policy: :isolated` with a deprecation warning; it will be removed in
266
+ 1.0.0.)
267
+
230
268
  ### Filter Worker
231
269
 
232
270
  The filter worker simply passes through the values which are given
@@ -453,7 +491,13 @@ example).
453
491
  ## Shifty::Worker
454
492
 
455
493
  The Shifty::Worker documentation (which was the old README) is now
456
- [here](docs/shifty/worker.md).
494
+ [here](docs/shifty/worker.md), and the
495
+ [wiki](https://github.com/joelhelbling/shifty/wiki) goes deeper on every
496
+ worker type, handoff policies, testing, migration, and performance.
457
497
 
458
498
  ## Roadmap
459
499
 
500
+ - **1.0.0** — remove the deprecated `:hardened` alias; stabilize the API.
501
+ - **Someday** — a Ractor-backed worker type for true parallelism; the
502
+ `:frozen` policy's shareable values are deliberately the groundwork.
503
+
@@ -0,0 +1,69 @@
1
+ # Handoff policy benchmark results
2
+
3
+ Run: `bundle exec ruby benchmark/handoff_policies.rb`
4
+ Environment: Ruby 4.0.5 (arm64-darwin25), Apple Silicon. 2026-07-10.
5
+ (Re-run on your own hardware; relative ordering is what matters.)
6
+
7
+ ## Method note (read first)
8
+
9
+ Rows labeled *fresh value* construct a new value inside each iteration, so
10
+ they include construction cost — that keeps `:frozen`'s freeze bit honest
11
+ (freezing is once-per-object; a pre-built value would be frozen after the
12
+ first iteration). The *steady state* row uses a pre-built, already-shareable
13
+ value and measures the policy call alone. Compare fresh rows against the
14
+ `:shared` fresh row (same construction overhead); read the steady-state row
15
+ as the per-handoff cost once a value is shareable.
16
+
17
+ `:hardened` (legacy) has no separate row: its mechanism — a Marshal
18
+ round-trip — is exactly what `:isolated` uses, so the `:isolated` rows are
19
+ its numbers.
20
+
21
+ ## Per-handoff throughput (i/s; higher is better)
22
+
23
+ | Shape | `:shared` (fresh) | `:frozen` first handoff (fresh) | `:frozen` steady state | `:isolated` (fresh) | `:isolated` on already-frozen (§8.3) |
24
+ |---|---|---|---|---|---|
25
+ | small string token | 10.56M | 5.02M | **14.04M** | 1.44M | 1.57M |
26
+ | array, 100 strings | 140.8k | 84.3k | **14.12M** | 34.5k | 46.6k |
27
+ | hash, 100 pairs | 48.6k | 38.4k | **13.87M** | 14.1k | 20.4k |
28
+ | nested document, ~5k nodes | 9.9k | 5.3k | **13.74M** | 1.9k | 2.4k |
29
+ | deep nesting, 500 levels | 49.0k | 21.3k | **13.99M** | 8.5k | 10.8k |
30
+
31
+ ## Allocations per handoff (GC pressure)
32
+
33
+ | Shape | `:shared` | `:frozen` (steady) | `:isolated` |
34
+ |---|---|---|---|
35
+ | small string token | 0 | 0 | 5 |
36
+ | array, 100 strings | 0 | 0 | 105 |
37
+ | hash, 100 pairs | 0 | 0 | 205 |
38
+ | nested document | 0 | 0 | 1,711 |
39
+ | deep nesting, 500 levels | 0 | 0 | 504 |
40
+
41
+ ## Findings
42
+
43
+ 1. **The §8.2 amortization claim holds.** Once a value is shareable,
44
+ `:frozen`'s per-handoff cost is a flat ~71ns **independent of value
45
+ size** (13.7–14.1M i/s across every shape) with zero allocations.
46
+ In an N-worker pipeline, only the first boundary pays the freeze
47
+ traversal; boundaries 2..N are effectively free. Combined with
48
+ copy-on-write task code, per-boundary cost is proportional to the
49
+ *change*, not the value.
50
+ 2. **First-handoff freeze costs roughly 0.6–1.9× the value's own
51
+ construction cost** (e.g. ~4.8µs extra for a 100-string array that
52
+ costs ~7.1µs to build) — paid once per object graph, not per hop.
53
+ 3. **`:isolated` is the expensive policy, as designed**: 3–7× slower than
54
+ `:shared` per boundary *per hop*, and the only policy that allocates
55
+ (a full copy of the graph per boundary — 1,711 objects per handoff for
56
+ the ~5k-node document). This is why it is not the default.
57
+ 4. **§8.3 fast-path decision: keep option (b) — no fast path.** The
58
+ "already-frozen" column is the pure Marshal cost on a pre-built value
59
+ (no construction overhead): 92µs for the deep-nested shape, 21µs for
60
+ the mid array, versus ~71ns if a `Ractor.shareable?` check skipped
61
+ the copy — so a fast path would make those handoffs roughly three
62
+ orders of magnitude cheaper *when inputs are already shareable*.
63
+ It still loses: it would hand the task a frozen object where the
64
+ `:isolated` contract promises a mutable scratch copy. Contract
65
+ simplicity wins at this gem's scale. Reopen trigger: a real workload
66
+ where `:isolated` boundaries dominate and their inputs are typically
67
+ already shareable.
68
+ 5. **The `:frozen` default is vindicated**: it is the only policy that is
69
+ simultaneously safe and, at steady state, the cheapest of the three.
@@ -0,0 +1,82 @@
1
+ # Handoff policy benchmarks (spec §8.4). Manual-run, not CI:
2
+ #
3
+ # bundle exec ruby benchmark/handoff_policies.rb
4
+ #
5
+ # Measures, per policy and value shape:
6
+ # 1. per-handoff cost of :shared / :frozen / :isolated / legacy :hardened
7
+ # (Marshal), including :frozen's first-handoff vs steady-state cost
8
+ # (the §8.2 amortization claim), and
9
+ # 2. the pure Marshal cost :isolated pays on an already-shareable
10
+ # value — the copy a Ractor.shareable? fast path would skip
11
+ # entirely (the §8.3 open question), and
12
+ # 3. allocation counts per policy (GC pressure).
13
+ #
14
+ # Results feed the wiki Performance page.
15
+
16
+ require "bundler/setup"
17
+ require "shifty"
18
+ require "benchmark/ips"
19
+
20
+ SHAPES = {
21
+ "small token (string)" => -> { +"a token" },
22
+ "mid array (100 strings)" => -> { Array.new(100) { |i| "item-#{i}" } },
23
+ "mid hash (100 pairs)" => -> { Array.new(100) { |i| ["key-#{i}", "value-#{i}"] }.to_h },
24
+ "large document (nested, ~5k nodes)" => -> {
25
+ Array.new(50) do |i|
26
+ {"id" => i, "name" => "record-#{i}",
27
+ "tags" => Array.new(10) { |t| "tag-#{t}" },
28
+ "children" => Array.new(10) { |c| {"idx" => c, "payload" => "data-#{i}-#{c}"} }}
29
+ end
30
+ },
31
+ "deep nesting (500 levels)" => -> {
32
+ (1..500).reduce([]) { |acc, i| [i, acc] }
33
+ }
34
+ }
35
+
36
+ null_worker = Object.new
37
+ def null_worker.name = "bench"
38
+
39
+ def null_worker.tags = []
40
+
41
+ puts "Ruby #{RUBY_VERSION} — #{RUBY_PLATFORM}"
42
+ puts
43
+
44
+ SHAPES.each do |label, build|
45
+ puts "=" * 72
46
+ puts "SHAPE: #{label}"
47
+ puts "=" * 72
48
+
49
+ Benchmark.ips do |x|
50
+ x.report(":shared") do
51
+ Shifty::Policy::Shared.call(build.call, worker: null_worker)
52
+ end
53
+ x.report(":frozen (first handoff — fresh value each time)") do
54
+ Shifty::Policy::Frozen.call(build.call, worker: null_worker)
55
+ end
56
+ frozen_value = Ractor.make_shareable(build.call)
57
+ x.report(":frozen (steady state — already shareable)") do
58
+ Shifty::Policy::Frozen.call(frozen_value, worker: null_worker)
59
+ end
60
+ x.report(":isolated (Marshal deep copy)") do
61
+ Shifty::Policy::Isolated.call(build.call, worker: null_worker)
62
+ end
63
+ x.report(":isolated on already-frozen value (§8.3 fast-path question)") do
64
+ Shifty::Policy::Isolated.call(frozen_value, worker: null_worker)
65
+ end
66
+ x.compare!
67
+ end
68
+
69
+ # Allocation pressure per single handoff
70
+ value = build.call
71
+ frozen_value = Ractor.make_shareable(build.call)
72
+ {":shared" => [Shifty::Policy::Shared, value],
73
+ ":frozen (steady)" => [Shifty::Policy::Frozen, frozen_value],
74
+ ":isolated" => [Shifty::Policy::Isolated, value]}.each do |name, (policy, v)|
75
+ GC.start
76
+ before = GC.stat(:total_allocated_objects)
77
+ 100.times { policy.call(v, worker: null_worker) }
78
+ allocated = GC.stat(:total_allocated_objects) - before
79
+ puts format("allocations per handoff %-20s %8.1f", name, allocated / 100.0)
80
+ end
81
+ puts
82
+ end
@@ -0,0 +1,40 @@
1
+ # Discovery Notes — Handoff Immutability Policies implementation planning
2
+
3
+ ## Tech stack
4
+ - Ruby gem "shifty", current version 0.5.0 (`lib/shifty/version.rb`). No `required_ruby_version` in gemspec yet (decision: add `>= 3.2`).
5
+ - Local Ruby: 4.0.5 (arm64-darwin). Runtime deps: none (stdlib only; `ostruct`, `forwardable`, `fiber`).
6
+ - Tests: RSpec 3.x + rspec-given (Given/When/Then style), simplecov; specs in `spec/shifty/{worker,gang,roster,dsl}_spec.rb`; `.rspec_status` persistence; monkey-patching disabled.
7
+ - Lint: standardrb (`.standard.yml` ignores Layout/ExtraSpacing). Rakefile standard bundler/gem_tasks (build via `rake build` / `gem build`).
8
+ - CI: `.github/` exists (workflows not yet examined by specialists — check if Ruby matrix needs updating for the 3.2 floor).
9
+
10
+ ## Source spec
11
+ - `docs/planning/handoff-immutability-policies.md` — full design doc, 13 sections + appendix. Treat as ground truth for WHAT. No plan-a-feature artifacts (no decision-log/team-findings/feature-technical-notes).
12
+
13
+ ## Code touch points
14
+ - `lib/shifty/worker.rb` — Worker: `initialize(p = {}, &block)` takes `:supply, :task, :context, :criteria, :tags`. Fiber-based `workflow` loop at line ~62: `Fiber.yield @task.call(value, supply, @context)` — THE handoff site for task output; `value = supply&.shift` is the intake site. Has `supply=` writer (no `task=` writer despite spec §5.4 mentioning one — only `@task ||= default_task` lazily). Extensive concurrency-model comment block (lines 90–130) added by PR #26; forward-references this plan and must be updated from "planned" to shipped.
15
+ - `lib/shifty/dsl.rb` — DSL worker constructors: `source_worker`, `relay_worker`, `side_worker` (has `mode: :hardened` → `Marshal.load(Marshal.dump(v))` at lines 40–41 — the feature to deprecate/subsume), `filter_worker`, `batch_worker` (BatchContext < OpenStruct), `splitter_worker` (mutates `parts` array internally via `parts.shift` — internal, not the handed-off value), `trailing_worker` (accumulates `trail` array in closure AND hands off `trail` itself — a live aliasing case under `:frozen`; hands off the same array it keeps mutating via unshift/pop!). `handoff` = `Fiber.yield`.
16
+ - `lib/shifty/gang.rb` — Gang wraps Roster; `initialize(workers = [], p = {})`; `shift` delegates to `roster.last.shift`; `Gang[*workers]` constructor; `|`/`supplies` composition same as Worker.
17
+ - `lib/shifty/roster.rb` — push/pop/shift/unshift rewires `supply` — mutation of topology, relevant to §5.4 `#freeze!`.
18
+ - `lib/shifty/taggable.rb` — tags/criteria mixin shared by Worker and Gang.
19
+ - `lib/shifty.rb` — requires all; no `Shifty.configure` exists yet (global config is new).
20
+ - No `Shifty::Testing`, no RSpec helpers shipped by the gem today.
21
+
22
+ ## Important behavioral details for policy implementation
23
+ - Composition (`|`) links workers by setting `supply`; there is NO chain/pipeline object for plain worker chains — a "pipeline" is just the last worker; Gang is the only reifying container. `.with_policy` on a composed chain must therefore live on Worker (and Gang) and propagate upstream via the supply chain.
24
+ - `filter_worker` and `batch_worker` tasks call `supply.shift` themselves mid-task — values obtained that way ALSO cross a boundary (policy must apply at intake, not just at task-output yield, or these secondary intakes leak).
25
+ - `splitter_worker` and `source_worker` use `handoff` (Fiber.yield) directly from inside the task — multiple handoffs per task call; policy application at the single `Fiber.yield @task.call(...)` site won't see these. Handoff site analysis is a core design task.
26
+ - `trailing_worker` hands off its live closure array — will break under `:frozen` default; the spec (§7.3) says this is exactly the bug class the policy should catch; the shipped worker itself must be fixed (hand off a snapshot).
27
+ - Worker's nil sentinel: nil marks end-of-stream; `relay_worker`/`side_worker` guard `value &&`. nil is already frozen/shareable — no policy cost.
28
+ - `@context` (OpenStruct) is closure-ish per-worker state passed to tasks — NOT a handoff; stays mutable per spec §12.
29
+
30
+ ## ADRs / coding standards
31
+ - None found (no docs/adr, no coding standards docs). Style = standardrb defaults. Doc precedent: `docs/use_cases.md`, `docs/shifty/worker.md`, README.md (rewritten by PR #26 with "Concurrency Model" section).
32
+
33
+ ## Recent activity (90 days)
34
+ - 1dc5433 planning doc (this feature); 847ee06/d14823a PR #26 concurrency docs (README.md, docs/use_cases.md, lib/shifty/worker.rb comments); 43d5948 concurrency analysis; 5e8ca16 arity fix in dsl.rb.
35
+
36
+ ## Gaps searched and not found
37
+ - No CLAUDE.md/AGENTS.md in repo. No CI examination yet. No benchmark tooling (no benchmark-ips in Gemfile). No existing implementation-plan format precedent in this repo.
38
+
39
+ ## User-committed decisions (from Joel, this session)
40
+ 1. Version 0.6.0, all four spec phases ship in it. 2. required_ruby_version >= 3.2. 3. API: worker `policy:` kwarg; chainable `.with_policy(:p)` on Worker AND Gang; `policy:` kwarg on Gang; `Shifty.configure { |c| c.default_policy = ... }`; built-in default `:frozen`; precedence worker > pipeline > global. 4. One PR per phase. 5. Wiki content drafted under `docs/wiki/`, pushed to GitHub wiki after merge. 6. `:hardened` deprecated → mapped to `:isolated` with warning. 7. §8.3 option (b) — no shareable? fast path initially; worker declarations authoritative, pipeline policy default-only. 8. Joel publishes the gem himself (2FA); plan ends at built gem + release prep.