farce 0.3.0 → 0.4.0

data/docs/compatibility.md

@@ -0,0 +1,74 @@
+<!--
+# @title Compatibility and Dependencies
+-->
+
+# Compatibility and Dependencies
+
+Farce has no mandatory dependencies beyond Ruby itself.
+
+## Ruby
+
+Each Farce release is expected to be compatible with:
+
+* The [latest patch release](https://www.ruby-lang.org/en/downloads/releases/) for each [CRuby](https://www.ruby-lang.org/en/) version [still receiving bug fixes](https://www.ruby-lang.org/en/downloads/branches/).
+* Ruby's [master branch](https://github.com/ruby/ruby/tree/master) at the time of release (i.e., the upcoming major version of CRuby).
+* The latest stable release of [JRuby](https://www.jruby.org/) and [TruffleRuby](https://truffleruby.dev/) (both in native and GraalVM modes).
+
+Moreover:
+
+* Dropping support for a CRuby version is only done in major releases.
+* If support for an older CRuby version is dropped, Farce will still backport security fixes for at least as long as that CRuby version is [still receiving security fixes](https://www.ruby-lang.org/en/downloads/branches/).
+
+## Optional Dependencies
+
+Farce's optional [integrations](integrations.md#integrations) depend on external libraries. Automated tests only run against recent versions of these libraries, but the code is written to be compatible with as wide a range as possible.
+
+Farce should be compatible with these versions:
+
+| Gem               | Minimum Version | Release Date |
+|-------------------|-----------------|--------------|
+| `ractor-tmvar`    | 0.4.0           | 2026-10-11   |
+| `msgpack`         | 1.7.0           | 2023-03-29   |
+| `concurrent-ruby` | 1.1.8           | 2021-01-20   |
+| `dry-types`       | 1.0.0           | 2019-04-23   |
+| `activesupport`   | 5.1.0           | 2017-04-27   |
+| `oj`              | 2.8.1           | 2014-04-21   |
+| `bson`            | 2.0.0           | 2013-12-02   |
+| `psych`           | 1.1.0           | 2011-03-30   |
+
+Any version of `cbor`, `json`, `ractor-sharing`, `sorted_set`, and `weakref` should be supported.
+
+If you want to use any of these in a version that is incompatible with Farce, you can [disable automatic integration loading](integrations.md#disable-automatic-loading).
+
+## Similar Projects
+
+* [ractor-shim](https://github.com/eregon/ractor-shim/) provides similar functionality to `Farce::Ractor`. See [the comparison document](gems/ractor-shim.md) for more details.
+* [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby) provides a more complete set of concurrency primitives than Farce, but is not compatible with Ractors.
+* [ratomic](https://mperham.github.io/ratomic/) has overlapping functionality with Farce for basic data structures like maps, counters, and queues.
+* [ractor_safe](https://github.com/jhawthorn/ractor_safe/) has overlapping functionality with Farce for basic data structures like maps, counters, and queues.
+* [ractor-sharing](https://github.com/ko1/ractor-sharing) has overlapping functionality with Farce for basic data structures like maps, counters, queues, as well as software transactional memory.
+
+All of the above projects can safely be used alongside Farce in the same application.
+
+## Known Issues and Limitations
+
+### Possible discrepancy regarding frozen state in Ruby and C
+
+> I agree that **freezing means the object's own state is immutable**, not just its instance variables, so we should not freeze [*shareable, mutable object*]. Forbidding instance variables on them is the right approach. [...] **A shareable object that is not frozen never has instance variables**. This should also hold when C extensions define such objects in the future.
+> — *Yukihiro Matsumoto* (Ruby Issue [#22291](https://bugs.ruby-lang.org/issues/22291#note-4), emphasis added)
+
+In Ruby, Farce objects reflect their frozen state accurately. If a map returns `true` for `frozen?`, you cannot add, remove, or replace its entries.
+
+There are some technical challenges implementing this behavior: From within Ruby, you cannot mark an object as Ractor shareable without freezing it first. This is possible from a C-extension, but then instance variables can no longer be used, so state tracking needs to happen purely at the C level or outside of the Ruby object.
+
+To work around this, Farce follows a hybrid approach, marking objects with state purely defined in a C extension as Ractor shareable without freezing them, and reimplementing freezing behavior in Ruby for objects where this isn't safely possible.
+
+This means:
+
+* `frozen?` and `freeze` will behave as expected for all Farce classes.
+* `Kernel.instance_method(:frozen?).bind_call(object)` might report a different value from `object.frozen?`
+* C-level checks for frozen state, such as `RB_OBJ_FROZEN`, might differ from what Ruby-level methods report.
+
+## Read Next
+
+[README](../README.md#features) · [Installation](installation.md) · [Integrations](integrations.md)

data/docs/concurrency.md

@@ -0,0 +1,151 @@
+<!--
+# @title Concurrency
+-->
+
+# Concurrency
+
+Farce is built for concurrent and parallel code execution.
+
+## Locks
+
+Farce ships with two lock classes:
+* `Lock` is a drop-in replacement for Ruby's `Mutex`.
+* `ReadWriteLock` exposes `with_read_lock`/`with_write_lock` to allow multiple concurrent reads, but exclusive write access.
+
+However, Farce exposes many APIs to eliminate the need for locks altogether.
+
+## Atomic Operations
+
+Farce's [data structures](data-structures.md#data-structures) all expose a range of methods for atomic operations:
+
+```ruby
+# A Hash-like object
+map = Farce::Map.new
+
+# Atomically store something for :key if it hasn't been set
+map.store_if_absent(:key) { "initial value" }
+
+# Atomically update :key
+map.update(:key, &:upcase)
+
+# An Array-like object
+list = Farce::Vector.new
+list[0] = 42
+
+# Atomically replace list[0] with 256 if the value is still 42
+list.compare_and_set(0, 42, 256)
+```
+
+## Observability and Signaling
+
+All the [data structures](data-structures.md#data-structures) come with extra observability methods, which eliminate the need for using [condition variables](https://docs.ruby-lang.org/en/4.0/Thread/ConditionVariable.html) and [mutexes](https://docs.ruby-lang.org/en/4.0/Thread/Mutex.html).
+
+Even setting aside that these don't work across ractors, this drastically reduces the risk of race conditions (very easy to do if you <u>don't</u> use the same mutex everywhere) or blocking code you don't need to block (very easy to do if you <u>do</u> use the same mutex everywhere).
+
+So instead of sharing locks you can simply wait for a change to happen!
+
+```ruby
+# block until the stored value for :key is greater than 10
+map.wait_until(:key) { it > 10 }
+
+# block until the value for an atom is no longer :initial
+atom.wait_until_changed(:initial)
+
+# block until a counter has reached at least 20
+counter.wait_while_below(20)
+```
+
+If these conditions are met right away, these never block.
+
+But what about more complex conditions, involving multiple variables, or objects from other libraries that don't implement similar methods? No need to reach for a lock! You can use a Signal!
+
+```ruby
+signal  = Farce::Signal.new
+target  = 100
+counter = Farce::Counter.new
+
+# A Thread that keeps reducing the target value every 20 milliseconds
+Thread.new do
+  while target.positive?
+    sleep 0.02
+    target -= 1
+    signal.broadcast # notify everyone else
+  end
+end
+
+# A Ractor that counts up in 10 millisecond intervals
+Ractor.new(counter, signal) do |counter, signal|
+  while counter < 100
+    sleep 0.01
+    counter.increment
+    signal.broadcast # notify everyone else
+  end
+end
+
+# wait until the counter is at or above the target
+signal.wait_until { counter >= target }
+```
+
+## Transactions
+
+This all sounds great. But you still might want to reach for a lock if you want to modify more than one data structure, if their state is tightly coupled (i.e., updating one without yet updating the other would leave your code in an invalid state).
+
+And a lock is an acceptable solution here. Again, you might want to wrap all the read access in a lock, as there will still be an invalid state. And you also want to roll back any changes already made while holding the lock, if an exception occurs. That is the standard approach in a lot of Ruby code.
+
+Farce offers an alternative. It implements [STM-style transactions](https://en.wikipedia.org/wiki/Software_transactional_memory). You supply a block of code that makes changes to multiple data structures. These changes are only written to these data structures in a commit phase after the block finishes, and they either all succeed or all fail, and they only succeed if the values you've read from any of these data structures haven't changed.
+
+Otherwise the block is rerun.
+
+And this isn't limited to special `TVar` containers, like the APIs provided by concurrent-ruby or ractor-sharing. It supports Farce's main data structures, including [maps](data-structures.md#maps), [vectors](data-structures.md#vectors), [sets](data-structures.md#sets), [atoms](data-structures.md#atoms), and [molecules](data-structures.md#molecules). Moreover, it also supports [mutables](sharing.md#mutables), meaning you can turn most Ruby objects into something transaction compatible fairly easily!
+
+In contrast to other implementations mentioned above, there is no implicit tracking. You need to explicitly add an object to a transaction. This also avoids any uncertainty around nested transactions and unexpected rollbacks.
+
+```ruby
+accounts = Farce::Map.new({a: 100, b: 200})
+
+# transfer 80 from :b to :a, but only if both succeed
+success = Farce.transaction(accounts) do |tx, accounts|
+  tx.abort! if accounts[:b] < 80
+  accounts[:a] += 80
+  accounts[:b] -= 80
+end
+
+if success
+  puts "Transaction succeeded"
+else
+  puts "Transaction failed"
+end
+```
+
+In the above example, `accounts` was added to the transaction right away. But you can also add new objects to the transaction programmatically by calling `tx[object]`. All reads and writes need to happen through the wrapper object returned by that call (or passed to the block).
+
+### Maps and Sets
+
+Maps and sets use fine-grained transaction tracking. In the above example, if accounts had another entry, `:c`, its value changing would not impact the transaction at all.
+
+Similarly, if your transaction uses `Transaction::Map#size` as input, only a change in size would trigger a rerun, not a change in content.
+
+This is ideal for scenarios where you use maps as a general data store and sets to track whether an operation was performed on an object (a common pattern to prevent infinite recursion).
+
+### TVars
+
+Farce doesn't come with a `TVar` class. You can just use an [atom](data-structures.md#atoms) instead.
+
+But it does come with built-in support for [`Concurrent::TVar`](https://ruby-concurrency.github.io/concurrent-ruby/master/Concurrent/TVar.html) and [`Ractor::TVar`](https://github.com/ko1/ractor-sharing/blob/main/docs/tvar.md) (assuming you also load these libraries, see [integrations](integrations.md#integrations)).
+
+```ruby
+tvar = Concurrent::TVar.new(50)
+map  = Farce::Map.new({a: 100, b: 200})
+
+Farce.transaction(map, tvar) do |_, map, tvar|
+  # subtract tvar's value from a and b, then set tvar to 0
+  # no invalid in-between state is ever visible to anything outside of this transaction
+  map[:a] -= tvar.value
+  map[:b] -= tvar.value
+  tvar.value = 0
+end
+```
+
+## Read Next
+
+[README](../README.md#features) · [Data Structures](data-structures.md) · [Scheduling Code](scheduling.md)

data/docs/data-structures.md

@@ -0,0 +1,313 @@
+<!--
+# @title Data Structures
+-->
+
+# Data Structures
+
+Farce provides a range of data structures that are ractor-safe, mutable, and expose high-level concurrency APIs in addition to standard Ruby APIs for the built-in classes it offers replacements for.
+
+## Contents
+
+- [Data Structures](#data-structures)
+  - [Contents](#contents)
+  - [Maps](#maps)
+  - [Vectors](#vectors)
+  - [Sets](#sets)
+  - [Atoms](#atoms)
+  - [Counters](#counters)
+  - [Flags](#flags)
+  - [References](#references)
+  - [Molecules](#molecules)
+  - [Queues](#queues)
+    - [Priority Queues](#priority-queues)
+    - [Timer Queues](#timer-queues)
+    - [Queue capacity](#queue-capacity)
+  - [Read Next](#read-next)
+
+## Maps
+
+Maps are Hash-like key-value data structures. They do not preserve insertion order.
+
+They implement almost all methods Ruby's Hash offers:
+
+```ruby
+map       = Farce::Map.new
+map[:foo] = :bar
+
+map.merge! answer: 42
+map.transform_values! { -it.to_s }
+map.to_h # => {answer: "42", foo: "bar"}
+```
+
+In addition, they come with a range of [atomic operations](concurrency.md#atomic-operations), and have built-in key normalization support:
+
+```ruby
+# normalize_keys can be a symbol, proc, hash, or another map
+map = Farce::Map.new({ a: 10 }, normalize_keys: :to_s)
+
+# Atomic upsert operation
+2.times { map.upsert(:b, 42) { it * map[:a] } }
+
+# Keys have been converted to strings
+map.keys.sort # => ["a", "b"]
+```
+
+Besides the standard map implementation, Farce includes a range of specialized maps:
+
+* `LRUMap` will evict the least recently used entry to not grow beyond the maximum size.
+* `LFUMap` will evict the least frequently used entry to not grow beyond the maximum size.
+* `LeaseMap` manages [leases](sharing.md#leases) associated with known keys.
+* `TreeMap` uses a [red-black tree](https://en.wikipedia.org/wiki/Red%E2%80%93black_tree) instead of a [hash table](https://en.wikipedia.org/wiki/Hash_table) to store its entries, keeping them sorted by the key's value.
+* `WeakMap` only holds weak references to its keys and values, automatically removing entries when the corresponding key or value gets garbage collected.
+* `WeakKeyMap` only holds weak references to its keys, automatically removing entries when the corresponding key gets garbage collected.
+* `WeakValueMap` only holds weak references to its values, automatically removing entries when the corresponding value gets garbage collected.
+
+For example, bounded maps are useful for implementing caches:
+
+```ruby
+cache = Farce::LRUMap.new(max_size: 2)
+
+cache[:first]  = 1
+cache[:second] = 2
+cache[:first] # makes sure :first was accessed more recently than :second
+cache[:third]  = 3
+
+cache.key?(:second) # => false
+```
+
+## Vectors
+
+`Farce::Vector` is to `Array` what `Farce::Map` is to `Hash`. It implements the same interface, with additional atomic operations.
+
+```ruby
+list = Farce::Vector.new
+10.times { list << it }
+list.to_a # => [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
+list.sum  # => 45
+```
+
+There is one important difference to arrays: Bulk mutations, such as `Vector#map!` are atomic. Either the block succeeds for all entries, or if it raises an exception then none of the changes are applied. Such mutations may also not write to the same vector from within the block.
+
+```ruby
+list = Farce::Vector[1, 2, 3, 4]
+
+begin
+  list.map! do |value|
+    raise "error" if value == 2
+    value * 2
+  end
+rescue => e
+end
+
+# This would return 2 if list was an array
+list.first # => 1
+```
+
+## Sets
+
+`Farce::Set` is to `Set` what `Farce::Map` is to `Hash`. It implements the same interface, with additional atomic operations.
+
+```ruby
+jobs = Farce::Set[:compile, :test]
+jobs.add? :publish    # => jobs
+jobs.add? :test       # => nil
+jobs.include? :publish # => true
+```
+
+Besides the standard set implementation, Farce includes two specialized sets:
+
+* `SortedSet` keeps its entries in order based on value comparison.
+* `WeakSet` only holds weak references to its entries.
+
+## Atoms
+
+Atoms are value containers. These store a single reference to any Ruby object, and expose atomic operations for mutating these values:
+
+```ruby
+atom = Farce::Atom.new("initial value")
+atom.value # => "initial value"
+
+# atoms are ractor-shareable
+Ractor.new(atom) do |atom|
+  atom.update { |current| current + " updated" } # => "initial value updated"
+end
+
+# waits for the other ractor to perform its update
+atom.wait_until_changed "initial value"
+```
+
+`WeakAtom` only keeps a weak reference for its current value, reverting to `nil` when the current value gets garbage collected.
+
+## Counters
+
+Ractor-shareable, atomic counter. This is a stateful, numeric object.
+
+It is similar to creating an [atom](#atoms) for an integer, but significantly faster, as it will use a truly lock-free implementation on most CPU/Ruby combinations.
+
+```ruby
+counter = Farce::Counter.new
+counter.value # => 0
+
+# Increase the counter by 1
+counter.increment
+counter.value # => 1
+
+# Increase the counter by 5 on another Ractor
+Ractor.new(counter) { it.add(5) }.join
+
+counter.value # => 6
+```
+
+## Flags
+
+Ractor-shareable, atomic boolean.
+
+Just like Counter for Integer, this is similar to creating an [atom](#atoms) for a boolean, but significantly faster, as it will use a truly lock-free implementation on most CPU/Ruby combinations.
+
+```ruby
+flag = Farce::Flag.new(true)
+flag.value # => true
+```
+
+## References
+
+Atoms, as well as other value objects, like counters and flags, can be wrapped in a reference object, which delegates all methods to the given atom for convenience:
+
+```ruby
+atom = Farce::Atom.new(42)
+ref  = Farce::Reference.new(atom)
+
+ref.to_s  # => "42"
+ref > 10  # => true
+
+atom.value = :foo
+ref.to_s  # => "foo"
+```
+
+## Molecules
+
+Molecules are structures made up of multiple atoms. Clever naming, right?
+
+They are similar to Ruby's built-in `Struct` and `Data` classes, but their members are stored in atoms:
+
+```ruby
+Request = Farce::Molecule.define(:verb, :path)
+request = Request.new("GET", "/index.html")
+request.verb  # => "GET"
+request.verb = "POST"
+
+# This doesn't succeed, as the verb isn't "HEAD"
+request.verb_atom.compare_and_set("HEAD", "GET")
+request.verb  # => "POST"
+```
+
+You can also pass a block to add additional methods:
+
+```ruby
+Request = Farce::Molecule.define(:verb, :path) do
+  def to_s = "#{verb} #{path}"
+end
+
+Request.new(verb: "GET", path: "/").to_s  # => "GET /"
+```
+
+You can also inherit from the generated class:
+
+```ruby
+class Request < Farce::Molecule.define(:verb, :path)
+  def to_s = "#{verb} #{path}"
+end
+```
+
+## Queues
+
+`Farce::Queue` is an alternative to Ruby's `Queue` and `SizedQueue`. It provides a superset of their API, offering both blocking and a non-blocking API, while implementing Ractor-sharing via [sharing modes](sharing.md#sharing-modes).
+
+```ruby
+queue = Farce::Queue.new
+queue.push(:ok)
+queue.pop     # => :ok
+queue.try_pop # => nil
+```
+
+Queues provide additional features over Ruby's queue:
+* Like the rest of Farce, they work with Ractors (implementing [sharing modes](sharing.md#sharing-modes) for their [default variant](variants-and-scopes.md#variants-and-scopes)).
+* In addition to closing, they allow sealing, which prevents pushes but still allows pulling from the queue until it has been drained.
+* They have an API to wait for them to be ready for a push or pull based on capacity without actually adding or removing values.
+* They can optionally monitor how long the oldest item has been sitting in the queue (which is great for building auto-scaling on top of the queue).
+
+The [strict variant](variants-and-scopes.md#available-variants) has comparable performance to `SizedQueue`, while the default variant pays a fixed overhead for [handling unshareable data](sharing.md#sharing-unshareable-data). Both variants outperform other third-party, Ractor-shareable queues (none of which are Fiber scheduler compatible, which Farce's queues are).
+
+### Priority Queues
+
+In addition to normal queues, which are FIFO, Farce also offers a `PriorityQueue`, where values are removed from the queue in order of priority instead of insertion order:
+
+```ruby
+# use order: :descending to reverse priorities
+queue = Farce::PriorityQueue.new
+
+queue.push :a, priority: 2
+queue.push :b, priority: 1
+
+queue.pop # => :b
+```
+
+### Timer Queues
+
+Timer queues are similar to priority queues, sorted by float values, but these values are timestamps (based on [Farce::Clock](utilities.md#additional-classes), not on Ruby's Time, which isn't monotonic). Any `pop` will block until the next timestamp has been reached.
+
+```ruby
+queue = Farce::TimerQueue.new
+queue.push(:ok, in: 1.5)
+
+# blocks for ~1.5 seconds
+queue.pop # => :ok
+```
+
+Timer queues are useful for implementing schedulers (which is what Farce uses them for under the hood). Both `push` and `try_push` return a cancellation handle when they succeed. Use `entry.delete` or `queue.delete(entry)` to cancel a scheduled value. The handle exposes its `queue` and scheduled `clock` time.
+
+```ruby
+queue = Farce::TimerQueue.new
+entry = queue.push(:work, in: 60)
+
+entry.delete # => true
+```
+
+You can also delete a value if you know its timestamp:
+
+```ruby
+queue     = Farce::TimerQueue.new
+timestamp = Farce.clock(in: 0.2)
+
+queue.push(:first,  at: timestamp - 0.1)
+queue.push(:second, at: timestamp)
+queue.push(:third,  at: timestamp + 0.1)
+
+fourth = queue.push(:fourth, at: timestamp + 0.2)
+queue.delete(fourth)
+
+queue.delete(:second, at: timestamp)
+
+result = []
+result << queue.pop until queue.empty?
+result # => [:first, :third]
+```
+
+### Queue capacity
+
+Queues support setting a maximum capacity. If the capacity has been reached it will block any pushes to create back pressure.
+
+```ruby
+queue = Farce::Queue.new(capacity: 2)
+
+queue.try_push(:a) # => true
+queue.try_push(:b) # => true
+queue.try_push(:c) # => false
+```
+
+Normal queues have a default capacity of 1024. You can explicitly set the capacity to `nil` to create an unbounded queue.
+Priority queues and timer queues do not have a default capacity, so earlier entries are not getting blocked unexpectedly.
+
+## Read Next
+
+[README](../README.md#features) · [Sharing Unshareable Data](sharing.md) · [Concurrency](concurrency.md) · [Variants and Scopes](variants-and-scopes.md)

data/docs/installation.md

@@ -0,0 +1,63 @@
+<!--
+# @title Installation
+-->
+
+# Installation
+
+## Globally
+
+To install Farce globally, you can use the following command:
+
+```console
+$ gem install farce
+```
+
+## As a project dependency
+
+If you want to use Farce directly in your project, it is recommended to do so via [Bundler](https://bundler.io).
+Add Farce to your `Gemfile`:
+
+```ruby
+source "https://gem.coop" # or "https://rubygems.org"
+
+gem "farce"
+```
+
+Then run `bundle install` to install the dependencies.
+
+## As a library dependency
+
+Farce's main purpose is to be used as a dependency for other libraries. As such, it will most commonly be added as a [runtime dependency](https://guides.rubygems.org/specification-reference/#add_dependency) to your gemspec:
+
+```ruby
+Gem::Specification.new do |spec|
+  # ...
+  spec.add_dependency "farce"
+end
+```
+
+## Local setup
+
+If you want to work on Farce itself, you can clone the repository and use [mise](https://mise.jdx.dev) to set everything up:
+
+For more details, or if you aren't using mise, check the [contribution guidelines](../CONTRIBUTING.md).
+
+```console
+$ git clone https://github.com/rkh/farce.git # prefix with `jj` if you're using Jujutsu
+$ cd farce
+$ mise run
+```
+
+## Loading Farce
+You should always require `farce`, rather than any other files in `lib`. Other files are not intended as entry points.
+
+```ruby
+require "farce"
+```
+
+Constants (classes, modules, etc.) under the `Farce` namespace are loaded lazily (thread- and ractor-safe), so there is no
+need to specifically load any particular file.
+
+## Read Next
+
+[README](../README.md#features) · [Introduction](introduction.md) · [Compatibility and Dependencies](compatibility.md)