hanikamu-operation 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b935969d9ffb9b34935f54e2f9d8adf907603906916a4c1b555499bd25e97f28
-  data.tar.gz: 2fada67131e16535eb5c6480c3a80999315efd0306753aae615a6a582d9cb43d
+  metadata.gz: 565d28c9c7d700db3b5bdb2be0ddd960951d07bf027a16b946cc8c527ad45ae6
+  data.tar.gz: 2c241ea17940bc687881d746361fa94fe85dfc3149d410c4b665c44d0367491b
 SHA512:
-  metadata.gz: cc5970b13fd6511a024b14071b273350604ef811f8bb944ca03f6839ed7f03051d106c63f550225f2b4de181d60c7698f545ecd1219faa1bb813aaa76be340a3
-  data.tar.gz: a691099d6fb2be177d4e19bcfff74e9760716465b2a9ceaff2fa0df33a63027e199377749e285f4a8b9cad02224c93e347e4b229eab60f159aa2d4bc5e585454
+  metadata.gz: da40b97d680e9f4f99b9819283c5d9441335b5a2e05b77143ee33b04635db03c6a3e00896f4b98f9481950d2b52721080c48b958eef8e3ba0133dda020da221d
+  data.tar.gz: 6bf7537f8d8b3351f122c4e3a9c0de036c736a97907dcdf460fa9bd932600051f2807f39aaefaf28ee79b5f48264d7db0a9dc0132470f475cc45c4392701d8f4

data/CHANGELOG.md

@@ -14,3 +14,22 @@
 - Removed `redis-client` as direct dependency (now transitive through `redlock`)
 - Updated CI to test only Ruby 3.4
 - Improved README with clearer FormError vs GuardError examples
+
+## [0.2.0] - 2026-03-10
+
+- Added conditional locking for `within_mutex` via `:if` / `:unless` options: the Redis lock is
+  skipped entirely when the condition opts out (e.g. an optional lock-key attribute is `nil`).
+  Passing both `:if` and `:unless`, or a non-callable condition, raises `ArgumentError` at class
+  definition time.
+
+## [0.3.0] - 2026-08-25
+
+- `within_mutex` is now reentrant within the same execution context (fiber-local, effectively
+  per-thread in the standard thread-per-request / thread-per-job model): a nested acquire of a key
+  the current context already holds runs inline instead of self-deadlocking. Only real acquisitions
+  talk to Redis. The bypass is lease-aware — each real acquire records a deadline derived from
+  Redis's own remaining TTL (clock-drift adjusted), and the inline bypass only applies while a lease
+  this context holds is still live. Once the lease could have lapsed, a nested call re-acquires for
+  real as its own lease (so deeper nested calls still bypass safely), or raises `Redlock::LockError`
+  if the key was taken over. Cross-thread / cross-fiber / cross-process locking is unchanged.
+  Reentrancy is the default with no opt-out flag.

data/README.md

@@ -66,7 +66,7 @@ Requires Ruby 3.4.0 or later.
 
 ```ruby
 # Gemfile
-gem 'hanikamu-operation', '~> 0.2.0'
+gem 'hanikamu-operation', '~> 0.3.0'
 ```
 
 ```bash
@@ -135,7 +135,7 @@ Requires Ruby 3.4.0 or later.
 
 ```ruby
 # Gemfile
-gem 'hanikamu-operation', '~> 0.2.0'
+gem 'hanikamu-operation', '~> 0.3.0'
 ```
 
 ```bash
@@ -364,6 +364,59 @@ within_mutex(:mutex_lock, unless: -> { order_id.nil? })
 
 The lambda is evaluated via `instance_exec` on the operation instance, so it has access to all attributes and methods. Providing both `:if` and `:unless` raises `ArgumentError`.
 
+**Reentrancy (same execution context, same key)**:
+
+`within_mutex` is **reentrant within the same execution context**. If an operation holding a lock key synchronously calls another operation that locks the **same** key — for example through a synchronous callback/event handler, or a nested operation/service call — the inner acquire runs inline instead of trying to re-acquire the lock. Only the outermost holder talks to Redis.
+
+This matters because Redlock itself is **not** reentrant: without this, the same execution context would block on its own lock until the TTL expired and then raise `Redlock::LockError`, even though there is no concurrent request. Reentrancy removes that self-deadlock, matching the semantics of `Monitor`, Java's `ReentrantLock`, and ActiveRecord's nested transactions.
+
+> **Scope: fiber-local (effectively per-thread).** The lease bookkeeping is stored in `Thread.current[...]`, which in Ruby is *fiber-local*. In the standard thread-per-request / thread-per-job model (Puma, Sidekiq) each thread runs a single root fiber, so reentrancy behaves per-thread. Under a fiber scheduler (e.g. Falcon/async) or if you manually spawn a `Fiber` for the nested call, that separate fiber has its own lease stack and will contend on Redis normally — this is deliberate: it never bypasses a lock that a different, independently scheduled context might hold.
+
+```ruby
+class UpdateResourceOperation < Hanikamu::Operation
+  attribute :resource_id, Types::Integer
+
+  within_mutex(:mutex_lock)
+
+  def execute
+    resource = Resource.find(resource_id)
+    resource.touch!
+
+    # Runs inline under the lock already held by this execution context —
+    # no second Redis acquire, no self-deadlock.
+    SyncResourceOperation.call!(resource_id: resource_id)
+
+    response resource: resource
+  end
+
+  def mutex_lock
+    "Resource$#{resource_id}"
+  end
+end
+
+class SyncResourceOperation < Hanikamu::Operation
+  attribute :resource_id, Types::Integer
+
+  within_mutex(:mutex_lock) # same key as the caller
+
+  def execute
+    response(synced: true)
+  end
+
+  def mutex_lock
+    "Resource$#{resource_id}"
+  end
+end
+```
+
+**Scope and guarantees**:
+
+- **Same execution context only**: reentrancy is keyed to the current fiber (via a fiber-local lease stack), which in the usual thread-per-request / thread-per-job model means the current thread. A synchronous callback cascade or nested call runs in that same context, so it is treated as the same holder.
+- **Cross-context is unchanged**: a different thread, process, or independently scheduled fiber (e.g. a separate background job or web request) still contends on Redis and still raises `Redlock::LockError` when the key is held elsewhere.
+- **No configuration, no opt-out**: a same-context re-acquire of a held key can only self-deadlock, so there is no valid non-reentrant use case. Reentrancy is always on.
+- **Lease-aware, not just lexical**: the inline bypass only happens while a lease this context actually holds on the key is still live. Each real acquire records a deadline derived from Redis's own remaining TTL (already clock-drift adjusted), so the bypass window never outlives the lease Redis granted. If an operation runs past its mutex TTL (so the lease could have lapsed and been taken over), a nested same-key call does **not** run inline — it performs a real acquire, which re-locks the key if it is free (its own lease frame, so deeper nested calls still bypass safely) or raises `Redlock::LockError` if another context now owns it. Reentrancy therefore never weakens mutual exclusion beyond what Redlock itself guarantees.
+- **TTL is not refreshed** by nested reentrant calls — the outermost acquire's expiry still applies.
+
 ### Database Transactions with `within_transaction`
 
 Ensure multiple database changes succeed or fail together atomically. If any database operation raises an exception, all changes are rolled back.

data/docs/projects/20260825_hanikamu_reentrant_mutex.md

@@ -0,0 +1,231 @@
+# Make `within_mutex` reentrant in `hanikamu-operation`
+
+**Status:** implemented; pending release
+**Owner:** Nicolai
+**Repo touched:** `hanikamu-operation`
+
+---
+
+## TL;DR
+
+`Hanikamu::Operation`'s `within_mutex` uses a **non-reentrant** Redlock. When an operation
+that holds a lock key synchronously triggers another operation that locks the **same** key —
+via a synchronous event handler, or a plain nested service/operation call — the second acquire
+blocks on a lock the same thread already holds. It spins for the retry window and then raises
+`Redlock::LockError`. This is a **latent self-deadlock**, and it becomes a hard, reproducible
+failure as soon as the mutex TTL is long enough that the outer lock hasn't expired by the time
+the inner acquire runs.
+
+The sustainable fix is to make `within_mutex` **reentrant per execution context** (fiber-local,
+effectively per-thread in the standard thread-per-request / thread-per-job model): if the current context
+already holds the resolved lock key, run inline instead of re-acquiring; only the outermost holder
+talks to Redis. This is the default behaviour — **no opt-in flag** (see "Design decision").
+
+---
+
+## Why this happens
+
+- A distributed lock keyed by some resource is re-entered by the **same execution context** within
+  one logical unit of work (a synchronous callback cascade, or a nested operation call), but Redlock
+  treats that re-entry as a competing writer.
+- Redlock is not reentrant, so the inner acquire waits on the outer's lock. It exhausts the retry
+  window and raises `Redlock::LockError` — with **no concurrent request** anywhere.
+- Raising the mutex TTL doesn't create the bug; it turns a race that was formerly "usually survived
+  by lock expiry" into a hard, deterministic failure.
+
+### The nesting shapes that trigger it
+
+1. **Synchronous callback cascade.** An operation holds `Resource$id` and publishes an event; a
+   **synchronous** handler for that event invokes another operation whose lock key is also
+   `Resource$id`.
+2. **Nested service/operation call.** An operation holds `Resource$id` and, inside `execute`, calls
+   another operation (directly or via a service) that locks the same `Resource$id`.
+
+Asynchronous callbacks (handlers that run in a separate job/thread) are never the problem — they run
+outside the publisher's mutex and contend on Redis normally.
+
+---
+
+## The fix (in `hanikamu-operation`)
+
+Only `#within_mutex!` changes, plus a small thread-local registry. File: `lib/hanikamu/operation.rb`.
+
+### Before
+
+```ruby
+def within_mutex!(&)
+  return yield if self.class._mutex_lock_key.blank?
+  return yield unless _should_apply_mutex?
+
+  lock_key = public_send(self.class._mutex_lock_key)
+  Hanikamu::Operation.redis_lock.lock!(lock_key, self.class._mutex_expire_milliseconds, &)
+end
+```
+
+### After (reentrant)
+
+```ruby
+def within_mutex!(&)
+  return yield if self.class._mutex_lock_key.blank?
+  return yield unless _should_apply_mutex?
+
+  # Freeze a copy so operation code can't mutate the key object and desync the registry.
+  lock_key = _stable_lock_key(public_send(self.class._mutex_lock_key))
+
+  # Reentrancy: a synchronous nested call while this context still holds a *live* lease
+  # on the key would otherwise re-acquire it and self-deadlock (Redlock is not
+  # reentrant). Run inline instead — but only while a lease this context holds is still
+  # valid (see below). Once it could have lapsed, fall through to a real acquire so a
+  # taken-over key still raises Redlock::LockError.
+  return yield if _reentrant_lease_valid?(lock_key)
+
+  _acquire_and_run(lock_key, &)
+end
+```
+
+Add to the `private` section (see `lib/hanikamu/operation.rb` for the full set):
+
+```ruby
+# Real acquire: push this lease's deadline onto this context's per-key stack, run,
+# then pop. A nested call that finds the lease expired lands here again and takes a
+# fresh, independent lease (its own stack frame), so it never contends with itself.
+def _acquire_and_run(lock_key, &)
+  Hanikamu::Operation.redis_lock.lock!(lock_key, self.class._mutex_expire_milliseconds) do
+    _push_lease(lock_key)
+    begin
+      yield
+    ensure
+      _pop_lease(lock_key)
+    end
+  end
+end
+
+def _stable_lock_key(key)
+  key.frozen? ? key : key.dup.freeze
+end
+
+# Fiber-local per-key stack of live-lease deadlines (Thread.current[...] is
+# fiber-local): correct scope because a synchronous cascade runs in the same fiber; a
+# separate job/request/fiber has its own stack and contends. A stack (not a single
+# value) lets a replacement lease taken after expiry restore the previous window when
+# it exits.
+def _lease_stacks
+  Thread.current[:hanikamu_operation_lease_stacks] ||= {}
+end
+
+def _monotonic_ms
+  Process.clock_gettime(Process::CLOCK_MONOTONIC, :millisecond)
+end
+
+# Bypass only while the innermost (top) lease this context holds is still live.
+def _reentrant_lease_valid?(lock_key)
+  stack = _lease_stacks[lock_key]
+  return false unless stack&.any?
+
+  _monotonic_ms < stack.last
+end
+
+# Anchor the deadline to Redis's authoritative remaining TTL (already clock-drift
+# adjusted by Redlock), captured right after acquisition, so the window never outlives
+# the lease Redis actually granted — even if acquisition retried/took time.
+def _push_lease(lock_key)
+  remaining = Hanikamu::Operation.redis_lock.get_remaining_ttl_for_resource(lock_key)
+  deadline = _monotonic_ms + (remaining || self.class._mutex_expire_milliseconds)
+  (_lease_stacks[lock_key] ||= []) << deadline
+end
+
+def _pop_lease(lock_key)
+  stack = _lease_stacks[lock_key]
+  return unless stack
+
+  stack.pop
+  _lease_stacks.delete(lock_key) if stack.empty?
+end
+```
+
+### Why this is correct / safe
+
+- **Return value preserved:** `redis_lock.lock!` returns the block's value; `_acquire_and_run`
+  returns the value of `yield` (the `ensure` around `_pop_lease` doesn't override it), and the bypass
+  path is a plain `yield`. Operation responses flow through unchanged.
+- **Exception-safe:** a lease is only pushed once we're inside the `lock!` block, and the `ensure`
+  always pops it. If `lock!` itself fails to acquire (real contention from another context), the
+  block never runs, so nothing is pushed and nothing leaks.
+- **Lease-aware, not merely lexical:** the bypass is gated on a live lease — each real acquire pushes
+  a deadline derived from Redis's own remaining TTL (already clock-drift adjusted), captured right
+  after acquisition, so the window never outlives the lease Redis granted (even under a slow/retried
+  acquire). If an operation outlives its mutex TTL, a nested same-key call re-acquires for real as
+  its own stack frame instead of running inline — re-locking a free key (deeper nested calls then
+  bypass *that* replacement lease, avoiding a fresh self-deadlock) or raising `Redlock::LockError` on
+  takeover. Reentrancy never weakens mutual exclusion beyond Redlock's inherent TTL guarantee.
+- **Stable key:** the key is snapshotted (frozen copy) before use, so an operation mutating the
+  original String object can't leak a stack entry or make a future call wrongly bypass Redis.
+- **Distributed guarantee intact:** reentrancy is strictly same-context (same fiber) and only while
+  the lease is valid. Cross-thread / cross-fiber / cross-process contention is unchanged — a lock
+  held by a *different* context still raises `Redlock::LockError`.
+- **No behaviour change for non-nesting code:** first acquire hits Redis and releases at the end,
+  exactly as today.
+- **TTL:** nested reentrant calls don't refresh the TTL — same as today for any single long op.
+
+---
+
+## Design decision: default, not configurable
+
+Reentrancy is the **default** with **no opt-out flag**. Rationale:
+
+- A same-context re-acquire of a held key has **no valid non-reentrant use case** — it can only
+  self-deadlock then raise. Nobody opts into that.
+- Matches `Monitor`, Java `ReentrantLock`, and ActiveRecord nested transactions (savepoints).
+- Adding `within_mutex(:key, reentrant: false)` would only re-expose the footgun and add config
+  surface (violates the repo's Simplicity-First rule). If an escape hatch is ever genuinely needed,
+  put it at config level (`config.reentrant_mutex`, default `true`) — not per declaration.
+
+---
+
+## Specs to add (`spec/hanikamu/operation_spec.rb`, `#within_mutex` describe block)
+
+The suite already uses a **real Redis** (`described_class.redis_lock`). Cover:
+
+1. **Nested, same key, same context → single Redis acquire, both run.**
+   Define an outer op whose `execute` calls an inner op with the **same** lock key. Spy on
+   `redis_lock.lock!` and assert it received the key **once**, and both ops ran (in order).
+2. **Nested, different keys → two acquires.** Outer op calls inner op with a **different** key;
+   assert `lock!` received both keys.
+3. **Cross-thread still contends.** Run an outer op that acquires the key and blocks inside `execute`
+   with its lease stack populated; a second thread calling the same key does **not** see the stack
+   and raises `Redlock::LockError`.
+4. **Stack cleanup (success).** After a nested run completes,
+   `Thread.current[:hanikamu_operation_lease_stacks]` is empty — no leak.
+5. **Stack cleanup (raise).** After a nested run that raises inside the inner block, the stack is
+   still empty — the `ensure` pops on the exception path.
+6. **Lease expiry → re-acquire.** An outer op with a short TTL that sleeps past its lease, then makes
+   a nested same-key call, must **re-acquire** (a second `lock!` for the key), not bypass.
+7. **Deep nesting after expiry (no self-deadlock).** An outer op whose short lease lapses calls a
+   middle op (same key) that re-acquires a fresh lease, which calls an inner op (same key): the inner
+   call must **bypass** the replacement lease inline and complete — proving the per-key stack tracks
+   the live replacement lease, not the expired outer deadline.
+8. **Mutable key snapshot.** An op whose lock-key method returns a String that `execute` mutates
+   still cleans up the stack (no leaked entry under the pre-mutation value).
+
+---
+
+## Release
+
+1. Implement the change + specs above.
+2. Bump `spec.version` `0.2.0 → 0.3.0` in the gemspec.
+3. CHANGELOG `## [0.3.0]`: *"`within_mutex` is now reentrant within the same execution context
+   (fiber-local, effectively per-thread): a nested acquire of a key the current context already holds
+   runs inline instead of self-deadlocking. Cross-thread / cross-fiber / cross-process locking is
+   unchanged."*
+4. Document the reentrancy behaviour in the README `within_mutex` section.
+5. Run gem `rspec` + `rubocop`, release (MFA required per rubygems).
+
+---
+
+## Verification
+
+- `make rspec` — the new reentrancy specs pass, and the existing `#within_mutex` specs
+  (single-acquire, `:if`/`:unless`, cross-thread `Redlock::LockError`) still pass unchanged.
+- `make cops` — clean.
+- Any downstream app that was working around this by namespacing lock keys for nested operations can
+  drop the workaround: locking the bare resource key from nested operations is now safe.

data/lib/hanikamu/operation.rb

@@ -2,8 +2,7 @@
 
 module Hanikamu
   # :nodoc:
-  # rubocop:disable Metrics/ClassLength
-  class Operation < Hanikamu::Service
+  class Operation < Hanikamu::Service # rubocop:disable Metrics/ClassLength
     include ActiveModel::Validations
 
     class Error < Hanikamu::Service::Error; end
@@ -112,8 +111,7 @@ module Hanikamu
 
       # Define guard validations using a block
       # The block is evaluated in the context of a Guard class
-      # rubocop:disable Metrics/MethodLength
-      def guard(&block)
+      def guard(&block) # rubocop:disable Metrics/MethodLength
         return unless block
 
         # Thread-safe constant definition with mutex
@@ -148,7 +146,6 @@ module Hanikamu
           const_set(:Guard, guard_class)
         end
       end
-      # rubocop:enable Metrics/MethodLength
 
       attr_reader :_mutex_lock_key, :_mutex_expire_milliseconds, :_mutex_if_condition, :_mutex_unless_condition,
                   :_transaction_klass, :_block
@@ -193,8 +190,26 @@ module Hanikamu
       return yield if self.class._mutex_lock_key.blank?
       return yield unless _should_apply_mutex?
 
-      lock_key = public_send(self.class._mutex_lock_key)
-      Hanikamu::Operation.redis_lock.lock!(lock_key, self.class._mutex_expire_milliseconds, &)
+      # Snapshot the resolved key: it is documented as a String, and freezing a
+      # copy prevents operation code from mutating the same object mid-run and
+      # desyncing the reentrancy registry / lease bookkeeping (the cleanup would
+      # otherwise look up a different value and leak the entry).
+      lock_key = _stable_lock_key(public_send(self.class._mutex_lock_key))
+
+      # Reentrancy: a nested operation invoked synchronously (e.g. from a synchronous
+      # event handler, or a nested service call) while this context still holds this
+      # exact key would otherwise re-acquire it. Redlock is not reentrant, so the same
+      # execution context would block on itself until the TTL expires and then raise
+      # Redlock::LockError. Skip the re-acquire and run inline; only real acquisitions
+      # talk to Redis. Different fibers/threads/processes still contend normally.
+      #
+      # The bypass is gated on a *live* lease, not merely lexical nesting: we bypass
+      # only while the innermost lease this context holds on the key is still valid.
+      # Once it could have lapsed we fall through to a real acquire — which re-acquires
+      # the key if it is free, or raises Redlock::LockError if it was taken over.
+      return yield if _reentrant_lease_valid?(lock_key)
+
+      _acquire_and_run(lock_key, &)
     end
 
     def within_transaction!(&)
@@ -212,6 +227,74 @@ module Hanikamu
       true
     end
 
+    # Acquire a real Redlock lease, push its deadline onto this context's stack for the
+    # key, run, then release. Nested reentrant calls ride on this lease without touching
+    # Redis; a nested call that finds the lease expired lands here again and takes a
+    # fresh, independent lease (its own stack frame), so it never contends with itself.
+    def _acquire_and_run(lock_key, &)
+      Hanikamu::Operation.redis_lock.lock!(lock_key, self.class._mutex_expire_milliseconds) do
+        _push_lease(lock_key)
+        begin
+          yield
+        ensure
+          _pop_lease(lock_key)
+        end
+      end
+    end
+
+    # A key is documented as a String; freeze a copy so it is a stable, immutable
+    # registry key regardless of what the operation does with the original object.
+    # Already-immutable values (frozen strings, symbols, integers) pass through.
+    def _stable_lock_key(key)
+      key.frozen? ? key : key.dup.freeze
+    end
+
+    # Per-key stack of monotonic deadlines (ms) for the real Redlock leases this context
+    # currently holds, scoped to the current execution context. Storage is
+    # `Thread.current[...]`, which in Ruby is fiber-local: this is the correct scope
+    # because a synchronous event cascade or nested call runs in the same fiber as the
+    # publishing operation, so it must be treated as the same holder. In the standard
+    # thread-per-request / thread-per-job model (Puma, Sidekiq) each thread has a single
+    # root fiber, so this is effectively per-thread. A separate job / request — or an
+    # independently scheduled fiber under a fiber scheduler — has its own stack and must
+    # still contend on Redis, which is exactly what we want (never bypass a lease held
+    # elsewhere). A stack (not a single value) is required so that a replacement lease
+    # taken after an outer lease expired restores the previous window when it exits.
+    def _lease_stacks
+      Thread.current[:hanikamu_operation_lease_stacks] ||= {}
+    end
+
+    def _monotonic_ms
+      Process.clock_gettime(Process::CLOCK_MONOTONIC, :millisecond)
+    end
+
+    # Bypass is valid only while the innermost lease this context holds on the key is
+    # still within its lease window (the top of the stack is the most recently acquired,
+    # hence longest-living, lease).
+    def _reentrant_lease_valid?(lock_key)
+      stack = _lease_stacks[lock_key]
+      return false unless stack&.any?
+
+      _monotonic_ms < stack.last
+    end
+
+    # Anchor the deadline to Redis's authoritative remaining TTL (already clock-drift
+    # adjusted by Redlock), captured right after acquisition, so the window never
+    # outlives the lease Redis actually granted — even if acquisition retried/took time.
+    def _push_lease(lock_key)
+      remaining = Hanikamu::Operation.redis_lock.get_remaining_ttl_for_resource(lock_key)
+      deadline = _monotonic_ms + (remaining || self.class._mutex_expire_milliseconds)
+      (_lease_stacks[lock_key] ||= []) << deadline
+    end
+
+    def _pop_lease(lock_key)
+      stack = _lease_stacks[lock_key]
+      return unless stack
+
+      stack.pop
+      _lease_stacks.delete(lock_key) if stack.empty?
+    end
+
     def transaction_class
       return if self.class._transaction_klass.nil?
       return ActiveRecord::Base if self.class._transaction_klass == :base
@@ -237,5 +320,4 @@ module Hanikamu
       raise Hanikamu::Operation::GuardError, @guard
     end
   end
-  # rubocop:enable Metrics/ClassLength
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: hanikamu-operation
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Nicolai Seerup
@@ -114,6 +114,7 @@ files:
 - README.md
 - Rakefile
 - docker-compose.yml
+- docs/projects/20260825_hanikamu_reentrant_mutex.md
 - lib/hanikamu-operation.rb
 - lib/hanikamu/operation.rb
 homepage: https://github.com/Hanikamu/hanikamu-operation