odsl 0.1.1 → 0.3.0

data/docs/superpowers/specs/2026-10-09-background-adapters-audit-design.md

@@ -0,0 +1,267 @@
+# ODSL — Background Adapters, Audit System & Distributed Locking
+
+**Date:** 2026-10-09  
+**Author:** alejandro gurovich  
+**Status:** Approved
+
+---
+
+## Overview
+
+This document describes three interconnected features for the `odsl` gem:
+
+1. **Background adapters** — pluggable backends for `run_in_background` (Thread, ActiveJob)
+2. **Audit system** — step-level execution tracking with idempotency and resume
+3. **Distributed locking** — optional Redlock integration to prevent concurrent duplicate runs
+
+The gem targets **Ruby on Rails applications**. All features are opt-in and backward-compatible: existing code with no `key:` or `lock` call behaves exactly as before.
+
+---
+
+## 1. Background Adapters
+
+### Architecture
+
+```
+lib/odsl/adapters/
+  base.rb        ← common interface
+  thread.rb      ← default (plain Ruby Thread)
+  active_job.rb  ← Rails bridge (Sidekiq, Solid Queue, Good Job, etc.)
+```
+
+### Interface (base.rb)
+
+```ruby
+module Odsl
+  module Adapters
+    class Base
+      def self.enqueue(service_class:, method_name:, kwargs:, key: nil)
+        raise NotImplementedError
+      end
+    end
+  end
+end
+```
+
+### Thread adapter
+
+Current implementation. Returns a `Thread`; caller can block on `.value` to get the result. Used as the default when no adapter is configured.
+
+### ActiveJob adapter
+
+Defines `Odsl::OdslJob < ActiveJob::Base` (lives in `lib/odsl/adapters/odsl_job.rb`) that receives `service_class`, `method_name`, `kwargs`, and `key`. Compatible with any ActiveJob backend: Solid Queue, Sidekiq, Good Job, Delayed::Job, Resque.
+
+Returns the enqueued job object (has `.job_id`).
+
+### Configuration
+
+```ruby
+# config/initializers/odsl.rb
+Odsl.configure do |config|
+  config.background_adapter = :active_job  # default: :thread
+end
+```
+
+---
+
+## 2. Audit System
+
+### Design decision: checkpoints at `.get()`
+
+`.get(:variable)` is already the natural checkpoint of the DSL — it names and stores a step result. The audit system hooks into `.get()` to record a `StepRecord` after each call, making it the single source of truth for step identity.
+
+### Database tables
+
+**`odsl_execution_contexts`**
+
+| Column | Type | Notes |
+|--------|------|-------|
+| `id` | bigint PK | |
+| `idempotency_key` | string | unique index |
+| `service_class` | string | e.g. `"Games::Service"` |
+| `method_name` | string | e.g. `"create"` |
+| `kwargs` | jsonb/text | initial kwargs, JSON serialized |
+| `status` | string | `pending / running / completed / failed` |
+| `expires_at` | datetime | indexed; nil = never expires |
+| `created_at` | datetime | |
+| `updated_at` | datetime | |
+
+**`odsl_step_records`**
+
+| Column | Type | Notes |
+|--------|------|-------|
+| `id` | bigint PK | |
+| `execution_context_id` | bigint FK | indexed |
+| `name` | string | variable name from `.get(:x)` |
+| `step_index` | integer | execution order |
+| `status` | string | `completed / failed` |
+| `result` | jsonb/text | serialized result |
+| `error_message` | text | populated on failure |
+| `non_resumable` | boolean | true if result couldn't be serialized |
+| `started_at` | datetime | |
+| `completed_at` | datetime | |
+
+### Public API
+
+```ruby
+# Run with audit
+service.run(:create, key: 'user-123-game-chess')
+
+# Resume from first failed/pending step
+Games::Service.resume(key: 'user-123-game-chess')
+
+# Background with audit + idempotency
+service.run_in_background(:create, key: 'user-123-game-chess')
+```
+
+### Run flow (with key)
+
+1. `run(:method, key:)` finds or creates an `ExecutionContext` for the key.
+2. If the context is already `completed`, returns the stored result immediately (idempotency).
+3. If `running`, raises `Odsl::AlreadyRunningError`.
+4. Sets context status to `running`.
+5. Executes the method. On each `.get(:x)` call, records a `StepRecord` with the serialized result.
+6. On success: sets context status to `completed`.
+7. On failure: sets context and the failing step to `failed` with the error message.
+
+### Resume flow
+
+1. Load `ExecutionContext` by key; raise `Odsl::ContextNotFoundError` if missing.
+2. Collect names of already-`completed` steps.
+3. Instantiate the service with the original `kwargs`.
+4. Re-execute the method. When `.get(:x)` is called:
+   - If `x` is in the completed set → load result from `StepRecord`, skip re-execution of the preceding step.
+   - Otherwise → execute normally and record.
+5. Continue until method finishes or a new failure occurs.
+
+### Serialization contract
+
+Results stored via `.get(:x)` must be JSON-serializable for resume to work (hashes, arrays, strings, numbers, structs with `to_h`). If a result cannot be serialized, the step is marked `completed` in the audit but flagged as `non_resumable: true`. On resume, non-resumable steps are re-executed (safe because idempotency lives in the business logic).
+
+### TTL and cleanup
+
+```ruby
+Odsl.configure do |config|
+  config.audit_ttl = 30.days  # nil = keep forever
+end
+
+# Purge expired contexts (cascade deletes step_records)
+Odsl.cleanup_expired!
+
+# Via rake
+rake odsl:cleanup
+```
+
+`expires_at` is set on context creation from `config.audit_ttl`. Cleanup can be scheduled via Solid Queue scheduler, Sidekiq-Cron, Good Job cron, or a plain cron job — the gem only provides the method.
+
+---
+
+## 3. Distributed Locking (Redlock)
+
+### Motivation
+
+Idempotency handles *"already completed — skip"*. Redlock handles *"currently running in another process — block"*. Without it, two simultaneous requests with the same key can both pass the idempotency check before either has written to the DB.
+
+### Optional dependency
+
+Redlock requires Redis. If `config.lock_redis_url` is nil (default), calling `.lock()` in a service raises `Odsl::LockingNotConfiguredError` at boot time.
+
+### DSL method
+
+```ruby
+def create
+  lock(:player, :game)   # compose lock key from variable values
+
+  build(Game, Player)
+    .get(:game_object)
+  # ...
+end
+```
+
+The lock key is composed as:
+```
+odsl:{ServiceClass}:{method}:{var1_value}:{var2_value}
+```
+
+Example: `odsl:Games::Service:create:alejandro:chess`
+
+### Behavior
+
+- Lock is acquired at the start of `run` if `.lock()` is called in the method.
+- If the lock cannot be acquired → raises `Odsl::LockError` immediately (fail fast).
+- Lock is released when `run` finishes (success or error).
+- A TTL fallback ensures the lock is released even if the process dies.
+
+### Configuration
+
+```ruby
+Odsl.configure do |config|
+  config.lock_redis_url = 'redis://localhost:6379'  # nil = disabled
+  config.lock_ttl       = 10_000                    # milliseconds, fallback TTL
+end
+```
+
+### Error handling
+
+```ruby
+rescue Odsl::LockError
+  render json: { error: 'This action is already in progress.' }, status: :conflict
+
+rescue Odsl::AlreadyRunningError
+  # Raised when run() is called with a key that is currently status: running
+
+rescue Odsl::ContextNotFoundError
+  # Raised when resume() is called with a key that doesn't exist in the DB
+
+rescue Odsl::LockingNotConfiguredError
+  # Raised at boot if .lock() is used but no Redis URL is configured
+```
+
+---
+
+## 4. Generator
+
+```bash
+rails generate odsl:install
+```
+
+Generates:
+- `db/migrate/TIMESTAMP_create_odsl_tables.rb` — creates both tables with correct indexes
+- `config/initializers/odsl.rb` — configuration template with all options commented
+
+---
+
+## 5. New files summary
+
+```
+lib/odsl/
+  adapters/
+    base.rb
+    thread.rb
+    active_job.rb
+    odsl_job.rb        ← ActiveJob::Base subclass used by active_job adapter
+  audit/
+    execution_context.rb   ← ActiveRecord model
+    step_record.rb         ← ActiveRecord model
+    serializer.rb          ← JSON serialize/deserialize
+  locking/
+    redlock_adapter.rb     ← wraps redlock-rb gem
+    null_adapter.rb        ← no-op when locking not configured
+  configuration.rb         ← Odsl::Configuration + Odsl.configure
+  railtie.rb               ← Rails integration (generator, rake tasks)
+lib/generators/
+  odsl/
+    install_generator.rb
+    templates/
+      migration.rb.tt
+      initializer.rb.tt
+```
+
+---
+
+## 6. Backward compatibility
+
+- `run(:method)` with no `key:` — no audit, no locking, behaves exactly as today.
+- `run_in_background(:method)` with no `key:` — uses Thread adapter unless `background_adapter` is configured.
+- `.lock()` without Redis configured — raises `Odsl::LockingNotConfiguredError` at boot.
+- No new required dependencies — `redlock` and `activerecord` are optional; the gem works standalone.

data/lib/generators/odsl/install_generator.rb

@@ -0,0 +1,23 @@
+require "rails/generators"
+require "rails/generators/active_record"
+
+module Odsl
+  module Generators
+    class InstallGenerator < Rails::Generators::Base
+      include ActiveRecord::Generators::Migration
+
+      source_root File.expand_path("templates", __dir__)
+
+      desc "Creates an Odsl initializer and migration for audit tables"
+
+      def create_initializer
+        template "odsl_initializer.rb.erb", "config/initializers/odsl.rb"
+      end
+
+      def create_migration
+        migration_template "create_odsl_tables.rb.erb",
+                           "db/migrate/create_odsl_tables.rb"
+      end
+    end
+  end
+end

data/lib/generators/odsl/templates/create_odsl_tables.rb.erb

@@ -0,0 +1,29 @@
+class CreateOdslTables < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
+  def change
+    create_table :odsl_execution_contexts do |t|
+      t.string   :idempotency_key, null: false
+      t.string   :service_class,   null: false
+      t.string   :method_name,     null: false
+      t.text     :kwargs
+      t.string   :status,          null: false, default: "pending"
+      t.datetime :expires_at
+      t.timestamps
+    end
+
+    add_index :odsl_execution_contexts, :idempotency_key, unique: true
+    add_index :odsl_execution_contexts, :expires_at
+
+    create_table :odsl_step_records do |t|
+      t.references :execution_context, null: false,
+                   foreign_key: { to_table: :odsl_execution_contexts }
+      t.string   :name,          null: false
+      t.integer  :step_index,    null: false
+      t.string   :status,        null: false, default: "pending"
+      t.text     :result
+      t.text     :error_message
+      t.boolean  :non_resumable, default: false
+      t.datetime :started_at
+      t.datetime :completed_at
+    end
+  end
+end

data/lib/generators/odsl/templates/odsl_initializer.rb.erb

@@ -0,0 +1,13 @@
+Odsl.configure do |config|
+  # Background adapter: :thread (default) or :active_job
+  # config.background_adapter = :active_job
+
+  # How long to keep audit records (in seconds). nil = keep forever.
+  # config.audit_ttl = 7 * 24 * 3600  # 1 week
+
+  # Redis URL for distributed locking via Redlock. nil = no locking.
+  # config.lock_redis_url = ENV["REDIS_URL"]
+
+  # Lock TTL in milliseconds (default: 10_000 = 10 seconds)
+  # config.lock_ttl = 10_000
+end

data/lib/odsl/adapters/active_job.rb

@@ -0,0 +1,21 @@
+# lib/odsl/adapters/active_job.rb
+module Odsl
+  module Adapters
+    class ActiveJob < Base
+      def self.enqueue(service_class:, method_name:, kwargs:, key: nil, callable: nil)
+        unless defined?(::ActiveJob)
+          raise Odsl::Error,
+            "ActiveJob is not available. Add `gem 'activejob'` to your Gemfile."
+        end
+
+        require_relative "odsl_job"
+        Odsl::OdslJob.perform_later(
+          service_class: service_class,
+          method_name:   method_name,
+          kwargs:        kwargs,
+          key:           key
+        )
+      end
+    end
+  end
+end

data/lib/odsl/adapters/base.rb

@@ -0,0 +1,11 @@
+# lib/odsl/adapters/base.rb
+module Odsl
+  module Adapters
+    class Base
+      def self.enqueue(service_class:, method_name:, kwargs:, key: nil, callable: nil)
+        raise NotImplementedError,
+          "#{self} must implement .enqueue(service_class:, method_name:, kwargs:, key:, callable:)"
+      end
+    end
+  end
+end

data/lib/odsl/adapters/odsl_job.rb

@@ -0,0 +1,12 @@
+# lib/odsl/adapters/odsl_job.rb
+# Only loaded when ActiveJob is available
+if defined?(ActiveJob)
+  module Odsl
+    class OdslJob < ActiveJob::Base
+      def perform(service_class:, method_name:, kwargs:, key: nil)
+        klass = Object.const_get(service_class)
+        klass.new(**kwargs.transform_keys(&:to_sym)).run(method_name.to_sym, key: key)
+      end
+    end
+  end
+end

data/lib/odsl/adapters/thread.rb

@@ -0,0 +1,17 @@
+# lib/odsl/adapters/thread.rb
+module Odsl
+  module Adapters
+    class Thread < Base
+      def self.enqueue(service_class:, method_name:, kwargs:, key: nil, callable: nil)
+        ::Thread.new do
+          if callable
+            callable.call
+          else
+            klass = Object.const_get(service_class)
+            klass.new(**kwargs.transform_keys(&:to_sym)).run(method_name.to_sym, key: key)
+          end
+        end
+      end
+    end
+  end
+end

data/lib/odsl/audit/execution_context.rb

@@ -0,0 +1,34 @@
+# lib/odsl/audit/execution_context.rb
+require "active_record"
+require "json"
+
+module Odsl
+  module Audit
+    class ExecutionContext < ActiveRecord::Base
+      self.table_name = "odsl_execution_contexts"
+
+      has_many :step_records,
+               class_name:  "Odsl::Audit::StepRecord",
+               foreign_key: :execution_context_id,
+               dependent:   :destroy
+
+      STATUSES = %w[pending running completed failed].freeze
+
+      validates :idempotency_key, presence: true, uniqueness: true
+      validates :service_class,   presence: true
+      validates :method_name,     presence: true
+      validates :status,          inclusion: { in: STATUSES }
+
+      scope :expired, -> { where("expires_at IS NOT NULL AND expires_at < ?", Time.now) }
+
+      def kwargs_hash
+        return {} if kwargs.nil?
+        JSON.parse(kwargs).transform_keys(&:to_sym)
+      end
+
+      def completed? = status == "completed"
+      def running?   = status == "running"
+      def failed?    = status == "failed"
+    end
+  end
+end

data/lib/odsl/audit/serializer.rb

@@ -0,0 +1,35 @@
+# lib/odsl/audit/serializer.rb
+require "json"
+
+module Odsl
+  module Audit
+    module Serializer
+      def self.dump(value)
+        JSON.generate(serialize(value))
+      end
+
+      def self.load(json_string)
+        return nil if json_string.nil?
+        JSON.parse(json_string)
+      end
+
+      def self.serializable?(value)
+        dump(value)
+        true
+      rescue JSON::GeneratorError, TypeError
+        false
+      end
+
+      def self.serialize(value)
+        case value
+        when Hash                                          then value.transform_keys(&:to_s).transform_values { |v| serialize(v) }
+        when Array                                         then value.map { |v| serialize(v) }
+        when Struct                                        then serialize(value.to_h)
+        when String, Numeric, TrueClass, FalseClass, NilClass then value
+        else raise TypeError, "Cannot serialize #{value.class}"
+        end
+      end
+      private_class_method :serialize
+    end
+  end
+end

data/lib/odsl/audit/step_record.rb

@@ -0,0 +1,27 @@
+# lib/odsl/audit/step_record.rb
+require "active_record"
+
+module Odsl
+  module Audit
+    class StepRecord < ActiveRecord::Base
+      self.table_name = "odsl_step_records"
+
+      belongs_to :execution_context,
+                 class_name:  "Odsl::Audit::ExecutionContext",
+                 foreign_key: :execution_context_id
+
+      STATUSES = %w[pending completed failed].freeze
+
+      validates :name,       presence: true
+      validates :step_index, presence: true
+      validates :status,     inclusion: { in: STATUSES }
+
+      def completed? = status == "completed"
+      def failed?    = status == "failed"
+
+      def deserialized_result
+        Audit::Serializer.load(result)
+      end
+    end
+  end
+end

data/lib/odsl/configuration.rb

@@ -0,0 +1,36 @@
+module Odsl
+  class Configuration
+    attr_accessor :background_adapter, :audit_ttl,
+                  :lock_redis_url, :lock_ttl
+
+    def initialize
+      @background_adapter = :thread
+      @audit_ttl          = nil
+      @lock_redis_url     = nil
+      @lock_ttl           = 10_000
+    end
+  end
+
+  CONFIG_MUTEX = Mutex.new
+
+  class << self
+    def configure
+      yield(config) if block_given?
+    end
+
+    def config
+      CONFIG_MUTEX.synchronize { @config ||= Configuration.new }
+    end
+
+    def reset_config!
+      CONFIG_MUTEX.synchronize { @config = Configuration.new }
+    end
+
+    def cleanup_expired!
+      return 0 unless defined?(Odsl::Audit::ExecutionContext)
+      count = Odsl::Audit::ExecutionContext.expired.count
+      Odsl::Audit::ExecutionContext.expired.destroy_all
+      count
+    end
+  end
+end

data/lib/odsl/dsl.rb

@@ -10,18 +10,17 @@ module Odsl
     extend Configurable
 
     def initialize(**kwargs)
-
       @delegated       = self
       @kwargs          = kwargs
       @context         = {}
       @last_results    = []
       @with_attributes = nil
       @finish_with     = nil
-  
+
       kwargs.each do |key, value|
         instance_variable_set("@#{key}", value)
         @context[key] = value
-      end     
+      end
     end
 
     class << self
@@ -30,41 +29,92 @@ module Odsl
       def inmutable(bool)
         @inmutable_mode = bool
       end
-    end   
-  
+
+      def subscribe(message, to:)
+        subscriptions[message] = to
+      end
+
+      def subscriptions
+        @subscriptions ||= {}
+      end
+
+      def dispatch(message, **kwargs)
+        method_name = subscriptions[message]
+        raise Error, "No subscription for '#{message}'" unless method_name
+
+        new(**kwargs).run(method_name)
+      end
+
+      def resume(key:)
+        ctx = Odsl::Audit::ExecutionContext.find_by(idempotency_key: key)
+        raise Odsl::ContextNotFoundError, "No execution context found for key: #{key}" unless ctx
+        raise Odsl::AlreadyRunningError, "Job with key '#{key}' is already running" if ctx.running?
+        return nil if ctx.completed?
+
+        kwargs   = ctx.kwargs_hash
+        instance = new(**kwargs)
+
+        completed_step_names = ctx.step_records
+                                  .where(status: "completed", non_resumable: false)
+                                  .pluck(:name)
+
+        instance.instance_variable_set(:@_execution_context, ctx)
+        instance.instance_variable_set(:@_resuming_steps,    completed_step_names)
+        instance.send(:_resume_method, ctx.method_name.to_sym, key: key)
+      end
+    end
+
     protected
 
     def instantiate(object, public: false)
-
       resolved_attrs = with_attributes_resolver
 
-      @last_results = [ ] << if resolved_attrs.any?
-
+      @last_results = [] << if resolved_attrs.any?
         resolved_attrs.is_a?(Array) ? object.new(*resolved_attrs) : object.new(**resolved_attrs)
       else
         object.new
-      end     
+      end
 
       self
     end
 
     def build(*modules)
-      @last_results = [ Odsl::Builder.build(*modules).new ]
+      @last_results = [Odsl::Builder.build(*modules).new]
       self
     end
-  
+
     def with(*arguments)
       @with_attributes = arguments
       self
     end
-  
+
     def value_return(*with)
       @finish_with = with
     end
-     
-    alias make      call
-    alias link      call
-    alias get       to
-    alias execute   run
+
+    def lock(*vars)
+      redis_url = Odsl.config.lock_redis_url
+      return unless redis_url
+
+      require "redlock"
+      require_relative "locking/lock_handle"
+
+      key_parts = [self.class.name] + vars.map { |v| instance_variable_get("@#{v}").to_s }
+      lock_key  = key_parts.join(":")
+      ttl       = Odsl.config.lock_ttl
+
+      client    = Redlock::Client.new([redis_url])
+      lock_info = client.lock(lock_key, ttl)
+      raise Odsl::LockError, "Could not acquire lock for key: #{lock_key}" unless lock_info
+
+      @_current_lock = Odsl::Locking::LockHandle.new(client, lock_info)
+    end
+
+    alias make        call
+    alias link        call
+    alias set         call
+    alias get         to
+    alias execute     run
+    alias finish_with value_return
   end
 end

data/lib/odsl/errors.rb

@@ -0,0 +1,8 @@
+module Odsl
+  class Error < StandardError; end
+
+  class LockError                 < Error; end
+  class LockingNotConfiguredError < Error; end
+  class AlreadyRunningError       < Error; end
+  class ContextNotFoundError      < Error; end
+end

data/lib/odsl/locking/lock_handle.rb

@@ -0,0 +1,9 @@
+module Odsl
+  module Locking
+    LockHandle = Struct.new(:client, :lock_info) do
+      def unlock
+        client.unlock(lock_info)
+      end
+    end
+  end
+end