igniter 0.3.1 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 659dfe833fdf98b7d1b446e08d02464545fd0fe9b61badd5cf8e7ca73beb6a1b
-  data.tar.gz: 5af168ce8c6fad1c18dd0d9703281ea64062c3a6d3e954b12172f29210e12c5b
+  metadata.gz: 00ad08d39fff726f2d88cefe18758dca71dc7b6fbccce872199347e23bf86e9a
+  data.tar.gz: 42791bdf591fbab256bb32a7da1b37466317e0c190d3693830ee82885f3fe238
 SHA512:
-  metadata.gz: 5865bc3fb30c137baa5bdf87b63f9023c95c50afa87701a052a47726d139ab6113ed52820f629d0b45de421247778d560133e2c2d2f5e756681a14af2cac0026
-  data.tar.gz: de8fa4b7f92a566016f21d81acdbf9d5ec4f4f0fa458ac99b18f8c5cced124390276ea541a1f252067009fd4e3ccf688d97d239080b8995227fc697f0d5b9e6e
+  metadata.gz: 45034ac4e94ee4c1864a8ca5222505d2192c3cf4667c1ba6149161cf15079c7000a80c75e2dad6baafb79dfde0b9cb5657fd0ffd2d869f3600210ec3429268a9
+  data.tar.gz: 4f797eb253d945fd459f2ef29b64a4330616ecfcc0d8d6f1dfbbf98da24bfe090da6e6c28cda2ddf8203f71c52af98ec275cb157829febd032ab9846ccc883e8

data/docs/DISTRIBUTED_CONTRACTS_V1.md

@@ -0,0 +1,493 @@
+# Distributed Contracts v1
+
+## Goal
+
+`Distributed Contracts` extend `igniter` from in-process orchestration to long-running, event-driven workflow execution.
+
+The feature should make distributed workflows:
+
+- explicit
+- correlation-aware
+- resumable
+- observable end to end
+- compatible with the existing graph model
+
+It should avoid pushing cross-system workflow state into ad hoc service objects, background jobs, or loosely related tables.
+
+## Problem Shape
+
+Typical legacy pain looks like this:
+
+- one business flow starts from an inbound request
+- later signals arrive from different systems
+- signals must be matched by business identity, not by process memory
+- some steps complete immediately, others wait minutes or hours
+- failures are not local, they are causal and temporal
+
+Examples:
+
+- vendor request -> callrail webhook -> ringcentral webhook -> operator match -> order creation -> billing decision -> invoice
+- lead accepted now, enrichment later
+- call started now, attribution resolved later
+
+`igniter` already has useful building blocks:
+
+- `pending`
+- snapshot / restore
+- store-backed execution
+- token-based resume
+- diagnostics and audit
+
+Distributed Contracts v1 should formalize these into a workflow model.
+
+## Core Principles
+
+### Correlation first
+
+The primary question is not:
+
+- "which service should run next?"
+
+It is:
+
+- "which execution does this external signal belong to?"
+
+Distributed execution must be resumable by stable business correlation keys.
+
+### External signals are graph-visible
+
+Waiting for an external system should be represented in the graph model, not hidden inside generic polling or callback code.
+
+### Transport stays outside
+
+Controllers, jobs, consumers, and webhook adapters remain outside `igniter`.
+
+They should translate transport concerns into workflow operations such as:
+
+- start workflow
+- deliver signal
+- resume execution
+
+`igniter` should not become a message bus.
+
+## Proposed v1 Surface
+
+### Starting a distributed workflow
+
+```ruby
+execution = CallLifecycleContract.start(
+  request_id: "req-123",
+  company_id: "42",
+  vendor_lead_id: "lead-77"
+)
+```
+
+This is conceptually similar to `new(...).resolve`, but v1 should expose a clearer workflow-oriented entry point.
+
+### Correlation metadata
+
+```ruby
+class CallLifecycleContract < Igniter::Contract
+  correlate_by :request_id, :company_id, :vendor_lead_id
+end
+```
+
+This should define the business keys used to find the execution later.
+
+Suggested v1 requirements:
+
+- correlation keys are declared explicitly
+- correlation values come from initial inputs and/or delivered signals
+- the execution store can index and look up executions by correlation values
+
+### Awaiting an external signal
+
+```ruby
+await :callrail_call,
+  event: :callrail_webhook_received
+
+await :ringcentral_match,
+  event: :ringcentral_call_matched
+```
+
+This is preferable to generic `defer` for distributed flows because it makes the waiting state explicit in the graph.
+
+### Delivering an external signal
+
+```ruby
+CallLifecycleContract.deliver_event(
+  :callrail_webhook_received,
+  correlation: { request_id: "req-123", company_id: "42" },
+  payload: { call_id: "cr-1", started_at: "2026-03-19T10:00:00Z" }
+)
+```
+
+Expected behavior:
+
+1. Find the matching execution by correlation.
+2. Find a matching `await` node for the event.
+3. Bind the payload to that waiting node.
+4. Resume the workflow.
+
+### Diagnostics for waiting workflows
+
+```ruby
+contract.diagnostics.to_h
+```
+
+Should make waiting state explicit:
+
+- current status
+- known correlation keys
+- expected external events
+- last delivered event
+- current blocked / waiting nodes
+
+## Scope of v1
+
+Supported:
+
+- explicit correlation keys
+- explicit external wait nodes
+- event delivery by correlation
+- store-backed lookup and resume
+- diagnostics for waiting state
+- audit trail of delivered external signals
+
+Not supported in v1:
+
+- message bus integration
+- retries / dead-letter routing
+- event schema registry
+- compensation / saga semantics
+- multi-execution joins
+- time-based wakeups or cron waits
+- out-of-order signal reconciliation beyond simple correlation lookup
+
+## DSL
+
+### Basic shape
+
+```ruby
+class CallLifecycleContract < Igniter::Contract
+  correlate_by :request_id, :company_id, :vendor_lead_id
+
+  define do
+    input :request_id, type: :string
+    input :company_id, type: :string
+    input :vendor_lead_id, type: :string
+
+    await :callrail_call, event: :callrail_webhook_received
+    await :ringcentral_match, event: :ringcentral_call_matched
+
+    aggregate :billing_context, with: %i[callrail_call ringcentral_match] do |callrail_call:, ringcentral_match:|
+      {
+        call_id: callrail_call[:call_id],
+        operator_id: ringcentral_match[:operator_id],
+        channel: ringcentral_match[:channel]
+      }
+    end
+
+    output :billing_context
+  end
+end
+```
+
+### Signal delivery
+
+```ruby
+CallLifecycleContract.deliver_event(
+  :ringcentral_call_matched,
+  correlation: {
+    request_id: "req-123",
+    company_id: "42",
+    vendor_lead_id: "lead-77"
+  },
+  payload: {
+    operator_id: 55,
+    channel: "paid_search"
+  }
+)
+```
+
+## Why `await` instead of reusing generic `defer`
+
+`defer` is useful as a low-level runtime primitive.
+
+But distributed workflows need more explicit semantics:
+
+- what event is being awaited
+- how to resume it
+- how to show it in diagnostics
+- how to route external signals safely
+
+`await` should be modeled as a graph primitive, not just a custom executor returning `DeferredResult`.
+
+## Runtime Semantics
+
+### Starting execution
+
+1. Create a normal execution.
+2. Persist workflow metadata:
+   - execution id
+   - graph name
+   - correlation keys
+   - workflow status
+3. Resolve until:
+   - success
+   - failure
+   - waiting on one or more `await` nodes
+
+### Await resolution
+
+When an `await` node is reached:
+
+1. Mark node status as `waiting_for_event`
+2. Persist expected event metadata
+3. Stop downstream execution until signal arrives
+
+### Event delivery
+
+When `deliver_event` is called:
+
+1. Resolve execution lookup by correlation
+2. Validate event name
+3. Match event to a waiting `await` node
+4. Persist audit metadata for the incoming signal
+5. Resume execution with delivered payload
+
+### Terminal states
+
+Suggested workflow-level statuses:
+
+- `running`
+- `waiting`
+- `succeeded`
+- `failed`
+
+Later possible additions:
+
+- `cancelled`
+- `timed_out`
+- `abandoned`
+
+## Graph Model
+
+Distributed Contracts v1 should introduce a dedicated node kind:
+
+- `:await`
+
+Suggested internal shape:
+
+```ruby
+AwaitNode.new(
+  name: :ringcentral_match,
+  event: :ringcentral_call_matched
+)
+```
+
+This should not be modeled as generic compute.
+
+## Correlation Model
+
+Correlation should be explicit and inspectable.
+
+Suggested v1 API:
+
+```ruby
+correlate_by :request_id, :company_id, :vendor_lead_id
+```
+
+Suggested stored metadata:
+
+```ruby
+{
+  request_id: "req-123",
+  company_id: "42",
+  vendor_lead_id: "lead-77"
+}
+```
+
+V1 should assume exact equality matching.
+
+Future versions may allow:
+
+- computed correlation values
+- partial lookup
+- secondary indexes
+
+## Store Requirements
+
+The execution store needs more than snapshot persistence.
+
+Distributed Contracts v1 should require:
+
+- fetch by `execution_id`
+- save snapshot
+- delete snapshot
+- lookup execution ids by correlation keys
+- persist workflow metadata
+
+Conceptually:
+
+```ruby
+store.save(snapshot:, metadata:)
+store.find_by_correlation(graph: "CallLifecycleContract", correlation: { ... })
+```
+
+This probably means a new workflow-aware store interface, not just extending the current minimal snapshot store informally.
+
+## Compile-Time Validation
+
+The compiler should validate:
+
+- declared correlation keys exist as contract inputs
+- `await` node names are unique
+- awaited event names are unique within a graph unless explicitly allowed
+- `await` nodes are valid dependencies for downstream nodes
+
+Suggested v1 restriction:
+
+- one `await` node per event name within a contract
+
+This keeps signal delivery unambiguous.
+
+## Runtime Validation
+
+At runtime, delivery should validate:
+
+- the target execution exists
+- the execution belongs to the expected contract graph
+- the event is currently awaited
+- the same awaited signal is not delivered twice unless idempotency policy allows it
+
+Suggested runtime errors:
+
+- `Igniter::WorkflowNotFoundError`
+- `Igniter::AwaitedEventMismatchError`
+- `Igniter::DuplicateSignalError`
+
+## Events and Audit
+
+Distributed Contracts v1 should add workflow-aware events.
+
+Suggested additions:
+
+- `workflow_started`
+- `workflow_waiting`
+- `workflow_resumed`
+- `external_event_received`
+- `await_satisfied`
+
+Suggested payload for incoming external signal:
+
+```ruby
+{
+  event: :ringcentral_call_matched,
+  correlation: {
+    request_id: "req-123",
+    company_id: "42",
+    vendor_lead_id: "lead-77"
+  }
+}
+```
+
+Audit should make it possible to answer:
+
+- which external events were received
+- in what order
+- which waiting node they satisfied
+- why the workflow is still blocked
+
+## Introspection
+
+### Graph
+
+`await` nodes should render distinctly from compute, branch, collection, and composition.
+
+The graph should show:
+
+- awaited event name
+- correlation keys at the workflow level
+
+### Plan
+
+Before execution:
+
+- `await` nodes look like normal pending nodes
+
+When waiting:
+
+- plan should show explicit waiting state
+- ideally also the awaited event name
+
+### Diagnostics
+
+Diagnostics should surface:
+
+- workflow status
+- correlation keys
+- awaited events
+- satisfied events
+- last external event
+- blocked nodes
+
+Example conceptual shape:
+
+```ruby
+{
+  status: :waiting,
+  correlation: {
+    request_id: "req-123",
+    company_id: "42"
+  },
+  waiting_on: [
+    { node: :ringcentral_match, event: :ringcentral_call_matched }
+  ],
+  last_external_event: :callrail_webhook_received
+}
+```
+
+## Relation to Existing Pending Support
+
+Distributed Contracts v1 should build on existing pending/store/resume support instead of replacing it.
+
+Conceptually:
+
+- `await` is a higher-level workflow primitive
+- internally it may still use pending state and resume mechanics
+- `deliver_event` is a safer, domain-oriented wrapper over generic token resume
+
+This keeps the runtime coherent and incremental.
+
+## Recommended Architectural Pattern
+
+Do not start with one giant distributed contract from request to invoice.
+
+Prefer stage-oriented workflow design:
+
+- `InboundLeadContract`
+- `CallAttributionContract`
+- `OrderBillingContract`
+- optional orchestration shell above them
+
+This keeps contracts:
+
+- understandable
+- diagnosable
+- evolvable
+
+## Future Extensions
+
+Possible later additions:
+
+- event payload schema validation
+- timeout / expiry policies on `await`
+- idempotency keys for signal delivery
+- compensation hooks
+- `await_any` / `await_all`
+- workflow versioning and migration
+- cross-workflow linking
+- visual workflow tracing
+
+These should not be part of v1.

data/examples/distributed_workflow.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+$LOAD_PATH.unshift File.join(__dir__, "../lib")
+require "igniter"
+
+store = Igniter::Runtime::Stores::MemoryStore.new
+
+class LeadWorkflow < Igniter::Contract
+  correlate_by :request_id, :company_id
+
+  define do
+    input :request_id
+    input :company_id
+
+    await :crm_data, event: :crm_webhook_received
+    await :billing_data, event: :billing_data_fetched
+
+    aggregate :report, with: %i[crm_data billing_data] do |crm_data:, billing_data:|
+      { crm: crm_data, billing: billing_data }
+    end
+
+    output :report
+  end
+end
+
+puts "==> Starting execution..."
+execution = LeadWorkflow.start(
+  { request_id: "req-1", company_id: "co-42" },
+  store: store
+)
+puts "pending? #{execution.pending?}"
+
+puts "\n==> Delivering CRM event..."
+execution = LeadWorkflow.deliver_event(
+  :crm_webhook_received,
+  correlation: { request_id: "req-1", company_id: "co-42" },
+  payload: { name: "Acme Corp", tier: "enterprise" },
+  store: store
+)
+puts "still pending? #{execution.pending?}"
+
+puts "\n==> Delivering billing event..."
+execution = LeadWorkflow.deliver_event(
+  :billing_data_fetched,
+  correlation: { request_id: "req-1", company_id: "co-42" },
+  payload: { plan: "pro", mrr: 500 },
+  store: store
+)
+
+puts "\n==> Final result:"
+puts "success? #{execution.success?}"
+puts "report: #{execution.result.report.inspect}"

data/lib/igniter/compiler/compiled_graph.rb

@@ -49,6 +49,10 @@ module Igniter
         raise KeyError, "Unknown dependency '#{name}'"
       end
 
+      def await_nodes
+        @nodes.select { |n| n.kind == :await }
+      end
+
       def to_h
         {
           name: name,
@@ -80,6 +84,7 @@ module Igniter
               base[:mode] = node.mode
               base[:mapper] = node.input_mapper.to_s if node.input_mapper?
             end
+            base[:event] = node.event_name if node.kind == :await
             base
           end,
           outputs: outputs.map do |output|
@@ -117,6 +122,13 @@ module Igniter
               metadata: node.metadata.reject { |key, _| key == :source_location }
             }
           end,
+          awaits: nodes.select { |node| node.kind == :await }.map do |node|
+            {
+              name: node.name,
+              event: node.event_name,
+              metadata: node.metadata.reject { |key, _| key == :source_location }
+            }
+          end,
           branches: nodes.select { |node| node.kind == :branch }.map do |node|
             {
               name: node.name,

data/lib/igniter/compiler/validation_pipeline.rb

@@ -8,7 +8,9 @@ module Igniter
         Validators::OutputsValidator,
         Validators::DependenciesValidator,
         Validators::TypeCompatibilityValidator,
-        Validators::CallableValidator
+        Validators::CallableValidator,
+        Validators::AwaitValidator,
+        Validators::RemoteValidator
       ].freeze
 
       def self.call(context, validators: DEFAULT_VALIDATORS)

data/lib/igniter/compiler/validators/await_validator.rb

@@ -0,0 +1,53 @@
+# frozen_string_literal: true
+
+module Igniter
+  module Compiler
+    module Validators
+      class AwaitValidator
+        def self.call(context)
+          new(context).call
+        end
+
+        def initialize(context)
+          @context = context
+        end
+
+        def call
+          await_nodes = @context.runtime_nodes.select { |n| n.kind == :await }
+          return if await_nodes.empty?
+
+          validate_correlation_keys_as_inputs!(await_nodes)
+          validate_unique_event_names!(await_nodes)
+        end
+
+        private
+
+        def validate_correlation_keys_as_inputs!(await_nodes) # rubocop:disable Metrics/AbcSize
+          correlation_keys = @context.graph.metadata[:correlation_keys] || []
+          return if correlation_keys.empty?
+
+          input_names = @context.runtime_nodes.select { |n| n.kind == :input }.map(&:name)
+          missing = correlation_keys.reject { |key| input_names.include?(key.to_sym) }
+          return if missing.empty?
+
+          raise @context.validation_error(
+            await_nodes.first,
+            "Correlation keys #{missing.inspect} must be declared as inputs"
+          )
+        end
+
+        def validate_unique_event_names!(await_nodes)
+          event_names = await_nodes.map(&:event_name)
+          duplicates = event_names.select { |e| event_names.count(e) > 1 }.uniq
+          return if duplicates.empty?
+
+          node = await_nodes.find { |n| duplicates.include?(n.event_name) }
+          raise @context.validation_error(
+            node,
+            "Duplicate await event names: #{duplicates.inspect}"
+          )
+        end
+      end
+    end
+  end
+end

data/lib/igniter/compiler/validators/dependencies_validator.rb

@@ -12,8 +12,10 @@ module Igniter
           @context = context
         end
 
-        def call
+        def call # rubocop:disable Metrics/AbcSize,Metrics/CyclomaticComplexity
           @context.runtime_nodes.each do |node|
+            next if node.kind == :await
+
             validate_composition_node!(node) if node.kind == :composition
             validate_branch_node!(node) if node.kind == :branch
             validate_collection_node!(node) if node.kind == :collection
@@ -40,6 +42,7 @@ module Igniter
           end
 
           validate_composition_input_mapping!(node, contract_class.compiled_graph)
+          validate_composition_cycle!(node)
         end
 
         def validate_composition_input_mapping!(node, child_graph)
@@ -126,6 +129,43 @@ module Igniter
           )
         end
 
+        def validate_composition_cycle!(node)
+          child_contract = node.contract_class
+          return unless child_contract.respond_to?(:compiled_graph) && child_contract.compiled_graph
+
+          current_name = @context.graph.name
+          # Skip anonymous contracts to avoid false positives when multiple
+          # anonymous contracts share the same name "AnonymousContract"
+          return if current_name == "AnonymousContract"
+
+          validate_direct_cycle!(node, child_contract, current_name)
+          validate_grandchild_cycles!(node, child_contract, current_name)
+        end
+
+        def validate_direct_cycle!(node, child_contract, current_name)
+          return unless child_contract.compiled_graph.name == current_name
+
+          raise @context.validation_error(
+            node,
+            "Composition cycle: '#{node.name}' composes '#{child_contract.name}' " \
+            "which is the same contract ('#{current_name}')"
+          )
+        end
+
+        def validate_grandchild_cycles!(node, child_contract, current_name) # rubocop:disable Metrics/AbcSize
+          child_contract.compiled_graph.nodes.select { |n| n.kind == :composition }.each do |grandchild|
+            next unless grandchild.contract_class.respond_to?(:compiled_graph)
+            next unless grandchild.contract_class.compiled_graph
+            next unless grandchild.contract_class.compiled_graph.name == current_name
+
+            raise @context.validation_error(
+              node,
+              "Composition cycle: '#{node.name}' -> '#{child_contract.name}' -> " \
+              "'#{grandchild.contract_class.name}' loops back to '#{current_name}'"
+            )
+          end
+        end
+
         def validate_collection_node!(node)
           unless node.contract_class.is_a?(Class) && node.contract_class <= Igniter::Contract
             raise @context.validation_error(node, "Collection '#{node.name}' must reference an Igniter::Contract subclass")