flow_chat 0.9.0 → 0.10.0

data/docs/architecture.md

@@ -1,510 +1,83 @@
-# FlowChat Architecture
+# Architecture
 
-FlowChat is built around a **composition-based architecture** with **pluggable gateways** that enables unified conversational interfaces across multiple platforms. This document explains the core architectural decisions and how they enable FlowChat's flexibility.
+FlowChat turns a stateless webhook into a stateful conversation by re-running your flow method from the top on every request and replaying its answers out of a session. This document explains the pieces that make that work.
 
-## Design Principles
+## The middleware stack
 
-### 1. Platform Abstraction
+A `FlowChat::Processor` builds one middleware stack per request, in a fixed order. Each layer wraps the next and calls it:
 
-FlowChat abstracts platform differences through a unified API, allowing the same flow code to work across USSD, WhatsApp, Telegram, HTTP, and custom platforms.
-
-```ruby
-# This exact code works on ALL platforms
-def survey_flow
-  name = app.screen(:name) { |p| p.ask "What's your name?" }
-  rating = app.screen(:rating) { |p| p.select "Rate us:", ["1", "2", "3", "4", "5"] }
-  app.say "Thanks #{name}! You rated us #{rating}/5"
-end
-```
-
-### 2. Composition Over Inheritance
-
-Rather than platform-specific classes, FlowChat uses composition to configure behavior:
-
-```ruby
-# Old approach (inheritance) - DEPRECATED
-class UssdApp < FlowChat::BaseApp; end
-class WhatsappApp < FlowChat::BaseApp; end
-
-# New approach (composition) - CURRENT
-processor = FlowChat::Processor.new(self) do |config|
-  config.use_gateway FlowChat::Ussd::Gateway::Nalo        # Platform behavior
-  config.use_session_store FlowChat::Session::CacheSessionStore  # Session behavior
-  config.use_middleware CustomLoggingMiddleware           # Custom behavior
-end
 ```
-
-### 3. Pluggable Gateways
-
-Each platform can have multiple gateway implementations, making FlowChat extensible:
-
-```ruby
-# Built-in USSD gateway
-config.use_gateway FlowChat::Ussd::Gateway::Nalo
-
-# Example custom USSD gateways (you would build these)
-config.use_gateway YourCompany::Ussd::Gateway::Africaist
-config.use_gateway YourCompany::Ussd::Gateway::MTN
-
-# Built-in WhatsApp gateway
-config.use_gateway FlowChat::Whatsapp::Gateway::CloudApi
-
-# Example custom WhatsApp gateways (you would build these)
-config.use_gateway YourCompany::Whatsapp::Gateway::OnPremise
-config.use_gateway YourCompany::Whatsapp::Gateway::Twilio
-
-# Example custom platforms (you would build these)
-config.use_gateway YourCompany::Sms::Gateway::Twilio
-config.use_gateway YourCompany::Voice::Gateway::Plivo
-config.use_gateway YourCompany::Slack::Gateway::BoltJS
+Gateway
+  -> Session::Middleware
+    -> platform middleware (and your custom middleware)
+      -> Executor
+        -> your Flow
 ```
 
-## Core Components
-
-### Processor
-
-The `FlowChat::Processor` is the central orchestrator that builds and executes the middleware stack.
-
-```ruby
-processor = FlowChat::Processor.new(controller) do |config|
-  config.use_gateway GatewayClass, *gateway_args
-  config.use_session_store SessionStoreClass
-  config.use_middleware CustomMiddleware
-  config.use_session_config(boundaries: [:flow], identifier: :msisdn)
-end
-
-processor.run FlowClass, :action_method
-```
+The order is set in `Processor#create_middleware_stack`:
 
-**Responsibilities:**
-- Build middleware stack based on configuration
-- Manage session configuration
-- Coordinate gateway and middleware interaction
-- Handle errors and instrumentation
+1. **Gateway** runs first. It parses the platform's webhook into normalized context values (`request.msisdn`, `context.input`, and so on) and, after the inner stack returns, renders the result back to the platform. See [gateway-development.md](gateway-development.md).
+2. **`Session::Middleware`** computes the session id from the configured boundaries and identifier, and attaches the session store. See [configuration.md](configuration.md#sessions).
+3. **Platform middleware** is inserted by the gateway if it defines `configure_middleware_stack`. USSD uses this to add pagination and choice-number mapping, and it is where your custom middleware runs. A gateway without that hook just runs your custom middleware here.
+4. **Executor** runs last. Nothing runs after it. It instantiates your flow and calls the action.
 
-### Gateway
+## The Executor and control flow by exception
 
-Gateways handle platform-specific request parsing and response rendering.
+`FlowChat::Executor` builds a `FlowChat::App`, instantiates your flow with it, and calls the action method. Control flow is driven by exceptions raised from inside the flow and caught here:
 
 ```ruby
-class MyCustomGateway
-  def initialize(app, *config_args)
-    @app = app
-    @config = config_args.first
-  end
-
-  def call(context)
-    # 1. Parse platform-specific request
-    context["request.msisdn"] = extract_phone_number(request)
-    context["request.platform"] = :my_platform
-    context.input = extract_user_input(request)
-    
-    # 2. Process through middleware stack
-    type, prompt, choices, media = @app.call(context)
-    
-    # 3. Render platform-specific response
-    send_platform_response(type, prompt, choices, media)
-  end
-
-  # Optional: Configure platform-specific middleware
-  def self.configure_middleware_stack(builder, custom_middleware)
-    builder.use MyPlatform::SpecialMiddleware
-    builder.use custom_middleware
-    builder.use MyPlatform::ResponseMiddleware
-  end
-end
-```
-
-**Gateway Interface:**
-- `initialize(app, *args)` - Set up gateway with app and config
-- `call(context)` - Process request through middleware stack
-- `self.configure_middleware_stack(builder, custom_middleware)` - Optional middleware configuration
-
-### App
-
-The `FlowChat::App` provides the unified interface that flows use to interact with users.
-
-```ruby
-class FlowChat::App
-  def screen(key, &block)
-    # Automatic state management and navigation
-  end
-  
-  def say(message, media: nil)
-    # Platform-appropriate message sending
-  end
-  
-  # Context accessors that work across platforms
-  def msisdn, user_id, platform, gateway
-  def message_id, timestamp, contact_name, location, media
-end
-```
-
-**Key Features:**
-- **Screen-based navigation** with automatic state management
-- **Platform-agnostic prompts** that adapt to each platform's capabilities
-- **Consistent context** regardless of underlying platform
-- **Session integration** for data persistence
-
-### Session
-
-Session management with configurable boundaries and storage backends.
-
-```ruby
-# Session boundaries control session ID generation
-config.use_session_config(
-  boundaries: [:flow, :platform, :gateway, :url],  # Session isolation
-  identifier: :msisdn,                              # Session key type
-  hash_identifiers: true                            # Privacy protection
-)
-
-# Session stores control where data is stored
-config.use_session_store FlowChat::Session::CacheSessionStore     # Rails sessions
-config.use_session_store FlowChat::Session::CacheSessionStore    # Rails cache
-config.use_session_store YourCompany::Session::DatabaseStore     # Custom store
-```
-
-### Middleware
-
-Extensible processing pipeline for custom logic.
-
-```ruby
-class CustomMiddleware
-  def initialize(app)
-    @app = app
-  end
-  
-  def call(context)
-    # Before processing
-    log_request(context)
-    
-    # Process through stack
-    result = @app.call(context)
-    
-    # After processing  
-    log_response(context, result)
-    
-    result
-  end
-end
-```
-
-## Request Processing Flow
-
-Here's how a request flows through FlowChat:
-
-```mermaid
-graph TD
-    A[Platform Request] --> B[Gateway]
-    B --> C[Parse Request]
-    C --> D[Set Context]
-    D --> E[Session Middleware]
-    E --> F[Custom Middleware]
-    F --> G[Executor]
-    G --> H[Flow/Action]
-    H --> I[App.screen]
-    I --> J[Prompt Processing]
-    J --> K[Response Generation]
-    K --> L[Gateway Response]
-    L --> M[Platform Response]
-```
-
-### Detailed Flow
-
-1. **Request Arrives** at platform (USSD, WhatsApp, HTTP)
-2. **Gateway** parses platform-specific request format
-3. **Context Setup** with normalized request data
-4. **Session Middleware** manages session state and boundaries
-5. **Custom Middleware** processes business logic
-6. **Executor** instantiates and calls flow
-7. **Flow Method** uses `app.screen()` for conversation logic
-8. **Prompt Processing** handles user input and validation
-9. **Response Generation** creates platform-agnostic response
-10. **Gateway Rendering** converts to platform-specific format
-11. **Platform Response** sent back to user
-
-## Middleware Stack
-
-The middleware stack is built dynamically based on gateway capabilities:
-
-```ruby
-def create_middleware_stack
-  middleware_stack = ::Middleware::Builder.new(name: @gateway_class.name) do |b|
-    # Gateway always comes first
-    b.use @gateway_class, *@gateway_args
-    
-    # Session middleware sets up session boundaries
-    b.use FlowChat::Session::Middleware, @session_options
-
-    # Platform-specific middleware (if gateway supports it)
-    if @gateway_class.respond_to?(:configure_middleware_stack)
-      @gateway_class.configure_middleware_stack(b, custom_middleware)
-    else
-      b.use custom_middleware
-    end
-
-    # Executor always goes last
-    b.use FlowChat::Executor
-  end
-  
-  # Conditionally inject logger based on configuration
-  middleware_stack.inject_logger(FlowChat.logger) if FlowChat::Config.inject_middleware_logger
-  
-  middleware_stack
-end
+flow = flow_class.new(app)
+flow.send(action)
+raise FlowChat::Interrupt::Terminate, "Unexpected end of flow."
+rescue FlowChat::Interrupt::RestartFlow
+  retry
+rescue FlowChat::Interrupt::Prompt => e
+  [:prompt, e.prompt, e.choices, e.media]
+rescue FlowChat::Interrupt::Terminate => e
+  context.session.destroy
+  [:terminal, e.prompt, nil, e.media]
 ```
 
-### Platform-Specific Middleware
+The three interrupts live in `FlowChat::Interrupt`:
 
-Gateways can configure their own middleware stack:
+| Interrupt | Raised by | Effect |
+|---|---|---|
+| `Prompt` | `prompt.ask` / `select` / `yes?` when no input is available | Unwinds to the Executor, which returns the prompt for the gateway to render. Carries the message, choices, and media. |
+| `Terminate` | `app.say` and `prompt.say` | Ends the flow, destroys the session, and returns a terminal message. |
+| `RestartFlow` | `app.go_back` | Caught with `retry`, which re-runs the flow from the top against a fresh `App`. |
 
-```ruby
-class FlowChat::Ussd::Gateway::Nalo
-  def self.configure_middleware_stack(builder, custom_middleware)
-    builder.use FlowChat::Ussd::Middleware::Pagination
-    builder.use custom_middleware
-    builder.use FlowChat::Ussd::Middleware::ChoiceMapper
-  end
-end
-
-class FlowChat::Whatsapp::Gateway::CloudApi
-  def self.configure_middleware_stack(builder, custom_middleware)
-    builder.use FlowChat::Whatsapp::Middleware::MediaProcessor
-    builder.use custom_middleware
-    builder.use FlowChat::Whatsapp::Middleware::TemplateRenderer
-  end
-end
-```
-
-## Session Architecture
-
-### Session ID Generation
-
-Session IDs are generated based on configurable boundaries:
-
-```ruby
-# Example session IDs based on boundaries
-boundaries: [:flow]                    # => "survey_flow:user123"
-boundaries: [:flow, :platform]        # => "survey_flow:ussd:user123"  
-boundaries: [:flow, :platform, :gateway] # => "survey_flow:ussd:nalo:user123"
-boundaries: [:flow, :url]              # => "survey_flow:tenant1.app.com:user123"
-```
-
-### Session Stores
-
-Different storage backends for different use cases:
-
-```ruby
-# Rails sessions (shorter-lived, good for testing)
-FlowChat::Session::CacheSessionStore
-
-# Rails cache (longer-lived, good for production)
-FlowChat::Session::CacheSessionStore  
-
-# Custom database store (permanent storage)
-class CustomDatabaseStore
-  def initialize(context)
-    @context = context
-  end
-  
-  def get(key)
-    SessionData.find_by(session_id: @context["session.id"], key: key)&.value
-  end
-  
-  def set(key, value)
-    SessionData.upsert({
-      session_id: @context["session.id"],
-      key: key,
-      value: value
-    })
-  end
-end
-```
-
-## Extensibility Points
-
-FlowChat provides multiple extension points:
-
-### 1. Custom Gateways
-
-Implement any platform by creating a gateway:
-
-```ruby
-class YourCompany::Telegram::Gateway::BotAPI
-  def initialize(app, bot_token)
-    @app = app
-    @bot_token = bot_token
-  end
-
-  def call(context)
-    # Parse Telegram webhook
-    update = JSON.parse(context.controller.request.body.read)
-    message = update.dig("message")
-    
-    context["request.user_id"] = message["from"]["id"]
-    context["request.platform"] = :telegram
-    context.input = message["text"]
-    
-    # Process through stack
-    type, prompt, choices, media = @app.call(context)
-    
-    # Send response via Telegram API
-    send_telegram_message(prompt, to: context["request.user_id"])
-  end
-end
-```
-
-### 2. Custom Middleware
-
-Add processing logic at any point:
-
-```ruby
-class AuthenticationMiddleware
-  def call(context)
-    user = authenticate_user(context["request.user_id"])
-    context["current_user"] = user
-    @app.call(context)
-  end
-end
-
-class RateLimitingMiddleware
-  def call(context)
-    check_rate_limit(context["request.user_id"])
-    @app.call(context)
-  end
-end
-```
-
-### 3. Custom Session Stores
-
-Implement any storage backend:
-
-```ruby
-class RedisSessionStore
-  def get(key)
-    Redis.current.hget(session_key, key)
-  end
-  
-  def set(key, value)
-    Redis.current.hset(session_key, key, value)
-    Redis.current.expire(session_key, 24.hours.to_i)
-  end
-end
-```
-
-### 4. Custom Renderers
-
-Platform-specific rendering logic:
-
-```ruby
-class MyPlatform::Renderer
-  def initialize(prompt, choices: nil, media: nil)
-    @prompt = prompt
-    @choices = choices
-    @media = media
-  end
-  
-  def render
-    {
-      text: transform_text(@prompt),
-      buttons: transform_choices(@choices),
-      attachments: transform_media(@media)
-    }
-  end
-end
-```
-
-## Configuration Architecture
-
-FlowChat uses a flexible configuration system:
-
-```ruby
-# Global configuration
-FlowChat::Config.logger = Rails.logger
-FlowChat::Config.cache = Rails.cache
-FlowChat::Config.inject_middleware_logger = true  # Default: true in Rails development
-
-# Platform-specific configuration
-FlowChat::Config.ussd.pagination_page_size = 160
-FlowChat::Config.whatsapp.message_handling_mode = :background
-
-# Per-processor configuration
-processor = FlowChat::Processor.new(self) do |config|
-  config.use_gateway GatewayClass, gateway_config
-  config.use_session_config(boundaries: [:flow], identifier: :user_id)
-  config.use_middleware CustomMiddleware
-end
-```
-
-## Error Handling & Instrumentation
-
-FlowChat includes comprehensive error handling and instrumentation:
-
-```ruby
-# Instrumentation events
-FlowChat.instrument(Events::MESSAGE_RECEIVED, {
-  from: user_id,
-  message: input,
-  platform: :ussd
-})
-
-FlowChat.instrument(Events::FLOW_EXECUTION_START, {
-  flow_name: "survey_flow",
-  action: "start"
-})
-
-# Error handling in flows
-def payment_flow
-  begin
-    process_payment(amount)
-  rescue PaymentError => e
-    app.say "Payment failed: #{e.message}"
-    FlowChat.instrument(Events::PAYMENT_FAILED, {
-      error: e.message,
-      user_id: app.user_id
-    })
-  end
-end
-```
+If the action method returns normally without prompting or terminating, the Executor raises `Terminate` with "Unexpected end of flow", since a flow is expected to either prompt for more input or end with a message.
 
-## Performance Considerations
+These interrupts subclass `Exception`, not `StandardError`. This is deliberate: a `rescue` in your flow code (a bare `rescue => e` catches `StandardError`) will not swallow a prompt or terminate and break the engine. Avoid `rescue Exception` inside a flow, since that would catch them.
 
-### Memory Efficiency
+## The replay model
 
-- **Stateless design**: No application state stored in memory
-- **Session isolation**: Users don't affect each other
-- **Lazy loading**: Components loaded only when needed
+There is no saved program counter between requests. Each webhook rebuilds the `App` and re-runs the flow method from the first line. Progress is reconstructed from the session:
 
-### Scalability
+- `app.screen(key)` checks the session for `key`. If a value is stored, it returns immediately without yielding the block. This fast-forwards through every screen already answered.
+- The first screen without a stored answer yields a `Prompt`. If the current turn has input for it, the prompt validates and transforms it, stores the result with `session.set(key, value)`, and returns, so execution falls through to the next screen. If not, the prompt raises `Interrupt::Prompt` and the turn ends there.
 
-- **Horizontal scaling**: Each request is independent
-- **Background processing**: Long-running tasks handled asynchronously
-- **Caching**: Session data cached for performance
+Two rules follow. One inbound message is consumed by one screen per turn: once a screen takes the input, later screens in the same run see no input and prompt. And a given screen key may be presented only once per run; re-entering a key raises `ArgumentError`. To revisit a screen, use `app.go_back`, which clears the current screen's answer and raises `RestartFlow`.
 
-### Network Efficiency
+## The App
 
-- **Platform optimization**: Each gateway optimizes for its platform
-- **Compression**: Large responses automatically paginated (USSD)
-- **Batching**: Multiple messages combined when possible (WhatsApp)
+`FlowChat::App` is the single object your flow talks to. It wraps the context and exposes:
 
-## Security
+- `screen(key)`, the unit of interaction.
+- `say(msg, media:)`, to end the flow.
+- `go_back`, to return to the previous screen.
+- Read accessors for the turn: `text`, `media`, `location`, `contact`, `attachment_type`, and identity values `msisdn`, `user_id`, `platform`, `gateway`, `message_id`, `timestamp`, `contact_name`.
 
-### Session Security
+`Flow` itself is a thin base class: it stores the `app` and nothing else. All conversation logic lives in the methods you write.
 
-- **Identifier hashing**: Phone numbers hashed by default
-- **Session boundaries**: Prevent cross-tenant data access
-- **Secure storage**: Session stores can encrypt data
+## Sessions
 
-### Input Validation
+The session is a key-value store keyed by the session id. `FlowChat::Session::CacheSessionStore` persists it in `FlowChat::Config.cache`, so a session outlives the request. `screen` reads and writes it, `Terminate` destroys it. Boundaries and identifiers, which decide what shares a session, are covered in [configuration.md](configuration.md#sessions).
 
-- **Automatic sanitization**: User input sanitized by default
-- **Validation pipelines**: Custom validation in prompts
-- **Type safety**: Input transformation prevents type errors
+## Async
 
-### Platform Security
+Gateways with an outbound API can run the flow in a background job instead of inline. The gateway detects async support, serializes the request, and enqueues a job that reconstructs a controller and re-runs the same stack in the background. USSD cannot do this, since its protocol needs a synchronous response. See [async-background-processing.md](async-background-processing.md) and [factory-pattern.md](factory-pattern.md).
 
-- **Signature validation**: Webhook signatures verified
-- **Rate limiting**: Built-in protection against abuse
-- **Access controls**: Gateway-level authentication
+## Instrumentation
 
-This architecture enables FlowChat to be both powerful and flexible, supporting current platforms while being easily extensible for future platforms and use cases. 
+Each stage emits `ActiveSupport::Notifications` events (flow execution, messages received and sent, session lifecycle, pagination, webhook verification). Subscribe to feed metrics and logs into your own backend. See [instrumentation.md](instrumentation.md).