conduit-sse 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 98cf3556d809dacc27ff9aebe5179bb0203539d0357f4d6ca52352833f76450d
+  data.tar.gz: dbeb4685dcef94355a0d71ad0e89cece01fbb69bcde53400e2d7755e2e252669
+SHA512:
+  metadata.gz: ffdcffe64f0ddd71565010439debaea23bc5bbb542d1318ef46a9846d4c5e54e3a3f43267de344851a6d6e89517d71dd199d8eeb2b2b79c2ececdd6b5f49ebbb
+  data.tar.gz: 679f209458cc1d9bb63e758dbb0decd1bdbba578edb538e758d0716e328e23e1e66ee6c1d0143c4b71aecf65aed9244a307ea6d923005aa87e07879f7a5c2334

data/CHANGELOG.md

@@ -0,0 +1,19 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - 2026-05-08
+
+### Added
+- Initial release of Conduit, a lightweight, zero-dependency Ruby gem for parsing Server-Sent Events (SSE) streams
+- Flexible callback-based architecture for processing real-time server push data
+- Custom parser support for transforming event data into any shape
+- SSE spec compliant parsing following the HTML Server-Sent Events specification
+- Built-in inspector for development and troubleshooting
+- Robust error handling to prevent stream interruption
+- Granular access callbacks (`on_frame`, `on_field`) for non-standard SSE implementations
+- Support for streaming AI responses, real-time analytics, and live updates
+- Comprehensive test coverage with fuzz testing

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 franbach
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,600 @@
+# Conduit
+
+Conduit is a lightweight, zero-dependency Ruby gem for parsing Server-Sent Events (SSE) streams. It provides a flexible callback-based architecture for processing real-time server push data with full control over every stage of the parsing pipeline.
+
+## Why Conduit?
+
+Building real-time applications with SSE shouldn't require wrestling with complex parsing logic or sacrificing performance for convenience. Conduit gives you:
+
+**🎯 Zero Dependencies** - Drop it into any Ruby project without worrying about dependency hell. Pure Ruby, no external gems.
+
+**🔧 Complete Control** - Hook into every stage of the parsing pipeline with callbacks. Whether you need to transform data, forward to services, emit to frontends, or add observability. Conduit adapts to your architecture.
+
+**📡 Production Ready** - Built for real-world use with robust error handling, SSE spec compliance, and a built-in inspector for debugging. Streams from AI providers, or any SSE endpoint just work.
+
+**⚡ Flexible Parsers** - Your parser lambda can do anything: JSON parsing, YAML loading, custom transformations, or domain-specific logic. You're not locked into any data shape.
+
+**🔍 Granular Access** - Need to handle non-standard SSE fields? Want raw frame access? Conduit provides both spec-compliant callbacks (`on_event`, `on_parsed`) and low-level access (`on_frame`, `on_field`) for maximum flexibility.
+
+Perfect for streaming AI responses, real-time analytics, live updates, and any application that needs to process server-push events efficiently.
+
+## Features
+
+- **Zero dependencies** - Pure Ruby, no external gems required
+- **Flexible callback system** - Hook into every stage of the parsing pipeline
+- **Custom parsers** - Transform event data into any shape your application needs
+- **SSE spec compliant** - Follows the HTML Server-Sent Events specification
+- **Debugging support** - Built-in inspector for development and troubleshooting
+- **Error handling** - Robust error routing to prevent stream interruption
+
+## Installation
+
+Install the gem and add to your application's Gemfile:
+
+```bash
+bundle add conduit-sse
+```
+
+If bundler is not being used, install the gem directly:
+
+```bash
+gem install conduit-sse
+```
+
+## Usage
+
+### Basic Example
+
+At its core, Conduit processes SSE data chunks and emits callbacks at each stage:
+
+```ruby
+require "conduit"
+
+# Create a stream with a parser that transforms event data
+stream = Conduit.new(parser: ->(data) { JSON.parse(data) })
+
+# Subscribe to parsed events
+stream.on_parsed do |parsed|
+  puts "Received: #{parsed}"
+end
+
+# Feed data chunks (typically from an HTTP stream)
+stream << "data: {\"message\": \"hello\"}\n\n"
+```
+
+### Real-World Example with Net::HTTP
+
+Here's a complete example connecting to an SSE endpoint:
+
+```ruby
+require "conduit"
+require "net/http"
+require "uri"
+require "json"
+
+stream = Conduit.new(parser: ->(d) { JSON.parse(d) rescue d })
+
+stream.on_parsed do |parsed|
+  next unless parsed.is_a?(Hash)
+  puts "#{parsed['wiki']}: #{parsed['title']} by #{parsed['user']}"
+end
+
+uri = URI("https://stream.wikimedia.org/v2/stream/recentchange")
+
+Net::HTTP.start(uri.host, uri.port, use_ssl: true) do |http|
+  http.read_timeout = nil # disable read timeout for SSE
+
+  http.request(Net::HTTP::Get.new(uri, "Accept" => "text/event-stream")) do |response|
+    response.read_body { |chunk| stream << chunk }
+  end
+end
+```
+
+### OpenAI Streaming Example
+
+Here's a complete example using Conduit to stream responses from OpenAI's Chat Completions API:
+
+```ruby
+require "conduit"
+require "net/http"
+require "uri"
+require "json"
+
+# Set your OpenAI API key
+api_key = "your-api-key-here"
+
+# Create the stream with a parser that extracts the delta content
+stream = Conduit.new(parser: ->(data) { JSON.parse(data) })
+
+result = +""
+
+# Approach 1: Use on_parsed to extract delta after JSON parsing
+# Since OpenAI sends structured JSON in the data field, the parser converts it to a Hash,
+# making it easy to extract the delta content directly.
+stream.on_parsed do |parsed_data|
+  type = parsed_data["type"]
+
+  if type == "response.output_text.delta"
+    delta = parsed_data["delta"]
+    if delta
+      puts "parsed delta: #{delta}"
+      result += delta
+
+      # You can also emit the delta to a frontend app here if you will.
+      # emit_to_frontend(delta)
+    end
+  end
+
+  if type == "response.completed"
+    puts "\n\nResult: #{result}"
+    stream.close
+  end
+end
+
+# Approach 2: Use on_field for more granular control
+# This approach gives you access to the raw field values before JSON parsing,
+# useful if you need to inspect or modify the raw data field content.
+stream.on_field do |name, value|
+  if name == "data"
+    data = JSON.parse(value)
+    type = data["type"]
+
+    if type == "response.output_text.delta"
+      delta = data["delta"]
+      if delta
+        puts "delta: #{delta}"
+        result += delta
+
+        # You can also emit the delta to a frontend app here if you will.
+        # emit_to_frontend(delta)
+      end
+    end
+
+    if type == "response.completed"
+      puts "\n\nResult: #{result}"
+      stream.close
+    end
+  end
+end
+
+# Make the streaming request
+uri = URI("https://api.openai.com/v1/responses")
+http = Net::HTTP.new(uri.host, uri.port)
+http.use_ssl = true
+
+request = Net::HTTP::Post.new(uri)
+request["Content-Type"] = "application/json"
+request["Authorization"] = "Bearer #{api_key}"
+
+request.body = JSON.generate({
+  model: "gpt-4.1-mini",
+  stream: true, # Enable streaming
+  input: [
+    { role: "user", content: "Write a haiku about programming" }
+  ]
+})
+
+http.request(request) do |response|
+  response.read_body do |chunk|
+    stream << chunk
+  end
+end
+```
+
+**Note:** OpenAI's Responses API uses `data:` fields with JSON payloads. The response format includes a `type` field to identify event types (`response.output_text.delta` for streaming text chunks, `response.completed` when the stream finishes). The `parser` extracts the delta content from each chunk as it arrives, allowing you to display the response in real-time.
+
+### Callback System
+
+Conduit provides callbacks at every stage of processing:
+
+```ruby
+stream = Conduit.new(parser: ->(data) { data })
+
+# Raw chunk as it arrived (after normalization)
+stream.on_chunk do |chunk|
+  puts "Chunk received: #{chunk.bytesize} bytes"
+end
+
+# Complete frame text (after sanitization)
+stream.on_frame do |frame|
+  puts "Frame: #{frame}"
+end
+
+# Individual SSE field lines
+stream.on_field do |name, value|
+  puts "Field: #{name}=#{value}"
+end
+
+# Fully parsed SSE event
+stream.on_event do |event|
+  puts "Event type: #{event.event}, id: #{event.id}"
+end
+
+# Result of your parser
+stream.on_parsed do |parsed|
+  puts "Parsed: #{parsed}"
+end
+
+# Ping/comment frames
+stream.on_ping do |frame|
+  puts "Ping received"
+end
+
+# Errors from callbacks or parser
+stream.on_error do |error|
+  puts "Error: #{error.message}"
+end
+```
+
+### Understanding Callback Differences
+
+It's important to understand the distinction between `on_frame`, `on_event`, and `on_parsed`/`each`:
+
+**`on_frame`** - Receives the raw frame text (string) after sanitization, regardless of whether it produces an event:
+
+```ruby
+stream.on_frame do |frame|
+  # frame is a string like "data: hello\n"
+  puts frame
+end
+```
+
+**`on_event`** - Receives a fully parsed `Conduit::Event` object with SSE metadata:
+
+```ruby
+stream.on_event do |event|
+  # event is a Conduit::Event object
+  puts event.event  # Event type (e.g., "message")
+  puts event.data   # Data field content (joined data lines)
+  puts event.id     # Last event ID (if sent by server)
+  puts event.retry  # Retry delay in ms (if sent by server)
+end
+```
+
+**`each` / `on_parsed`** - Receives the result of your custom parser (the `parser:` lambda):
+
+```ruby
+stream = Conduit.new(parser: ->(data) { JSON.parse(data) })
+
+stream.each do |parsed|
+  # parsed is whatever your parser returns
+  # In this case, a Hash from JSON.parse(data)
+  puts parsed
+end
+```
+
+**The processing flow:**
+
+1. Raw chunk arrives → `on_chunk` (string)
+2. Chunks are buffered and split into frames
+3. Frame is sanitized → `on_frame` (string)
+4. Frame is parsed into SSE fields → `on_field` (name, value pairs)
+5. Event object is constructed → `on_event` (Conduit::Event)
+6. Your parser transforms the data → `on_parsed`/`each` (your custom output)
+
+**Key nuance:** The parser receives **only the data field content** (joined by newlines), not the entire frame. If you need access to other fields (event type, id, retry), use `on_event` instead.
+
+### Callback Philosophy
+
+Conduit's callback system is designed around two complementary approaches:
+
+**SSE-Spec Callbacks** (`on_event`, `on_parsed`)
+
+- These callbacks are tied to the SSE specification
+- `on_event` receives a structured `Conduit::Event` object with standard SSE fields (event type, data, id, retry)
+- `on_parsed` receives the output of your custom parser, which operates on the data field content
+- Use these when working with spec-compliant SSE streams or when you want structured, predictable data
+
+**Granular Control Callbacks** (`on_frame`, `on_field`)
+
+- These provide low-level access to the raw stream data, independent of SSE specification
+- `on_frame` gives you the complete frame text before field parsing
+- `on_field` gives you individual field lines as they're parsed, including custom/non-standard fields
+- Use these when dealing with non-standard SSE implementations, custom field names, or when you need complete control over the parsing process
+
+**Choosing between approaches:**
+
+- If the SSE stream follows the specification, `on_event` with `Conduit::Event` provides a structured, spec-compliant representation of the event
+- If the frame deviates from the SSE specification or uses custom/non-standard fields, `on_frame` gives you raw access to the frame content, allowing you to handle it independently of the specification
+- Use `on_field` to inspect individual fields when you need to handle custom or non-standard field names
+- Your `parser` lambda can implement any logic needed: JSON parsing, YAML loading, custom transformations, validation, or domain-specific processing
+
+### Common Use Cases
+
+Conduit's callback system makes it easy to integrate SSE streams into your application architecture:
+
+**Forwarding to Services**
+
+```ruby
+stream.on_parsed do |parsed|
+  # Forward parsed events to a message queue, database, or external service
+  MessageQueue.publish("events", parsed)
+end
+```
+
+**Emitting to Frontend Applications**
+
+```ruby
+stream.on_parsed do |parsed|
+  # Stream real-time updates to connected WebSocket clients
+  WebSocketBroadcaster.broadcast("updates", parsed)
+end
+```
+
+**Adding Observability**
+
+```ruby
+stream.on_event do |event|
+  # Track metrics for monitoring
+  Metrics.increment("sse.events.received", tags: { type: event.event })
+end
+
+stream.on_error do |error|
+  # Log errors for debugging
+  Logger.error("SSE processing error", error: error.message)
+end
+```
+
+**Data Transformation**
+
+```ruby
+stream = Conduit.new(parser: ->(data) {
+  # Transform raw data into your domain models
+  raw = JSON.parse(data)
+  MyDomainModel.new(raw)
+})
+
+stream.on_parsed do |model|
+  # Work with your domain objects directly
+  model.process!
+end
+```
+
+**Multi-Consumer Pattern**
+
+```ruby
+# Multiple callbacks can handle the same event
+stream.on_parsed do |parsed|
+  # Consumer 1: Update cache
+  Cache.set(parsed["id"], parsed)
+end
+
+stream.on_parsed do |parsed|
+  # Consumer 2: Trigger webhook
+  WebhookService.trigger(parsed)
+end
+
+stream.on_parsed do |parsed|
+  # Consumer 3: Update analytics
+  Analytics.track("event_received", parsed)
+end
+```
+
+### Event Object
+
+Parsed events are returned as `Conduit::Event` objects with the following attributes:
+
+- `event` - Event type (defaults to "message")
+- `data` - The event data string
+- `id` - Last event ID (from SSE spec)
+- `retry` - Retry delay in milliseconds (from SSE spec)
+
+```ruby
+stream.on_event do |event|
+  puts "Type: #{event.event}"
+  puts "Data: #{event.data}"
+  puts "ID: #{event.id}"
+  puts "Retry: #{event.retry}ms" if event.retry
+end
+```
+
+### Customization Options
+
+You can customize the parsing behavior with these options:
+
+```ruby
+stream = Conduit.new(
+  # Required: A callable that receives the joined data field content (string)
+  # and returns whatever shape your application needs (e.g., JSON.parse, YAML.load, etc.)
+  parser: ->(data) { JSON.parse(data) },
+
+  # Optional: Transforms incoming chunks before processing.
+  # The default normalizer performs UTF-8 conversion and CRLF→LF normalization.
+  # NOTE: Providing your own completely replaces the default behavior,
+  # including UTF-8 conversion. If you need UTF-8 handling, you must implement it yourself.
+  chunk_normalizer: ->(chunk) { chunk.upcase },
+
+  # Optional: Delimiter that separates frames in the stream (default: "\n\n")
+  frame_separator: "\r\n\r\n",
+
+  # Optional: Prefix used to identify the data field.
+  # The trailing ":" is stripped to derive the field name (default: "data:")
+  payload_start: "data:",
+
+  # Optional: Pattern identifying ping/comment frames (default: ":")
+  ping_pattern: ":",
+
+  # Optional: Cleans or validates frame content after splitting.
+  # The default sanitizer strips whitespace and performs UTF-8 conversion.
+  # NOTE: Providing your own completely replaces the default behavior,
+  # including UTF-8 handling. If you need UTF-8 handling, you must implement it yourself.
+  sanitize_pattern: ->(frame) { frame.strip }
+)
+```
+
+### Using `each` for Enumerable Interface
+
+For a simpler interface, use `each` to iterate over parsed events:
+
+```ruby
+stream = Conduit.new(parser: ->(data) { data })
+
+stream.each do |parsed|
+  puts "Received: #{parsed}"
+end
+
+# Feed data
+stream << "data: hello\n\n"
+stream << "data: world\n\n"
+```
+
+### Accessing SSE State
+
+Conduit tracks SSE spec state that you can access:
+
+```ruby
+stream = Conduit.new(parser: ->(data) { data })
+
+stream << "id: 123\ndata: hello\n\n"
+
+puts stream.last_event_id  # => "123"
+puts stream.retry_ms       # => nil (unless server sends retry field)
+```
+
+### Handling Stream Completion
+
+Use `finish` (or its alias `close`) to process any remaining data in the buffer when the stream ends:
+
+```ruby
+stream = Conduit.new(parser: ->(data) { JSON.parse(data) })
+
+http.request(request) do |response|
+  response.read_body do |chunk|
+    stream << chunk
+  end
+end
+
+# Process any trailing data not terminated by the frame separator
+stream.finish
+# or
+stream.close
+```
+
+This is useful when the HTTP connection closes cleanly without a trailing `\n\n`, which is common with many SSE servers. The method is safe to call multiple times and on empty buffers.
+
+### Error Handling
+
+Errors in callbacks are routed to the `on_error` handler, preventing stream interruption:
+
+```ruby
+stream = Conduit.new(parser: ->(data) { JSON.parse(data) })
+
+stream.on_error do |error|
+  puts "Caught error: #{error.message}"
+  # Stream continues processing
+end
+
+stream.on_parsed do |parsed|
+  # If this raises, it's caught by on_error
+  process_data(parsed)
+end
+
+stream << "data: invalid json\n\n"  # Parser fails, but stream continues
+```
+
+### Debugging with Inspector
+
+Use the built-in inspector to log all stream activity during development:
+
+```ruby
+require "net/http"
+require "uri"
+require "json"
+
+stream = Conduit.new(parser: ->(data) { JSON.parse(data) })
+
+# Attach inspector to log everything to stdout
+Conduit::Inspector.attach(stream)
+
+# Or log to a different IO
+Conduit::Inspector.attach(stream, io: $stderr)
+
+# You'll see [CHUNK], [FRAME], [FIELD], [EVENT], [PARSED] lines as data flows.
+# Wikimedia tends to emit event:, id:, data: and occasional : ping keep-alives.
+uri = URI("https://stream.wikimedia.org/v2/stream/recentchange")
+
+Net::HTTP.start(uri.host, uri.port, use_ssl: true) do |http|
+  http.read_timeout = nil # disable read timeout for SSE
+
+  http.request(Net::HTTP::Get.new(uri, "Accept" => "text/event-stream")) do |response|
+    response.read_body { |chunk| stream << chunk }
+  end
+end
+
+```
+
+The inspector logs:
+
+- Chunks with byte counts
+- Frames with byte counts
+- Individual fields
+- Pings
+- Events with metadata
+- Parsed results
+- Errors
+
+### Multiple Callbacks
+
+You can register multiple callbacks for the same event type:
+
+```ruby
+stream = Conduit.new(parser: ->(data) { data })
+
+stream.on_parsed do |parsed|
+  puts "Handler 1: #{parsed}"
+end
+
+stream.on_parsed do |parsed|
+  puts "Handler 2: #{parsed}"
+end
+
+stream << "data: hello\n\n"
+# Both handlers execute in registration order
+```
+
+### Custom Field Handling
+
+Conduit emits all SSE fields, including custom ones:
+
+```ruby
+stream = Conduit.new(parser: ->(data) { data })
+
+stream.on_field do |name, value|
+  case name
+  when "data"
+    puts "Data: #{value}"
+  when "custom-field"
+    puts "Custom: #{value}"
+  end
+end
+
+stream << "data: hello\ncustom-field: value\n\n"
+```
+
+## Architecture
+
+Conduit processes data through these stages:
+
+1. **Chunk Normalization** - Raw chunks are normalized (UTF-8 conversion, CRLF→LF)
+2. **Buffering** - Chunks are buffered until frame boundaries are found
+3. **Frame Splitting** - Frames are split by the separator (default: `\n\n`)
+4. **Sanitization** - Frames are sanitized (default: strip whitespace)
+5. **Ping Detection** - Ping/comment frames are identified
+6. **Field Parsing** - SSE fields are parsed per the HTML spec
+7. **Event Construction** - Events are built from parsed fields
+8. **Parser Application** - Your custom parser transforms event data
+9. **Callback Emission** - Callbacks are invoked at each stage
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/conduit.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/Rakefile

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "minitest/test_task"
+
+Minitest::TestTask.create
+
+require "rubocop/rake_task"
+
+RuboCop::RakeTask.new
+
+task default: %i[test rubocop]

data/lib/conduit/callbacks.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+module Conduit
+  # Manages callback registration and execution for Conduit::Stream.
+  class Callbacks
+    FAILED = Object.new.freeze
+
+    def initialize
+      @callbacks = {}
+    end
+
+    def on(name, &block)
+      @callbacks[name] = compose(@callbacks[name], block)
+    end
+
+    def emit(name, *args)
+      callback = @callbacks[name]
+      return if callback.nil?
+
+      callback.call(*args)
+    rescue StandardError => e
+      raise unless name != :error && @callbacks[:error]
+
+      @callbacks[:error].call(e)
+    end
+
+    def call_safely(callable, *args)
+      callable.call(*args)
+    rescue StandardError => e
+      raise unless @callbacks[:error]
+
+      @callbacks[:error].call(e)
+      FAILED
+    end
+
+    private
+
+    def compose(previous, current)
+      return previous if current.nil?
+      return current if previous.nil?
+
+      proc do |*args|
+        previous.call(*args)
+        current.call(*args)
+      end
+    end
+  end
+end

data/lib/conduit/defaults.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+module Conduit
+  # Default configurations for Conduit::Stream
+  module Defaults
+    PING_PATTERN    = ":"
+    FRAME_SEPARATOR = "\n\n"
+    PAYLOAD_START   = "data:"
+
+    module_function
+
+    def to_utf8(string)
+      string.dup
+            .force_encoding("UTF-8")
+            .encode("UTF-8", invalid: :replace, undef: :replace)
+            .gsub("\r\n", "\n")
+    end
+
+    SANITIZE_PATTERN  = ->(frame) { to_utf8(frame).strip }
+    CHUNK_NORMALIZER  = ->(chunk) { to_utf8(chunk) }
+  end
+end

data/lib/conduit/event.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module Conduit
+  Event = Data.define(:event, :data, :id, :retry)
+end

data/lib/conduit/inspector.rb

@@ -0,0 +1,108 @@
+# frozen_string_literal: true
+
+module Conduit
+  # Attach to a Conduit::Stream to log every layer of activity to an IO.
+  # Intended for development/debugging only.
+  #
+  #   stream = Conduit.new(parser: ->(d) { JSON.parse(d) })
+  #   Conduit::Inspector.attach(stream)
+  #
+  # Pass io: to redirect (e.g. a StringIO in tests, a file, $stderr).
+  class Inspector
+    def self.attach(stream, io: $stdout)
+      new(stream, io: io).attach
+    end
+
+    attr_reader :counts
+
+    def initialize(stream, io:)
+      @stream = stream
+      @io     = io
+      @counts = Hash.new(0)
+    end
+
+    def attach
+      log_chunks
+      log_frames
+      log_fields
+      log_pings
+      log_events
+      log_parsed
+      log_errors
+      self
+    end
+
+    # Print a one-line summary of everything seen so far.
+    def summary
+      @io.puts(
+        "[SUMMARY] " \
+        "chunks=#{@counts[:chunk]} " \
+        "frames=#{@counts[:frame]} " \
+        "events=#{@counts[:event]} " \
+        "parsed=#{@counts[:parsed]} " \
+        "pings=#{@counts[:ping]} " \
+        "fields=#{@counts[:field]} " \
+        "errors=#{@counts[:error]} " \
+        "last_event_id=#{@stream.last_event_id.inspect} " \
+        "retry_ms=#{@stream.retry_ms.inspect}"
+      )
+    end
+
+    private
+
+    def log_chunks
+      @stream.on_chunk do |chunk|
+        @counts[:chunk] += 1
+        @io.puts "\n[CHUNK ##{@counts[:chunk]} | #{chunk.bytesize} bytes]"
+        @io.puts chunk
+      end
+    end
+
+    def log_frames
+      @stream.on_frame do |frame|
+        @counts[:frame] += 1
+        @io.puts "\n[FRAME ##{@counts[:frame]} | #{frame.bytesize} bytes]"
+        @io.puts frame
+      end
+    end
+
+    def log_fields
+      @stream.on_field do |name, value|
+        @counts[:field] += 1
+        @io.puts "-->[FIELD] #{name}=#{value.inspect}"
+      end
+    end
+
+    def log_pings
+      @stream.on_ping do |frame|
+        @counts[:ping] += 1
+        @io.puts "\n[PING ##{@counts[:ping]}] #{frame.inspect}"
+      end
+    end
+
+    def log_events
+      @stream.on_event do |event|
+        @counts[:event] += 1
+        @io.puts "\n[EVENT ##{@counts[:event]}] " \
+                 "event=#{event.event.inspect} " \
+                 "id=#{event.id.inspect} " \
+                 "retry=#{event.retry.inspect}"
+        @io.puts "  data: #{event.data.inspect}"
+      end
+    end
+
+    def log_parsed
+      @stream.on_parsed do |result|
+        @counts[:parsed] += 1
+        @io.puts "[PARSED ##{@counts[:parsed]}] #{result.inspect}"
+      end
+    end
+
+    def log_errors
+      @stream.on_error do |error|
+        @counts[:error] += 1
+        @io.puts "\n[ERROR ##{@counts[:error]}] #{error.class}: #{error.message}"
+      end
+    end
+  end
+end

data/lib/conduit/stream.rb

@@ -0,0 +1,298 @@
+# frozen_string_literal: true
+
+require_relative "callbacks"
+require_relative "defaults"
+require_relative "event"
+
+module Conduit
+  # Core streaming parser for Server-Sent Events (SSE).
+  class Stream
+    # Initialize a new Stream with optional customizations.
+    #
+    # @param parser [Proc] Required. Callable that receives the joined data string of an SSE event and returns whatever shape the application wants.
+    # @param chunk_normalizer [Proc] Optional. Transforms incoming chunks before processing.
+    # @param frame_separator [String] Optional. Delimiter that separates frames in the stream.
+    # @param payload_start [String] Optional. Prefix used to identify the data field (the trailing ":" is stripped to derive the field name).
+    # @param ping_pattern [String] Optional. Pattern identifying ping frames.
+    # @param sanitize_pattern [Proc] Optional. Cleans or validates frame content.
+    def initialize(
+      parser:,
+      chunk_normalizer: nil,
+      frame_separator: nil,
+      payload_start: nil,
+      ping_pattern: nil,
+      sanitize_pattern: nil
+    )
+      raise ArgumentError, "parser must be a Proc (respond to #call)" unless parser.respond_to?(:call)
+
+      @parser           = parser
+      @chunk_normalizer = chunk_normalizer || Defaults::CHUNK_NORMALIZER
+      @sanitize_pattern = sanitize_pattern || Defaults::SANITIZE_PATTERN
+      @frame_separator  = frame_separator  || Defaults::FRAME_SEPARATOR
+      @payload_start    = payload_start    || Defaults::PAYLOAD_START
+      @ping_pattern     = ping_pattern     || Defaults::PING_PATTERN
+      @data_field       = @payload_start.chomp(":")
+      @buffer           = +""
+      @callbacks        = Callbacks.new
+      @last_event_id    = nil
+      @retry_ms         = nil
+    end
+
+    # Stream state — last id/retry seen, per SSE spec semantics.
+    attr_reader :last_event_id, :retry_ms
+
+    # Raw chunk as it arrived (after normalization).
+    #
+    # The chunk is a string that has been normalized (UTF-8 encoded, CRLF→LF).
+    # This is called for every chunk fed to the stream via <<, regardless of
+    # whether the chunk contains complete frames or partial data.
+    #
+    # @yield [chunk] The normalized chunk string
+    def on_chunk(&block)
+      @callbacks.on(:chunk, &block)
+    end
+
+    # Complete frame text (after sanitization), regardless of whether it produces an event.
+    #
+    # A frame is the text between frame separators (default: "\n\n").
+    # This callback receives the raw frame string after sanitization (default: strip).
+    # This includes frames that may not produce events (e.g., frames without data fields).
+    # Ping frames are handled separately by on_ping and do not trigger this callback.
+    #
+    # @yield [frame] The sanitized frame string
+    def on_frame(&block)
+      @callbacks.on(:frame, &block)
+    end
+
+    # Fully parsed SSE event as a {Conduit::Event}.
+    #
+    # This callback receives a Conduit::Event object with the following attributes:
+    # - event: Event type (defaults to "message")
+    # - data: Joined data field content (data lines joined by "\n")
+    # - id: Last event ID from the SSE spec
+    # - retry: Retry delay in milliseconds from the SSE spec
+    #
+    # Only called for frames that contain at least one data field.
+    # Use this callback when you need access to SSE metadata (event type, id, retry).
+    #
+    # @yield [event] A Conduit::Event object
+    def on_event(&block)
+      @callbacks.on(:event, &block)
+    end
+
+    # Result of running the configured parser over an event's data.
+    #
+    # The parser receives ONLY the data field content (joined by "\n"), not the entire frame.
+    # If you need access to other SSE fields (event type, id, retry), use on_event instead.
+    #
+    # If the parser raises an error and an on_error handler is registered,
+    # the error is routed to on_error and this callback is NOT invoked for that event.
+    #
+    # @yield [parsed] Whatever your parser lambda returns
+    def on_parsed(&block)
+      @callbacks.on(:parsed, &block)
+    end
+
+    # Ping/comment frame.
+    #
+    # Ping frames are identified by the ping_pattern (default: ":").
+    # These are typically used for keep-alive messages or comments.
+    # Ping frames do NOT trigger on_frame or on_event callbacks.
+    #
+    # @yield [frame] The ping frame string
+    def on_ping(&block)
+      @callbacks.on(:ping, &block)
+    end
+
+    # Every parsed SSE field line. Yields (name, value) for every field, including
+    # the standard ones (data/event/id/retry) and any custom fields a server emits.
+    #
+    # Per the SSE spec, fields are parsed one per line with the format "name: value".
+    # This callback is invoked for each field line as it's parsed from the frame.
+    #
+    # @yield [name, value] The field name and value as strings
+    def on_field(&block)
+      @callbacks.on(:field, &block)
+    end
+
+    # Errors raised by any callback or by the parser.
+    #
+    # When a callback (other than on_error itself) or the parser raises an error,
+    # it's routed to this handler if registered. This prevents errors from interrupting
+    # the stream processing.
+    #
+    # If on_error is not registered, errors will bubble up and interrupt processing.
+    # If on_error itself raises, that error will bubble up.
+    #
+    # @yield [error] The exception that was raised
+    def on_error(&block)
+      @callbacks.on(:error, &block)
+    end
+
+    # Feed a chunk of data to the stream for processing.
+    #
+    # Chunks are typically received from an HTTP stream (e.g., Net::HTTP response body).
+    # The chunk is normalized, buffered, and then processed for complete frames.
+    # Returns self for method chaining.
+    #
+    # @param chunk [String] Raw data chunk from the stream
+    # @return [self]
+    def <<(chunk)
+      chunk = normalize_chunk(chunk)
+
+      @callbacks.emit(:chunk, chunk)
+      @buffer << chunk
+
+      process_frames
+      self
+    end
+
+    # Signal end-of-input. Processes any bytes left in the buffer as a final frame,
+    # so trailing data not terminated by the frame separator still produces an event.
+    #
+    # Call this when the underlying transport closes cleanly without a trailing "\n\n"
+    # (typical for many HTTP SSE servers). Safe to call multiple times; safe to call on
+    # an empty buffer; safe to keep using the stream afterwards.
+    #
+    # @return [self]
+    def finish
+      return self if @buffer.empty?
+
+      remainder = @buffer.slice!(0, @buffer.length)
+      process_frame(remainder)
+      self
+    end
+    alias close finish
+
+    # Enumerable interface for iterating over parsed events.
+    #
+    # Provides a convenient way to iterate over the results of your parser.
+    # Without a block, returns an Enumerator. With a block, registers an on_parsed
+    # callback and returns self for chaining.
+    #
+    # @yield [parsed] The result of your parser
+    # @return [Enumerator, self]
+    def each(&block)
+      return enum_for(:each) unless block
+
+      on_parsed(&block)
+      self
+    end
+
+    private
+
+    # Process buffered chunks to extract complete frames.
+    #
+    # Scans the buffer for the frame separator and extracts complete frames.
+    # Incomplete frames remain in the buffer for the next chunk.
+    # This is called automatically after each chunk is fed via <<.
+    def process_frames
+      loop do
+        idx = @buffer.index(@frame_separator)
+        break unless idx
+
+        frame = @buffer.slice!(0, idx + @frame_separator.length)
+        process_frame(frame)
+      end
+    end
+
+    # Process a single frame through the parsing pipeline.
+    #
+    # Processing stages:
+    # 1. Sanitize the frame (default: strip whitespace)
+    # 2. Check if it's a ping frame (if so, emit on_ping and return)
+    # 3. Emit on_frame with the sanitized frame
+    # 4. Parse SSE fields from the frame
+    # 5. Emit on_field for each field line
+    # 6. Track SSE state (id, retry) from standard fields
+    # 7. If data fields present, build Event object and emit on_event
+    # 8. Apply user parser to the data content
+    # 9. Emit on_parsed with the parser result
+    #
+    # @param frame [String] The raw frame string
+    def process_frame(frame)
+      frame = sanitize(frame)
+      return if frame.empty?
+
+      if ping?(frame)
+        @callbacks.emit(:ping, frame)
+        return
+      end
+
+      @callbacks.emit(:frame, frame)
+
+      type = nil
+      data_lines = []
+
+      parse_fields(frame).each do |name, value|
+        @callbacks.emit(:field, name, value)
+
+        case name
+        when @data_field then data_lines << value
+        when "event"     then type = value
+        when "id"        then @last_event_id = value unless value.include?("\u0000")
+        when "retry"     then @retry_ms = lenient_int(value)
+        end
+      end
+
+      return if data_lines.empty?
+
+      data = data_lines.join("\n")
+      event = Event.new(
+        event: type || "message",
+        data: data,
+        id: @last_event_id,
+        retry: @retry_ms
+      )
+
+      @callbacks.emit(:event, event)
+
+      parsed = @callbacks.call_safely(@parser, data)
+      return if parsed.equal?(Callbacks::FAILED)
+
+      @callbacks.emit(:parsed, parsed)
+    end
+
+    # Per https://html.spec.whatwg.org/multipage/server-sent-events.html, parse one field per line:
+    #   - empty line: skipped
+    #   - line with no ":" : whole line is the field name, value is ""
+    #   - otherwise: field name = before first ":", value = after, with one optional leading space stripped
+    #   - empty field name (line starting with ":") is a comment and ignored
+    def parse_fields(frame)
+      frame.lines.filter_map do |line|
+        line = line.chomp
+        next if line.empty?
+
+        idx = line.index(":")
+        if idx.nil?
+          [line, ""]
+        else
+          name = line[0...idx]
+          next if name.empty?
+
+          value = line[(idx + 1)..] || ""
+          value = value[1..] if value.start_with?(" ")
+          [name, value]
+        end
+      end
+    end
+
+    def lenient_int(value)
+      Integer(value, 10)
+    rescue ArgumentError, TypeError
+      value
+    end
+
+    def normalize_chunk(chunk)
+      @chunk_normalizer.call(chunk)
+    end
+
+    def sanitize(frame)
+      @sanitize_pattern.call(frame)
+    end
+
+    def ping?(frame)
+      frame.start_with?(@ping_pattern)
+    end
+  end
+end

data/lib/conduit/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module Conduit
+  VERSION = "0.1.0"
+end

data/lib/conduit.rb

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require_relative "conduit/version"
+require_relative "conduit/stream"
+require_relative "conduit/inspector"
+
+# Conduit is a lightweight, zero-dependency Ruby gem for parsing Server-Sent Events (SSE) streams.
+module Conduit
+  def self.new(**)
+    Stream.new(**)
+  end
+end

metadata

@@ -0,0 +1,58 @@
+--- !ruby/object:Gem::Specification
+name: conduit-sse
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- franbach
+bindir: exe
+cert_chain: []
+date: 2026-05-09 00:00:00.000000000 Z
+dependencies: []
+description: Conduit provides a flexible callback-based architecture for processing
+  real-time server push data with full control over every stage of the parsing pipeline.
+  Perfect for streaming AI responses, real-time analytics, and live updates.
+email:
+- franciscobach@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE.txt
+- README.md
+- Rakefile
+- lib/conduit.rb
+- lib/conduit/callbacks.rb
+- lib/conduit/defaults.rb
+- lib/conduit/event.rb
+- lib/conduit/inspector.rb
+- lib/conduit/stream.rb
+- lib/conduit/version.rb
+homepage: https://github.com/franbach/conduit
+licenses:
+- MIT
+metadata:
+  allowed_push_host: https://rubygems.org
+  homepage_uri: https://github.com/franbach/conduit
+  source_code_uri: https://github.com/franbach/conduit
+  changelog_uri: https://github.com/franbach/conduit/blob/main/CHANGELOG.md
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 3.2.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.6.2
+specification_version: 4
+summary: A lightweight, zero-dependency Ruby gem for parsing Server-Sent Events (SSE)
+  streams
+test_files: []