io-stream 0.11.1 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- checksums.yaml.gz.sig +0 -0
- data/context/getting-started.md +145 -0
- data/context/high-performance-io.md +225 -0
- data/context/index.yaml +16 -0
- data/lib/io/stream/buffered.rb +4 -25
- data/lib/io/stream/connection_reset_error.rb +3 -0
- data/lib/io/stream/duplex.rb +109 -0
- data/lib/io/stream/generic.rb +2 -1
- data/lib/io/stream/openssl.rb +1 -45
- data/lib/io/stream/readable.rb +5 -2
- data/lib/io/stream/shim/timeout.rb +25 -0
- data/lib/io/stream/version.rb +1 -1
- data/lib/io/stream.rb +10 -1
- data/license.md +1 -1
- data/readme.md +35 -8
- data/releases.md +7 -0
- data.tar.gz.sig +0 -0
- metadata +9 -4
- metadata.gz.sig +0 -0
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 2c6d9477b767c150a9cf33bc11f6889f33a15a86da4814bdd90bc57eba91d47c
|
|
4
|
+
data.tar.gz: fc9981e2657df9a82471e2d80de424c87186d64f356a6a06966d2723d137ae4e
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a20fee7d1e21a51b8604502f7e90b90acfd46131fcde7f33639f9cdeccd570de733a64d0d7290beb1625857c9531242c37f45d673add9bc41444f5d424290dab
|
|
7
|
+
data.tar.gz: 3f78ca242a41714a6b3768b593b44c63669b50a72be93a7e9ad0eab6e2e0eaa5c9a80d914f36894f3cd3cd8e6fc4f845c9f2ca2d14762754016bd071542bae5b
|
checksums.yaml.gz.sig
CHANGED
|
Binary file
|
|
@@ -0,0 +1,145 @@
|
|
|
1
|
+
# Getting Started
|
|
2
|
+
|
|
3
|
+
This guide explains how to use `io-stream` to add efficient buffering to Ruby IO objects.
|
|
4
|
+
|
|
5
|
+
## Overview
|
|
6
|
+
|
|
7
|
+
`io-stream` provides a buffered stream wrapper for any IO-like object in Ruby. It wraps standard Ruby IO instances (files, sockets, pipes) and adds buffering for both reading and writing operations, significantly improving performance for applications that perform many small reads or writes.
|
|
8
|
+
|
|
9
|
+
## Installation
|
|
10
|
+
|
|
11
|
+
Add the gem to your project:
|
|
12
|
+
|
|
13
|
+
~~~ bash
|
|
14
|
+
$ bundle add io-stream
|
|
15
|
+
~~~
|
|
16
|
+
|
|
17
|
+
## Core Concepts
|
|
18
|
+
|
|
19
|
+
### Buffered Streams
|
|
20
|
+
|
|
21
|
+
`io-stream` provides buffering through the {IO::Stream::Buffered} class, which wraps any IO object. Buffering reduces the number of system calls by accumulating data in memory before actually reading from or writing to the underlying IO.
|
|
22
|
+
|
|
23
|
+
### Read and Write Buffers
|
|
24
|
+
|
|
25
|
+
The stream maintains separate buffers for reading and writing:
|
|
26
|
+
|
|
27
|
+
- **Read buffer**: Accumulates data from the underlying IO, allowing multiple small reads without system calls
|
|
28
|
+
- **Write buffer**: Accumulates data to write, flushing to the underlying IO only when the buffer is full or explicitly flushed
|
|
29
|
+
|
|
30
|
+
## Usage
|
|
31
|
+
|
|
32
|
+
### Wrapping an IO Object
|
|
33
|
+
|
|
34
|
+
You can wrap any IO-like object using {IO::Stream}:
|
|
35
|
+
|
|
36
|
+
~~~ ruby
|
|
37
|
+
require 'io/stream'
|
|
38
|
+
|
|
39
|
+
# Wrap a file
|
|
40
|
+
file = File.open("data.txt", "w+")
|
|
41
|
+
stream = IO::Stream(file)
|
|
42
|
+
|
|
43
|
+
# Wrap a socket
|
|
44
|
+
require 'socket'
|
|
45
|
+
socket = TCPSocket.new("example.com", 80)
|
|
46
|
+
stream = IO::Stream(socket)
|
|
47
|
+
~~~
|
|
48
|
+
|
|
49
|
+
### Opening Files Directly
|
|
50
|
+
|
|
51
|
+
You can also open files directly as buffered streams:
|
|
52
|
+
|
|
53
|
+
~~~ ruby
|
|
54
|
+
require 'io/stream'
|
|
55
|
+
|
|
56
|
+
# Open a file for reading
|
|
57
|
+
stream = IO::Stream::Buffered.open("data.txt", "r")
|
|
58
|
+
data = stream.read
|
|
59
|
+
stream.close
|
|
60
|
+
|
|
61
|
+
# Open with a block (auto-closes)
|
|
62
|
+
IO::Stream::Buffered.open("data.txt", "w") do |stream|
|
|
63
|
+
stream.write("Hello, World!")
|
|
64
|
+
stream.flush
|
|
65
|
+
end
|
|
66
|
+
~~~
|
|
67
|
+
|
|
68
|
+
### Reading Data
|
|
69
|
+
|
|
70
|
+
The {IO::Stream::Readable} module provides various methods for reading:
|
|
71
|
+
|
|
72
|
+
~~~ ruby
|
|
73
|
+
require 'io/stream'
|
|
74
|
+
|
|
75
|
+
IO::Stream::Buffered.open("data.txt", "r") do |stream|
|
|
76
|
+
# Read entire stream
|
|
77
|
+
content = stream.read
|
|
78
|
+
|
|
79
|
+
# Read specific number of bytes
|
|
80
|
+
chunk = stream.read(1024)
|
|
81
|
+
|
|
82
|
+
# Read a line
|
|
83
|
+
line = stream.gets
|
|
84
|
+
|
|
85
|
+
# Read all lines
|
|
86
|
+
lines = stream.readlines
|
|
87
|
+
|
|
88
|
+
# Check for end of stream
|
|
89
|
+
if stream.eof?
|
|
90
|
+
puts "Reached end of file"
|
|
91
|
+
end
|
|
92
|
+
end
|
|
93
|
+
~~~
|
|
94
|
+
|
|
95
|
+
### Writing Data
|
|
96
|
+
|
|
97
|
+
The {IO::Stream::Writable} module provides methods for writing:
|
|
98
|
+
|
|
99
|
+
~~~ ruby
|
|
100
|
+
require 'io/stream'
|
|
101
|
+
|
|
102
|
+
IO::Stream::Buffered.open("output.txt", "w") do |stream|
|
|
103
|
+
# Write data (buffered)
|
|
104
|
+
stream.write("Hello, ")
|
|
105
|
+
stream.write("World!")
|
|
106
|
+
|
|
107
|
+
# Write with automatic newline
|
|
108
|
+
stream.puts("This is a line")
|
|
109
|
+
|
|
110
|
+
# Flush buffer to ensure data is written
|
|
111
|
+
stream.flush
|
|
112
|
+
end
|
|
113
|
+
~~~
|
|
114
|
+
|
|
115
|
+
## Important Behaviors
|
|
116
|
+
|
|
117
|
+
### Automatic Flushing
|
|
118
|
+
|
|
119
|
+
The write buffer automatically flushes when:
|
|
120
|
+
|
|
121
|
+
- The buffer size reaches the minimum write size (default: 64KB).
|
|
122
|
+
- You call {IO::Stream::Writable#puts} (always flushes immediately).
|
|
123
|
+
- You call {IO::Stream::Writable#flush} explicitly.
|
|
124
|
+
- The stream is closed.
|
|
125
|
+
|
|
126
|
+
### Manual Flushing
|
|
127
|
+
|
|
128
|
+
For applications that need precise control over when data is written:
|
|
129
|
+
|
|
130
|
+
~~~ ruby
|
|
131
|
+
stream.write("Important data")
|
|
132
|
+
stream.flush # Ensure data is written immediately
|
|
133
|
+
~~~
|
|
134
|
+
|
|
135
|
+
### Buffer Sizes
|
|
136
|
+
|
|
137
|
+
You can customize buffer sizes when creating streams:
|
|
138
|
+
|
|
139
|
+
~~~ ruby
|
|
140
|
+
# Smaller buffer for interactive applications
|
|
141
|
+
stream = IO::Stream::Buffered.new(io, minimum_write_size: 4096)
|
|
142
|
+
|
|
143
|
+
# Larger buffer for bulk operations
|
|
144
|
+
stream = IO::Stream::Buffered.new(io, minimum_write_size: 256 * 1024)
|
|
145
|
+
~~~
|
|
@@ -0,0 +1,225 @@
|
|
|
1
|
+
# High Performance IO
|
|
2
|
+
|
|
3
|
+
This guide explains how to achieve optimal performance when using `io-stream` by understanding and controlling flush behavior.
|
|
4
|
+
|
|
5
|
+
## Overview
|
|
6
|
+
|
|
7
|
+
The key to high-performance IO with `io-stream` is understanding when and how to flush your write buffer. Improper flush timing can significantly impact throughput, latency, and CPU usage. This guide helps you choose the right buffering strategy for your application.
|
|
8
|
+
|
|
9
|
+
## Why Buffering Matters
|
|
10
|
+
|
|
11
|
+
Every write to an underlying IO object (file, socket, pipe) involves a system call, which has overhead:
|
|
12
|
+
|
|
13
|
+
- **Context switching**: Transferring control between userspace and kernel space.
|
|
14
|
+
- **System call overhead**: The cost of invoking kernel functions.
|
|
15
|
+
- **Network packet overhead**: For sockets, each small write may trigger a separate packet.
|
|
16
|
+
|
|
17
|
+
Buffering solves this by accumulating data in memory and performing larger, less frequent writes. However, buffering introduces latency - data sits in memory until flushed.
|
|
18
|
+
|
|
19
|
+
Use buffering when you need:
|
|
20
|
+
- **High throughput**: Maximize data transfer rate for bulk operations.
|
|
21
|
+
- **Reduced CPU usage**: Minimize system call overhead when writing many small pieces.
|
|
22
|
+
- **Efficient network utilization**: Avoid sending many tiny packets.
|
|
23
|
+
|
|
24
|
+
## The Flush/Throughput Tradeoff
|
|
25
|
+
|
|
26
|
+
There's a fundamental tradeoff between responsiveness and throughput:
|
|
27
|
+
|
|
28
|
+
```mermaid
|
|
29
|
+
graph LR
|
|
30
|
+
A[Immediate Flush] -->|Low Latency| B[Responsive]
|
|
31
|
+
A -->|Many System Calls| C[Lower Throughput]
|
|
32
|
+
D[Delayed Flush] -->|Higher Latency| E[Buffered]
|
|
33
|
+
D -->|Fewer System Calls| F[Higher Throughput]
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
**Immediate flushing** (after every write):
|
|
37
|
+
- ✅ Data is sent immediately - low latency.
|
|
38
|
+
- ✅ Simple mental model - predictable behavior.
|
|
39
|
+
- ❌ High system call overhead.
|
|
40
|
+
- ❌ Lower maximum throughput.
|
|
41
|
+
- ❌ More CPU usage.
|
|
42
|
+
- ❌ Network inefficiency (many small packets).
|
|
43
|
+
|
|
44
|
+
**Buffered flushing** (accumulate before sending):
|
|
45
|
+
- ✅ Fewer system calls - higher throughput.
|
|
46
|
+
- ✅ Better CPU efficiency.
|
|
47
|
+
- ✅ More efficient network packet utilization.
|
|
48
|
+
- ❌ Data is delayed - higher latency.
|
|
49
|
+
- ❌ Requires careful flush management.
|
|
50
|
+
|
|
51
|
+
## Automatic Flush Behavior
|
|
52
|
+
|
|
53
|
+
`io-stream` automatically flushes in these situations:
|
|
54
|
+
|
|
55
|
+
~~~ ruby
|
|
56
|
+
# 1. Buffer reaches minimum_write_size (default: 64KB)
|
|
57
|
+
stream.write("x" * 65536) # Automatically flushes
|
|
58
|
+
|
|
59
|
+
# 2. Using puts() always flushes
|
|
60
|
+
stream.puts("This is flushed immediately")
|
|
61
|
+
|
|
62
|
+
# 3. Closing the stream
|
|
63
|
+
stream.close # Flushes any remaining data
|
|
64
|
+
~~~
|
|
65
|
+
|
|
66
|
+
## Choosing Your Flush Strategy
|
|
67
|
+
|
|
68
|
+
### Strategy 1: Let Automatic Flushing Handle It
|
|
69
|
+
|
|
70
|
+
Best for: Bulk data transfer, file processing, log writing.
|
|
71
|
+
|
|
72
|
+
~~~ ruby
|
|
73
|
+
require 'io/stream'
|
|
74
|
+
|
|
75
|
+
# Default behavior - automatic flush at 64KB
|
|
76
|
+
stream = IO::Stream::Buffered.open("large_file.dat", "w")
|
|
77
|
+
|
|
78
|
+
# Write lots of data
|
|
79
|
+
1000.times do |i|
|
|
80
|
+
stream.write("Record #{i}\n" * 1000)
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
stream.close # Final flush on close
|
|
84
|
+
~~~
|
|
85
|
+
|
|
86
|
+
**When to use:**
|
|
87
|
+
- Writing large amounts of data continuously.
|
|
88
|
+
- Throughput is more important than latency.
|
|
89
|
+
- You don't need interactive feedback.
|
|
90
|
+
|
|
91
|
+
### Strategy 2: Manual Flush at Logical Boundaries
|
|
92
|
+
|
|
93
|
+
Best for: Request/response protocols, transaction processing, structured logging.
|
|
94
|
+
|
|
95
|
+
~~~ ruby
|
|
96
|
+
require 'io/stream'
|
|
97
|
+
require 'socket'
|
|
98
|
+
|
|
99
|
+
socket = TCPSocket.new("example.com", 80)
|
|
100
|
+
stream = IO::Stream(socket)
|
|
101
|
+
|
|
102
|
+
# Build complete HTTP request
|
|
103
|
+
stream.write("GET / HTTP/1.1\r\n")
|
|
104
|
+
stream.write("Host: example.com\r\n")
|
|
105
|
+
stream.write("Connection: close\r\n")
|
|
106
|
+
stream.write("\r\n")
|
|
107
|
+
|
|
108
|
+
# Flush after complete request
|
|
109
|
+
stream.flush # Send request as one operation
|
|
110
|
+
~~~
|
|
111
|
+
|
|
112
|
+
**When to use:**
|
|
113
|
+
- Message-based protocols (HTTP, Redis, etc.)
|
|
114
|
+
- You need to send complete "units" of data
|
|
115
|
+
- Each logical operation should complete atomically
|
|
116
|
+
- Balance between throughput and responsiveness
|
|
117
|
+
|
|
118
|
+
### Strategy 3: Immediate Flush for Interactive Applications
|
|
119
|
+
|
|
120
|
+
Best for: Chat applications, streaming responses, real-time dashboards.
|
|
121
|
+
|
|
122
|
+
~~~ ruby
|
|
123
|
+
require 'io/stream'
|
|
124
|
+
|
|
125
|
+
# Use smaller buffer for more frequent automatic flushes
|
|
126
|
+
stream = IO::Stream::Buffered.new(
|
|
127
|
+
socket,
|
|
128
|
+
minimum_write_size: 512 # Smaller buffer = more responsive
|
|
129
|
+
)
|
|
130
|
+
|
|
131
|
+
# Or flush after every message
|
|
132
|
+
stream.write(message)
|
|
133
|
+
stream.flush # Ensure immediate delivery
|
|
134
|
+
~~~
|
|
135
|
+
|
|
136
|
+
**When to use:**
|
|
137
|
+
- Real-time user interaction required.
|
|
138
|
+
- Low latency is critical.
|
|
139
|
+
- Data arrives in small, discrete chunks.
|
|
140
|
+
|
|
141
|
+
### Strategy 4: Time-Based Flushing
|
|
142
|
+
|
|
143
|
+
Best for: Streaming data, progress updates, monitoring
|
|
144
|
+
|
|
145
|
+
~~~ ruby
|
|
146
|
+
require 'io/stream'
|
|
147
|
+
|
|
148
|
+
stream = IO::Stream::Buffered.open("stream.log", "w")
|
|
149
|
+
last_flush = Time.now
|
|
150
|
+
|
|
151
|
+
loop do
|
|
152
|
+
stream.write(generate_log_entry)
|
|
153
|
+
|
|
154
|
+
# Flush every second or when buffer is large
|
|
155
|
+
if Time.now - last_flush > 1.0
|
|
156
|
+
stream.flush
|
|
157
|
+
last_flush = Time.now
|
|
158
|
+
end
|
|
159
|
+
end
|
|
160
|
+
~~~
|
|
161
|
+
|
|
162
|
+
**When to use:**
|
|
163
|
+
- Ensuring regular progress visibility.
|
|
164
|
+
- Protecting against data loss (periodic flush to disk).
|
|
165
|
+
- Streaming applications with real-time monitoring.
|
|
166
|
+
|
|
167
|
+
### Strategy 5: Readiness based flushing
|
|
168
|
+
|
|
169
|
+
Best for: interactive protocols, terminal applications, chat servers.
|
|
170
|
+
|
|
171
|
+
~~~ ruby
|
|
172
|
+
require 'io/stream'
|
|
173
|
+
|
|
174
|
+
stream = IO::Stream::Buffered.new(socket, minimum_write_size: 1024)
|
|
175
|
+
|
|
176
|
+
loop do
|
|
177
|
+
# Blocking read from a queue of messages to send:
|
|
178
|
+
chunk = queue.pop
|
|
179
|
+
stream.write(chunk)
|
|
180
|
+
|
|
181
|
+
if queue.empty?
|
|
182
|
+
# Flush when we are likely to block on the queue:
|
|
183
|
+
stream.flush
|
|
184
|
+
end
|
|
185
|
+
end
|
|
186
|
+
~~~
|
|
187
|
+
|
|
188
|
+
**When to use:**
|
|
189
|
+
- When you have unpredictable message arrival patterns.
|
|
190
|
+
- When you want to ensure the lowest possible latency while still benefiting from buffering when messages arrive in bursts.
|
|
191
|
+
|
|
192
|
+
## Buffer Size Configuration
|
|
193
|
+
|
|
194
|
+
The `minimum_write_size` parameter controls when automatic flushing occurs:
|
|
195
|
+
|
|
196
|
+
~~~ ruby
|
|
197
|
+
# Very small buffer - more responsive, lower throughput
|
|
198
|
+
stream = IO::Stream::Buffered.new(io, minimum_write_size: 1024)
|
|
199
|
+
|
|
200
|
+
# Default - balanced (64KB)
|
|
201
|
+
stream = IO::Stream::Buffered.new(io)
|
|
202
|
+
|
|
203
|
+
# Large buffer - maximum throughput, higher latency
|
|
204
|
+
stream = IO::Stream::Buffered.new(io, minimum_write_size: 512 * 1024)
|
|
205
|
+
~~~
|
|
206
|
+
|
|
207
|
+
### Choosing Buffer Size
|
|
208
|
+
|
|
209
|
+
**Small buffers (1-8KB):**
|
|
210
|
+
- Interactive protocols (terminal, chat).
|
|
211
|
+
- Real-time data visualization.
|
|
212
|
+
- Acceptable: Lower throughput.
|
|
213
|
+
|
|
214
|
+
**Medium buffers (8-64KB):**
|
|
215
|
+
- Web servers (default is good).
|
|
216
|
+
- Application servers.
|
|
217
|
+
- Database connections.
|
|
218
|
+
- Balance of throughput and responsiveness.
|
|
219
|
+
|
|
220
|
+
**Large buffers (64KB-1MB):**
|
|
221
|
+
- File processing.
|
|
222
|
+
- Bulk data transfer.
|
|
223
|
+
- Video encoding.
|
|
224
|
+
- Logging systems.
|
|
225
|
+
- Only latency-insensitive applications.
|
data/context/index.yaml
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
# Automatically generated context index for Utopia::Project guides.
|
|
2
|
+
# Do not edit then files in this directory directly, instead edit the guides and then run `bake utopia:project:agent:context:update`.
|
|
3
|
+
---
|
|
4
|
+
description: Provides a generic stream wrapper for IO instances.
|
|
5
|
+
metadata:
|
|
6
|
+
documentation_uri: https://socketry.github.io/io-stream/
|
|
7
|
+
source_code_uri: https://github.com/socketry/io-stream.git
|
|
8
|
+
files:
|
|
9
|
+
- path: getting-started.md
|
|
10
|
+
title: Getting Started
|
|
11
|
+
description: This guide explains how to use `io-stream` to add efficient buffering
|
|
12
|
+
to Ruby IO objects.
|
|
13
|
+
- path: high-performance-io.md
|
|
14
|
+
title: High Performance IO
|
|
15
|
+
description: This guide explains how to achieve optimal performance when using `io-stream`
|
|
16
|
+
by understanding and controlling flush behavior.
|
data/lib/io/stream/buffered.rb
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
# Released under the MIT License.
|
|
4
|
-
# Copyright, 2024-
|
|
4
|
+
# Copyright, 2024-2026, by Samuel Williams.
|
|
5
5
|
|
|
6
6
|
require_relative "generic"
|
|
7
7
|
require_relative "connection_reset_error"
|
|
@@ -96,7 +96,7 @@ module IO::Stream
|
|
|
96
96
|
|
|
97
97
|
protected
|
|
98
98
|
|
|
99
|
-
if RUBY_VERSION
|
|
99
|
+
if RUBY_VERSION < "3.3.6"
|
|
100
100
|
def sysclose
|
|
101
101
|
# https://bugs.ruby-lang.org/issues/20723
|
|
102
102
|
Thread.new{@io.close}.join
|
|
@@ -107,29 +107,8 @@ module IO::Stream
|
|
|
107
107
|
end
|
|
108
108
|
end
|
|
109
109
|
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
return @io.write(buffer)
|
|
113
|
-
end
|
|
114
|
-
else
|
|
115
|
-
def syswrite(buffer)
|
|
116
|
-
while true
|
|
117
|
-
result = @io.write_nonblock(buffer, exception: false)
|
|
118
|
-
|
|
119
|
-
case result
|
|
120
|
-
when :wait_readable
|
|
121
|
-
@io.wait_readable(@io.timeout) or raise ::IO::TimeoutError, "read timeout"
|
|
122
|
-
when :wait_writable
|
|
123
|
-
@io.wait_writable(@io.timeout) or raise ::IO::TimeoutError, "write timeout"
|
|
124
|
-
else
|
|
125
|
-
if result == buffer.bytesize
|
|
126
|
-
return
|
|
127
|
-
else
|
|
128
|
-
buffer = buffer.byteslice(result, buffer.bytesize)
|
|
129
|
-
end
|
|
130
|
-
end
|
|
131
|
-
end
|
|
132
|
-
end
|
|
110
|
+
def syswrite(buffer)
|
|
111
|
+
return @io.write(buffer)
|
|
133
112
|
end
|
|
134
113
|
|
|
135
114
|
# Reads data from the underlying stream as efficiently as possible.
|
|
@@ -1,5 +1,8 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
# Released under the MIT License.
|
|
4
|
+
# Copyright, 2025-2026, by Samuel Williams.
|
|
5
|
+
|
|
3
6
|
module IO::Stream
|
|
4
7
|
# Represents a connection reset error in IO streams, usually occurring when the remote side closes the connection unexpectedly.
|
|
5
8
|
class ConnectionResetError < Errno::ECONNRESET
|
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Released under the MIT License.
|
|
4
|
+
# Copyright, 2026, by Samuel Williams.
|
|
5
|
+
|
|
6
|
+
module IO::Stream
|
|
7
|
+
# A low-level duplex IO adapter that composes distinct readable and writable endpoints.
|
|
8
|
+
class Duplex
|
|
9
|
+
# Initialize a duplex transport from separate readable and writable endpoints.
|
|
10
|
+
# @parameter input [IO] The readable endpoint.
|
|
11
|
+
# @parameter output [IO] The writable endpoint.
|
|
12
|
+
def initialize(input, output = input)
|
|
13
|
+
@input = input
|
|
14
|
+
@output = output
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
attr :input
|
|
18
|
+
attr :output
|
|
19
|
+
|
|
20
|
+
# Return the underlying IO used to represent this duplex stream.
|
|
21
|
+
# @returns [IO] The readable endpoint if available, otherwise the writable endpoint.
|
|
22
|
+
def to_io
|
|
23
|
+
@input || @output
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
# Return the maximum timeout across both endpoints.
|
|
27
|
+
# @returns [Numeric | Nil] The effective timeout, or `nil` if no timeout is configured.
|
|
28
|
+
def timeout
|
|
29
|
+
[@input.timeout, @output.timeout].compact.max
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
# Update the timeout on both endpoints.
|
|
33
|
+
# @parameter duration [Numeric | Nil] The timeout to assign.
|
|
34
|
+
def timeout=(duration)
|
|
35
|
+
@input.timeout = duration
|
|
36
|
+
@output.timeout = duration
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
# Check whether both endpoints are closed.
|
|
40
|
+
# @returns [Boolean] True if the duplex stream can no longer read or write.
|
|
41
|
+
def closed?
|
|
42
|
+
@input.closed? && @output.closed?
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
# Close the readable endpoint.
|
|
46
|
+
def close_read
|
|
47
|
+
return if @input.closed?
|
|
48
|
+
|
|
49
|
+
if @input.respond_to?(:close_read)
|
|
50
|
+
@input.close_read
|
|
51
|
+
else
|
|
52
|
+
@input.close
|
|
53
|
+
end
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
# Close the writable endpoint.
|
|
57
|
+
def close_write
|
|
58
|
+
return if @output.closed?
|
|
59
|
+
|
|
60
|
+
if @output.respond_to?(:close_write)
|
|
61
|
+
@output.close_write
|
|
62
|
+
else
|
|
63
|
+
@output.close
|
|
64
|
+
end
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
# Check whether the readable endpoint may still produce data.
|
|
68
|
+
# @returns [Boolean] True if the readable endpoint reports it is readable.
|
|
69
|
+
def readable?
|
|
70
|
+
@input.readable?
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
# Close both endpoints.
|
|
74
|
+
def close
|
|
75
|
+
@output.close unless @output.closed?
|
|
76
|
+
@input.close unless @input.closed?
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
# Write data to the writable endpoint.
|
|
80
|
+
# @parameter buffer [String] The data to write.
|
|
81
|
+
# @returns [Integer] The number of bytes written.
|
|
82
|
+
def write(buffer)
|
|
83
|
+
@output.write(buffer)
|
|
84
|
+
end
|
|
85
|
+
|
|
86
|
+
# Read data from the readable endpoint without blocking.
|
|
87
|
+
# @parameter size [Integer] The maximum number of bytes to read.
|
|
88
|
+
# @parameter buffer [String] The destination buffer.
|
|
89
|
+
# @parameter exception [Boolean] Whether to raise on `:wait_readable` and EOF conditions.
|
|
90
|
+
# @returns [String | Symbol | Nil] Data read from the endpoint, or the underlying non-blocking result.
|
|
91
|
+
def read_nonblock(size, buffer, exception: false)
|
|
92
|
+
@input.read_nonblock(size, buffer, exception: exception)
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
# Wait until the readable endpoint can be read.
|
|
96
|
+
# @parameter duration [Numeric | Nil] The maximum time to wait.
|
|
97
|
+
# @returns [Boolean] True if the endpoint became readable.
|
|
98
|
+
def wait_readable(duration = @timeout)
|
|
99
|
+
@input.wait_readable(duration)
|
|
100
|
+
end
|
|
101
|
+
|
|
102
|
+
# Wait until the writable endpoint can be written.
|
|
103
|
+
# @parameter duration [Numeric | Nil] The maximum time to wait.
|
|
104
|
+
# @returns [Boolean] True if the endpoint became writable.
|
|
105
|
+
def wait_writable(duration = @timeout)
|
|
106
|
+
@output.wait_writable(duration)
|
|
107
|
+
end
|
|
108
|
+
end
|
|
109
|
+
end
|
data/lib/io/stream/generic.rb
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
# Released under the MIT License.
|
|
4
|
-
# Copyright, 2023-
|
|
4
|
+
# Copyright, 2023-2026, by Samuel Williams.
|
|
5
5
|
|
|
6
6
|
require_relative "string_buffer"
|
|
7
7
|
require_relative "readable"
|
|
@@ -9,6 +9,7 @@ require_relative "writable"
|
|
|
9
9
|
|
|
10
10
|
require_relative "shim/buffered"
|
|
11
11
|
require_relative "shim/readable"
|
|
12
|
+
require_relative "shim/timeout"
|
|
12
13
|
|
|
13
14
|
require_relative "openssl"
|
|
14
15
|
|
data/lib/io/stream/openssl.rb
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
# Released under the MIT License.
|
|
4
|
-
# Copyright, 2024-
|
|
4
|
+
# Copyright, 2024-2026, by Samuel Williams.
|
|
5
5
|
|
|
6
6
|
require "openssl"
|
|
7
7
|
|
|
@@ -11,50 +11,6 @@ module OpenSSL
|
|
|
11
11
|
module SSL
|
|
12
12
|
# SSL socket extensions for stream compatibility.
|
|
13
13
|
class SSLSocket
|
|
14
|
-
unless method_defined?(:close_read)
|
|
15
|
-
# Close the read end of the SSL socket.
|
|
16
|
-
def close_read
|
|
17
|
-
# Ignored.
|
|
18
|
-
end
|
|
19
|
-
end
|
|
20
|
-
|
|
21
|
-
unless method_defined?(:close_write)
|
|
22
|
-
# Close the write end of the SSL socket.
|
|
23
|
-
def close_write
|
|
24
|
-
self.stop
|
|
25
|
-
end
|
|
26
|
-
end
|
|
27
|
-
|
|
28
|
-
unless method_defined?(:wait_readable)
|
|
29
|
-
# Wait for the SSL socket to become readable.
|
|
30
|
-
def wait_readable(...)
|
|
31
|
-
to_io.wait_readable(...)
|
|
32
|
-
end
|
|
33
|
-
end
|
|
34
|
-
|
|
35
|
-
unless method_defined?(:wait_writable)
|
|
36
|
-
# Wait for the SSL socket to become writable.
|
|
37
|
-
def wait_writable(...)
|
|
38
|
-
to_io.wait_writable(...)
|
|
39
|
-
end
|
|
40
|
-
end
|
|
41
|
-
|
|
42
|
-
unless method_defined?(:timeout)
|
|
43
|
-
# Get the timeout for SSL socket operations.
|
|
44
|
-
# @returns [Numeric | Nil] The timeout value.
|
|
45
|
-
def timeout
|
|
46
|
-
to_io.timeout
|
|
47
|
-
end
|
|
48
|
-
end
|
|
49
|
-
|
|
50
|
-
unless method_defined?(:timeout=)
|
|
51
|
-
# Set the timeout for SSL socket operations.
|
|
52
|
-
# @parameter value [Numeric | Nil] The timeout value.
|
|
53
|
-
def timeout=(value)
|
|
54
|
-
to_io.timeout = value
|
|
55
|
-
end
|
|
56
|
-
end
|
|
57
|
-
|
|
58
14
|
unless method_defined?(:buffered?)
|
|
59
15
|
# Check if the SSL socket is buffered.
|
|
60
16
|
# @returns [Boolean] True if the SSL socket is buffered.
|
data/lib/io/stream/readable.rb
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
# Released under the MIT License.
|
|
4
|
-
# Copyright, 2025, by Samuel Williams.
|
|
4
|
+
# Copyright, 2025-2026, by Samuel Williams.
|
|
5
5
|
|
|
6
6
|
require_relative "string_buffer"
|
|
7
7
|
|
|
@@ -254,11 +254,14 @@ module IO::Stream
|
|
|
254
254
|
# Don't read less than @minimum_read_size to avoid lots of small reads:
|
|
255
255
|
fill_read_buffer(read_size > @minimum_read_size ? read_size : @minimum_read_size)
|
|
256
256
|
end
|
|
257
|
+
|
|
257
258
|
return @read_buffer[..([size, @read_buffer.size].min - 1)]
|
|
258
259
|
end
|
|
260
|
+
|
|
259
261
|
until (block_given? && yield(@read_buffer)) or @finished
|
|
260
262
|
fill_read_buffer
|
|
261
263
|
end
|
|
264
|
+
|
|
262
265
|
return @read_buffer
|
|
263
266
|
end
|
|
264
267
|
|
|
@@ -366,7 +369,7 @@ module IO::Stream
|
|
|
366
369
|
end
|
|
367
370
|
|
|
368
371
|
# This effectively ties the input and output stream together.
|
|
369
|
-
flush
|
|
372
|
+
self.flush
|
|
370
373
|
|
|
371
374
|
if @read_buffer.empty?
|
|
372
375
|
if sysread(size, @read_buffer)
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Released under the MIT License.
|
|
4
|
+
# Copyright, 2024-2026, by Samuel Williams.
|
|
5
|
+
|
|
6
|
+
require "stringio"
|
|
7
|
+
|
|
8
|
+
class StringIO
|
|
9
|
+
unless method_defined?(:timeout)
|
|
10
|
+
# Return the configured timeout for this in-memory stream.
|
|
11
|
+
# @returns [Numeric | Nil] The configured timeout, if any.
|
|
12
|
+
def timeout
|
|
13
|
+
@timeout
|
|
14
|
+
end
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
unless method_defined?(:timeout=)
|
|
18
|
+
# Store timeout state for compatibility with IO-like timeout interfaces.
|
|
19
|
+
# @parameter duration [Numeric | Nil] The timeout to assign.
|
|
20
|
+
# @returns [Numeric | Nil] The assigned timeout.
|
|
21
|
+
def timeout=(duration)
|
|
22
|
+
@timeout = duration
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
end
|
data/lib/io/stream/version.rb
CHANGED
data/lib/io/stream.rb
CHANGED
|
@@ -1,15 +1,24 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
# Released under the MIT License.
|
|
4
|
-
# Copyright, 2023-
|
|
4
|
+
# Copyright, 2023-2026, by Samuel Williams.
|
|
5
5
|
|
|
6
6
|
require_relative "stream/version"
|
|
7
7
|
require_relative "stream/buffered"
|
|
8
|
+
require_relative "stream/duplex"
|
|
8
9
|
|
|
9
10
|
# @namespace
|
|
10
11
|
class IO
|
|
11
12
|
# @namespace
|
|
12
13
|
module Stream
|
|
14
|
+
# Construct a buffered duplex stream from separate input and output endpoints.
|
|
15
|
+
# @parameter input [IO] The readable endpoint.
|
|
16
|
+
# @parameter output [IO] The writable endpoint.
|
|
17
|
+
# @parameter options [Hash] Additional options passed to the buffered stream wrapper.
|
|
18
|
+
# @returns [IO::Stream::Buffered] A buffered stream wrapping a duplex transport.
|
|
19
|
+
def self.Duplex(input, output = input, **options)
|
|
20
|
+
Buffered.wrap(Duplex.new(input, output), **options)
|
|
21
|
+
end
|
|
13
22
|
end
|
|
14
23
|
|
|
15
24
|
# Convert any IO-like object into a buffered stream.
|
data/license.md
CHANGED
data/readme.md
CHANGED
|
@@ -4,13 +4,30 @@ Provide a buffered stream implementation for Ruby, independent of the underlying
|
|
|
4
4
|
|
|
5
5
|
[](https://github.com/socketry/io-stream/actions?workflow=Test)
|
|
6
6
|
|
|
7
|
+
## Motivation
|
|
8
|
+
|
|
9
|
+
I built this gem because working with IO in Ruby can be surprisingly difficult. Ruby provides buffering, but the inconsistencies between different IO types made it impossible to write clean, generic code. `OpenSSL::SSL::SSLSocket` maintains its own buffering implementation that behaves differently from regular IO. Some IO types raise `OpenSSL::SSL::SSLError` on connection reset while others raise `Errno::ECONNRESET`. EOF semantics vary. Close operations can hang (especially with SSL sockets). And if you want to work with non-blocking IO using `read_nonblock` and `write_nonblock`, you're constantly handling `:wait_readable` and `:wait_writable` conditions, managing timeouts, and dealing with edge cases that differ across implementations.
|
|
10
|
+
|
|
11
|
+
By providing a standard interface for buffered IO, `io-stream` allows you to write code that works the same way regardless of the underlying IO type. You can wrap any IO object and get consistent buffering behavior, unified error handling, and proper management of blocking/non-blocking operations. This makes it much easier to write high-performance IO code without worrying about the quirks of each specific IO implementation. Over time, as we've upstreamed more fixes into Ruby, we've been able to reduce the number of workarounds needed, but the core value of `io-stream` remains: a single, predictable interface for all your IO needs.
|
|
12
|
+
|
|
7
13
|
## Usage
|
|
8
14
|
|
|
9
|
-
Please see the [project documentation](https://socketry.github.io/io-stream) for more details.
|
|
15
|
+
Please see the [project documentation](https://socketry.github.io/io-stream/) for more details.
|
|
16
|
+
|
|
17
|
+
- [Getting Started](https://socketry.github.io/io-stream/guides/getting-started/index) - This guide explains how to use `io-stream` to add efficient buffering to Ruby IO objects.
|
|
18
|
+
|
|
19
|
+
- [High Performance IO](https://socketry.github.io/io-stream/guides/high-performance-io/index) - This guide explains how to achieve optimal performance when using `io-stream` by understanding and controlling flush behavior.
|
|
10
20
|
|
|
11
21
|
## Releases
|
|
12
22
|
|
|
13
|
-
Please see the [project releases](https://socketry.github.io/io-
|
|
23
|
+
Please see the [project releases](https://socketry.github.io/io-stream/releases/index) for all releases.
|
|
24
|
+
|
|
25
|
+
### v0.12.0
|
|
26
|
+
|
|
27
|
+
- Introduce `IO::Stream::Duplex` as a low-level duplex transport for composing separate input and output endpoints.
|
|
28
|
+
- Add `IO::Stream::Duplex(input, output)` as a convenient constructor that returns a buffered stream wrapping a duplex transport.
|
|
29
|
+
- Add a timeout compatibility shim for `StringIO` so duplex streams composed from in-memory endpoints can participate in the timeout interface consistently.
|
|
30
|
+
- Remove old OpenSSL method shims.
|
|
14
31
|
|
|
15
32
|
### v0.11.0
|
|
16
33
|
|
|
@@ -53,12 +70,6 @@ Please see the [project releases](https://socketry.github.io/io-streamreleases/i
|
|
|
53
70
|
- Add support for `read_until(limit:)` parameter to limit the amount of data read.
|
|
54
71
|
- Minor documentation improvements.
|
|
55
72
|
|
|
56
|
-
### v0.4.3
|
|
57
|
-
|
|
58
|
-
- Add comprehensive tests for `buffered?` method on `SSLSocket`.
|
|
59
|
-
- Ensure TLS connections have correct buffering behavior.
|
|
60
|
-
- Improve test suite organization and readability.
|
|
61
|
-
|
|
62
73
|
## See Also
|
|
63
74
|
|
|
64
75
|
- [async-io](https://github.com/socketry/async-io) — Where this implementation originally came from.
|
|
@@ -73,6 +84,22 @@ We welcome contributions to this project.
|
|
|
73
84
|
4. Push to the branch (`git push origin my-new-feature`).
|
|
74
85
|
5. Create new Pull Request.
|
|
75
86
|
|
|
87
|
+
### Running Tests
|
|
88
|
+
|
|
89
|
+
To run the test suite:
|
|
90
|
+
|
|
91
|
+
``` shell
|
|
92
|
+
bundle exec sus
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
### Making Releases
|
|
96
|
+
|
|
97
|
+
To make a new release:
|
|
98
|
+
|
|
99
|
+
``` shell
|
|
100
|
+
bundle exec bake gem:release:patch # or minor or major
|
|
101
|
+
```
|
|
102
|
+
|
|
76
103
|
### Developer Certificate of Origin
|
|
77
104
|
|
|
78
105
|
In order to protect users of this project, we require all contributors to comply with the [Developer Certificate of Origin](https://developercertificate.org/). This ensures that all contributions are properly licensed and attributed.
|
data/releases.md
CHANGED
|
@@ -1,5 +1,12 @@
|
|
|
1
1
|
# Releases
|
|
2
2
|
|
|
3
|
+
## v0.12.0
|
|
4
|
+
|
|
5
|
+
- Introduce `IO::Stream::Duplex` as a low-level duplex transport for composing separate input and output endpoints.
|
|
6
|
+
- Add `IO::Stream::Duplex(input, output)` as a convenient constructor that returns a buffered stream wrapping a duplex transport.
|
|
7
|
+
- Add a timeout compatibility shim for `StringIO` so duplex streams composed from in-memory endpoints can participate in the timeout interface consistently.
|
|
8
|
+
- Remove old OpenSSL method shims.
|
|
9
|
+
|
|
3
10
|
## v0.11.0
|
|
4
11
|
|
|
5
12
|
- Introduce `class IO::Stream::ConnectionResetError < Errno::ECONNRESET` to standardize connection reset error handling across different IO types.
|
data.tar.gz.sig
CHANGED
|
Binary file
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: io-stream
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.12.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Samuel Williams
|
|
@@ -42,14 +42,19 @@ executables: []
|
|
|
42
42
|
extensions: []
|
|
43
43
|
extra_rdoc_files: []
|
|
44
44
|
files:
|
|
45
|
+
- context/getting-started.md
|
|
46
|
+
- context/high-performance-io.md
|
|
47
|
+
- context/index.yaml
|
|
45
48
|
- lib/io/stream.rb
|
|
46
49
|
- lib/io/stream/buffered.rb
|
|
47
50
|
- lib/io/stream/connection_reset_error.rb
|
|
51
|
+
- lib/io/stream/duplex.rb
|
|
48
52
|
- lib/io/stream/generic.rb
|
|
49
53
|
- lib/io/stream/openssl.rb
|
|
50
54
|
- lib/io/stream/readable.rb
|
|
51
55
|
- lib/io/stream/shim/buffered.rb
|
|
52
56
|
- lib/io/stream/shim/readable.rb
|
|
57
|
+
- lib/io/stream/shim/timeout.rb
|
|
53
58
|
- lib/io/stream/string_buffer.rb
|
|
54
59
|
- lib/io/stream/version.rb
|
|
55
60
|
- lib/io/stream/writable.rb
|
|
@@ -60,7 +65,7 @@ homepage: https://github.com/socketry/io-stream
|
|
|
60
65
|
licenses:
|
|
61
66
|
- MIT
|
|
62
67
|
metadata:
|
|
63
|
-
documentation_uri: https://socketry.github.io/io-stream
|
|
68
|
+
documentation_uri: https://socketry.github.io/io-stream/
|
|
64
69
|
source_code_uri: https://github.com/socketry/io-stream.git
|
|
65
70
|
rdoc_options: []
|
|
66
71
|
require_paths:
|
|
@@ -69,14 +74,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
|
69
74
|
requirements:
|
|
70
75
|
- - ">="
|
|
71
76
|
- !ruby/object:Gem::Version
|
|
72
|
-
version: '3.
|
|
77
|
+
version: '3.3'
|
|
73
78
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
74
79
|
requirements:
|
|
75
80
|
- - ">="
|
|
76
81
|
- !ruby/object:Gem::Version
|
|
77
82
|
version: '0'
|
|
78
83
|
requirements: []
|
|
79
|
-
rubygems_version:
|
|
84
|
+
rubygems_version: 4.0.6
|
|
80
85
|
specification_version: 4
|
|
81
86
|
summary: Provides a generic stream wrapper for IO instances.
|
|
82
87
|
test_files: []
|
metadata.gz.sig
CHANGED
|
Binary file
|