io-extra 1.5.0 → 1.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 7411da9ba5734ed08fc6678a9d715629a6873593533835f54fcadda0349ac718
4
- data.tar.gz: d4e87cac61878af96b99030777a02870329b5180abce8eb8b4a4033217bf480f
3
+ metadata.gz: 473503dcaf54b7ee6ce1c289a17f1398296703e6d8aa7cc508a865923bbc1047
4
+ data.tar.gz: 0f5b24b543b6c4c6685bd5e6fc1802afab1aeaf58fccb4435a1cf5263f7d0a37
5
5
  SHA512:
6
- metadata.gz: 55b9ce62dac9906bc9b75c2033395a8185894baa0f31eabe681dda4fc76bd59c5ed566975a86c890138176e9ae37b424d2aaeaf8124b5f60c8c610fe11cc6fe1
7
- data.tar.gz: 8f1dd271f9046af836f2024f1e8857e71ab21542d2c56b8dfd861a8c681f092c6e69cfefa259d5ef77e78934e0478357d674637ac50b0bd11917b7fcb34c8bc5
6
+ metadata.gz: 1db0cb9ff89e94be8bb28a824d5d1acdadc74f8bb896548b2fc8ad3f45f3e5ed4c4ff1e6eb69bbe832343c4ca4e82880bf24aec0475cd8e2137448d62028431d
7
+ data.tar.gz: 38d3cb48ca1261dca1d62948591c6c15adfc846f708244c0b4f0d4fc08b77b7ecff319f042dbb2ce9a448629cc966def90a572a5135c28a93f38c01e3a6c14be
checksums.yaml.gz.sig CHANGED
Binary file
data/CHANGES.md CHANGED
@@ -1,3 +1,7 @@
1
+ ## 1.6.0 - 20-Nov-2026
2
+ * Added an IO.readv method.
3
+ * Updated the github workflow, removed some older Ruby versions, and added 4.0.
4
+
1
5
  ## 1.5.0 - 8-Jan-2026
2
6
  * The fdwalk method now works on MacOS.
3
7
  * Lots of internal refactoring in general, mostly for the writev method,
data/README.md CHANGED
@@ -3,7 +3,7 @@
3
3
  ## Description
4
4
  The io-extra library provides a few extra IO methods that you may find
5
5
  handy. They are `IO.closefrom`, `IO.fdwalk`, `IO.pread`, `IO.pwrite`,
6
- `IO.writev`, `IO#directio?` and `IO#directio=`.
6
+ `IO.writev`, `IO.readv`, `IO#directio?` and `IO#directio=`.
7
7
 
8
8
  This library is not supported on MS Windows.
9
9
 
@@ -31,8 +31,21 @@ IO.fdwalk(0){ |handle|
31
31
 
32
32
  # Write an array to an IO object efficiently
33
33
  IO.writev(fh.fileno, %w[a b c])
34
+
35
+ # Scatter a read into binary strings; an optional offset preserves position.
36
+ IO.readv(fh, [4, 8]) # => ["...", "..."]
37
+ IO.readv(fh, [4, 8], 0) # uses preadv when available
34
38
  ```
35
39
 
40
+ `IO.readv` accepts an IO object or file descriptor and an array of nonnegative
41
+ byte lengths. It returns one binary string per length. A short read leaves
42
+ subsequent strings empty; at EOF all strings are empty. It makes one read
43
+ operation, so a short read does not necessarily mean EOF. With an offset it
44
+ uses `preadv`, or raises `NotImplementedError` if unavailable. The method is
45
+ available when the platform supports `readv`. These calls bypass Ruby's IO
46
+ buffering: flush pending writes and avoid mixing them with buffered reads on
47
+ the same descriptor.
48
+
36
49
  ## Developer's Notes
37
50
  The "require 'io-extra'" is preferred over 'io/extra' because this is a mix
38
51
  of pure Ruby and C extension. The former require's the latter, so that way
data/Rakefile CHANGED
@@ -85,6 +85,11 @@ namespace :example do
85
85
  task :pread => [:build] do
86
86
  ruby '-Iext examples/example_pread.rb'
87
87
  end
88
+
89
+ desc "Run the IO.writev benchmark program."
90
+ task :writev => [:build] do
91
+ ruby '-Iext examples/writev_benchmark.rb'
92
+ end
88
93
  end
89
94
 
90
95
  desc "Run the test suite"
@@ -0,0 +1,171 @@
1
+ #!/usr/bin/env ruby
2
+ ##############################################################################
3
+ # example_writev.rb
4
+ #
5
+ # Demonstrates practical use cases for IO.writev, which writes multiple
6
+ # buffers in a single system call without concatenating them first.
7
+ #
8
+ # When to use writev:
9
+ # 1. Writing structured data with separate components (headers, body, etc.)
10
+ # 2. Logging with multiple fields that don't need string interpolation
11
+ # 3. Building protocol messages from discrete parts
12
+ # 4. Any scenario where you have an array of strings and want to avoid
13
+ # the memory overhead of joining them into one large string
14
+ ##############################################################################
15
+ require_relative '../lib/io-extra'
16
+
17
+ puts "IO.writev Example - Version #{IO::EXTRA_VERSION}"
18
+ puts "IOV_MAX (max buffers per call): #{IO::IOV_MAX}"
19
+ puts "-" * 60
20
+
21
+ ##############################################################################
22
+ # Example 1: Writing HTTP-like Response
23
+ # Instead of interpolating or joining strings, write each part directly
24
+ ##############################################################################
25
+ puts "\n=== Example 1: HTTP-like Response ==="
26
+
27
+ http_parts = [
28
+ "HTTP/1.1 200 OK\r\n",
29
+ "Content-Type: text/html\r\n",
30
+ "Content-Length: 13\r\n",
31
+ "Connection: close\r\n",
32
+ "\r\n",
33
+ "Hello, World!"
34
+ ]
35
+
36
+ File.open('http_response.txt', 'w') do |f|
37
+ bytes = IO.writev(f, http_parts)
38
+ puts "Wrote #{bytes} bytes for HTTP response"
39
+ end
40
+
41
+ puts "Contents:"
42
+ puts File.read('http_response.txt')
43
+ File.delete('http_response.txt')
44
+
45
+ ##############################################################################
46
+ # Example 2: Log Entry with Structured Fields
47
+ # Avoid string interpolation overhead for high-volume logging
48
+ ##############################################################################
49
+ puts "\n=== Example 2: Structured Log Entry ==="
50
+
51
+ def write_log_entry(io, timestamp, level, component, message)
52
+ parts = [
53
+ "[", timestamp, "] ",
54
+ "[", level.to_s.upcase.ljust(5), "] ",
55
+ "[", component, "] ",
56
+ message, "\n"
57
+ ]
58
+ IO.writev(io, parts)
59
+ end
60
+
61
+ File.open('app.log', 'w') do |log|
62
+ write_log_entry(log, Time.now.iso8601, :info, "Database", "Connection established")
63
+ write_log_entry(log, Time.now.iso8601, :warn, "Cache", "Cache miss for key 'user:123'")
64
+ write_log_entry(log, Time.now.iso8601, :error, "Auth", "Invalid token received")
65
+ end
66
+
67
+ puts "Log contents:"
68
+ puts File.read('app.log')
69
+ File.delete('app.log')
70
+
71
+ ##############################################################################
72
+ # Example 3: CSV Generation Without Building Large Strings
73
+ # Each row's fields can be written efficiently
74
+ ##############################################################################
75
+ puts "\n=== Example 3: CSV Generation ==="
76
+
77
+ def write_csv_row(io, fields, separator: ",", newline: "\n")
78
+ parts = []
79
+ fields.each_with_index do |field, i|
80
+ parts << separator if i > 0
81
+ parts << field.to_s
82
+ end
83
+ parts << newline
84
+ IO.writev(io, parts)
85
+ end
86
+
87
+ File.open('data.csv', 'w') do |csv|
88
+ # Header
89
+ write_csv_row(csv, ['Name', 'Email', 'Department', 'Salary'])
90
+
91
+ # Data rows
92
+ employees = [
93
+ ['Alice Smith', 'alice@example.com', 'Engineering', '95000'],
94
+ ['Bob Jones', 'bob@example.com', 'Marketing', '75000'],
95
+ ['Carol White', 'carol@example.com', 'Sales', '82000'],
96
+ ]
97
+
98
+ employees.each { |emp| write_csv_row(csv, emp) }
99
+ end
100
+
101
+ puts "CSV contents:"
102
+ puts File.read('data.csv')
103
+ File.delete('data.csv')
104
+
105
+ ##############################################################################
106
+ # Example 4: Building Network Protocol Messages
107
+ # Many protocols have fixed headers + variable payload
108
+ ##############################################################################
109
+ puts "\n=== Example 4: Protocol Message Construction ==="
110
+
111
+ def build_message(io, msg_type, payload)
112
+ header = [msg_type].pack('C') # 1-byte message type
113
+ length = [payload.bytesize].pack('N') # 4-byte length (big-endian)
114
+
115
+ # Write header, length, and payload in one syscall
116
+ bytes = IO.writev(io, [header, length, payload])
117
+ puts " Message type #{msg_type}: #{bytes} bytes (#{payload.bytesize} payload)"
118
+ bytes
119
+ end
120
+
121
+ File.open('messages.bin', 'wb') do |f|
122
+ build_message(f, 1, "HELLO")
123
+ build_message(f, 2, "This is a longer message payload")
124
+ build_message(f, 3, "BYE")
125
+ end
126
+
127
+ puts "Binary file size: #{File.size('messages.bin')} bytes"
128
+ File.delete('messages.bin')
129
+
130
+ ##############################################################################
131
+ # Example 5: Performance Comparison
132
+ # Shows memory efficiency when writing many small strings
133
+ ##############################################################################
134
+ puts "\n=== Example 5: Memory Efficiency Comparison ==="
135
+
136
+ # Simulate a scenario with many small pieces of data
137
+ pieces = 500.times.map { |i| "item_#{i}|" }
138
+
139
+ # Method 1: Join then write (creates intermediate string)
140
+ joined = pieces.join
141
+ puts "Joined string size: #{joined.bytesize} bytes"
142
+ puts " - Creates a new string object in memory"
143
+
144
+ # Method 2: writev (no intermediate string needed)
145
+ File.open('writev_output.txt', 'w') do |f|
146
+ bytes = IO.writev(f, pieces)
147
+ puts "writev wrote: #{bytes} bytes"
148
+ puts " - No intermediate string allocation!"
149
+ end
150
+
151
+ File.delete('writev_output.txt')
152
+
153
+ ##############################################################################
154
+ # Key Takeaways
155
+ ##############################################################################
156
+ puts "\n" + "=" * 60
157
+ puts "KEY TAKEAWAYS - When to use IO.writev:"
158
+ puts "=" * 60
159
+ puts <<~TIPS
160
+
161
+ ✓ You have pre-existing array of strings to write
162
+ ✓ You want to avoid memory allocation from Array#join
163
+ ✓ You're writing structured data with discrete components
164
+ ✓ High-performance logging or protocol implementations
165
+ ✓ Building output from template parts
166
+
167
+ ✗ NOT needed for single strings (use regular write)
168
+ ✗ NOT needed if you already have a joined string
169
+ ✗ Watch out for IOV_MAX limit (#{IO::IOV_MAX} buffers max)
170
+
171
+ TIPS
@@ -1,5 +1,5 @@
1
1
  require 'benchmark'
2
- require 'io-extra'
2
+ require 'io/extra'
3
3
 
4
4
  a = (0..1023).to_a.map(&:to_s)
5
5
 
data/ext/extconf.rb CHANGED
@@ -11,6 +11,8 @@ have_func('closefrom')
11
11
  have_func('fdwalk')
12
12
  have_func('directio')
13
13
  have_func('writev')
14
+ have_func('readv', 'sys/uio.h')
15
+ have_func('preadv', 'sys/uio.h')
14
16
  have_func('ttyname')
15
17
 
16
18
  case RbConfig::CONFIG['host_os']
data/ext/io/extra.c CHANGED
@@ -486,6 +486,122 @@ static VALUE s_io_writev(VALUE klass, VALUE fd, VALUE ary) {
486
486
  }
487
487
  #endif
488
488
 
489
+ #ifdef HAVE_READV
490
+ struct readv_args {
491
+ int fd, count, positional;
492
+ off_t offset;
493
+ struct iovec *iov;
494
+ VALUE lengths;
495
+ ssize_t result;
496
+ int error;
497
+ };
498
+
499
+ static void *nogvl_readv(void *ptr) {
500
+ struct readv_args *args = ptr;
501
+ #ifdef HAVE_PREADV
502
+ if (args->positional)
503
+ args->result = preadv(args->fd, args->iov, args->count, args->offset);
504
+ else
505
+ #endif
506
+ args->result = readv(args->fd, args->iov, args->count);
507
+ args->error = errno;
508
+ return NULL;
509
+ }
510
+
511
+ static VALUE readv_cleanup(VALUE ptr) {
512
+ struct readv_args *args = (struct readv_args *)ptr;
513
+ int i;
514
+ if (args->iov) {
515
+ for (i = 0; i < args->count; i++) xfree(args->iov[i].iov_base);
516
+ xfree(args->iov);
517
+ }
518
+ return Qnil;
519
+ }
520
+
521
+ static VALUE readv_body(VALUE ptr) {
522
+ struct readv_args *args = (struct readv_args *)ptr;
523
+ VALUE result;
524
+ size_t total = 0;
525
+ ssize_t remaining;
526
+ int i;
527
+
528
+ args->iov = ALLOC_N(struct iovec, args->count);
529
+ memset(args->iov, 0, sizeof(struct iovec) * args->count);
530
+ for (i = 0; i < args->count; i++) {
531
+ VALUE length = rb_ary_entry(args->lengths, i);
532
+ long n = NUM2LONG(length);
533
+ if (n < 0 || (size_t)n > (size_t)SSIZE_MAX - total)
534
+ rb_raise(rb_eArgError, "invalid buffer length or total exceeds SSIZE_MAX");
535
+ total += (size_t)n;
536
+ args->iov[i].iov_len = (size_t)n;
537
+ args->iov[i].iov_base = n ? xmalloc((size_t)n) : NULL;
538
+ }
539
+
540
+ do {
541
+ rb_thread_call_without_gvl(nogvl_readv, args, RUBY_UBF_IO, NULL);
542
+ if (args->result >= 0) break;
543
+ errno = args->error;
544
+ if (errno == EINTR) {
545
+ rb_thread_check_ints();
546
+ continue;
547
+ }
548
+ if ((errno == EAGAIN || errno == EWOULDBLOCK) && rb_io_wait_readable(args->fd))
549
+ continue;
550
+ errno = args->error;
551
+ rb_sys_fail(args->positional ? "preadv" : "readv");
552
+ } while (1);
553
+
554
+ remaining = args->result;
555
+ result = rb_ary_new_capa(args->count);
556
+ for (i = 0; i < args->count; i++) {
557
+ size_t n = args->iov[i].iov_len;
558
+ if (n > (size_t)remaining) n = (size_t)remaining;
559
+ rb_ary_push(result, rb_str_new(args->iov[i].iov_base, (long)n));
560
+ remaining -= (ssize_t)n;
561
+ }
562
+ return result;
563
+ }
564
+
565
+ /*
566
+ * IO.readv(io, lengths, offset = nil) -> array of binary strings
567
+ *
568
+ * Reads up to each length in order using readv. With an offset, uses preadv
569
+ * and leaves the file position unchanged. Short reads produce shorter strings
570
+ * and empty strings for any remaining entries; EOF yields all empty strings.
571
+ * This operates on the underlying descriptor, bypassing Ruby IO buffering.
572
+ *
573
+ * File.binwrite('example.txt', 'Hello world')
574
+ * File.open('example.txt', 'rb') do |file|
575
+ * IO.readv(file, [5, 1, 5]) # => ["Hello", " ", "world"]
576
+ * end
577
+ */
578
+ static VALUE s_io_readv(int argc, VALUE *argv, VALUE klass) {
579
+ VALUE fd, lengths, offset;
580
+ struct readv_args args = {0};
581
+ long count;
582
+ rb_scan_args(argc, argv, "21", &fd, &lengths, &offset);
583
+ Check_Type(lengths, T_ARRAY);
584
+ count = RARRAY_LEN(lengths);
585
+ if (count > IOV_MAX || count > INT_MAX)
586
+ rb_raise(rb_eArgError, "array is larger than IOV_MAX");
587
+ if (rb_respond_to(fd, rb_intern("fileno")))
588
+ fd = rb_funcall(fd, rb_intern("fileno"), 0);
589
+ args.fd = NUM2INT(fd);
590
+ args.count = (int)count;
591
+ args.lengths = lengths;
592
+ if (!NIL_P(offset)) {
593
+ #ifdef HAVE_PREADV
594
+ args.offset = NUM2OFFT(offset);
595
+ if (args.offset < 0) rb_raise(rb_eArgError, "offset must be non-negative");
596
+ args.positional = 1;
597
+ #else
598
+ rb_raise(rb_eNotImpError, "preadv is not available on this platform");
599
+ #endif
600
+ }
601
+ return rb_ensure(readv_body, (VALUE)&args, readv_cleanup, (VALUE)&args);
602
+ }
603
+ #endif
604
+
489
605
  #ifdef HAVE_TTYNAME
490
606
  /*
491
607
  * io.ttyname
@@ -543,6 +659,10 @@ void Init_extra(void){
543
659
  rb_define_singleton_method(rb_cIO, "writev", s_io_writev, 2);
544
660
  #endif
545
661
 
662
+ #ifdef HAVE_READV
663
+ rb_define_singleton_method(rb_cIO, "readv", s_io_readv, -1);
664
+ #endif
665
+
546
666
  #ifdef HAVE_TTYNAME
547
667
  rb_define_method(rb_cIO, "ttyname", io_get_ttyname, 0);
548
668
  #endif
data/io-extra.gemspec CHANGED
@@ -8,7 +8,7 @@ Gem::Specification.new do |spec|
8
8
  end
9
9
 
10
10
  spec.name = 'io-extra'
11
- spec.version = '1.5.0'
11
+ spec.version = '1.6.0'
12
12
  spec.author = 'Daniel J. Berger'
13
13
  spec.license = 'Apache-2.0'
14
14
  spec.email = 'djberg96@gmail.com'
@@ -44,7 +44,7 @@ Gem::Specification.new do |spec|
44
44
  }
45
45
 
46
46
  spec.description = <<-EOF
47
- Adds the IO.closefrom, IO.fdwalk, IO.pread, IO.pwrite, and IO.writev
47
+ Adds the IO.closefrom, IO.fdwalk, IO.pread, IO.pwrite, IO.readv, and IO.writev
48
48
  singleton methods as well as the IO#directio, IO#directio? and IO#ttyname
49
49
  instance methods (for those platforms that support them).
50
50
  EOF
data/lib/io-extra.rb CHANGED
@@ -3,7 +3,7 @@ require 'io/extra'
3
3
  # Reopen the core IO class to define pread and pwrite singleton methods
4
4
  # for backwards compatibility.
5
5
  class IO
6
- EXTRA_VERSION = '1.5.0'.freeze
6
+ EXTRA_VERSION = '1.6.0'.freeze
7
7
 
8
8
  # Singleton version of the IO#pwrite method.
9
9
  #
@@ -27,7 +27,7 @@ describe IO do
27
27
 
28
28
  context 'constants' do
29
29
  example 'EXTRA_VERSION' do
30
- expect(IO::EXTRA_VERSION).to eq('1.5.0')
30
+ expect(IO::EXTRA_VERSION).to eq('1.6.0')
31
31
  expect(IO::EXTRA_VERSION).to be_frozen
32
32
  end
33
33
 
@@ -120,6 +120,62 @@ describe IO do
120
120
  end
121
121
  end
122
122
 
123
+ context 'readv' do
124
+ before do
125
+ skip 'readv unavailable' unless IO.respond_to?(:readv)
126
+ @fh.flush
127
+ @fh.rewind
128
+ end
129
+
130
+ example 'reads binary segments and advances the descriptor' do
131
+ expect(IO.readv(@fh, [4, 5, 0])).to eq(['The ', 'quick', ''])
132
+ expect(@fh.pos).to eq(9)
133
+ end
134
+
135
+ example 'returns short and empty segments at EOF' do
136
+ @fh.seek(-3, IO::SEEK_END)
137
+ expect(IO.readv(@fh.fileno, [2, 4, 1])).to eq(["ck", "\n", ''])
138
+ expect(IO.readv(@fh, [1, 2])).to eq(['', ''])
139
+ end
140
+
141
+ example 'reads at an offset without changing position' do
142
+ @fh.seek(2)
143
+ begin
144
+ expect(IO.readv(@fh, [5, 1], 4)).to eq(['quick', ' '])
145
+ rescue NotImplementedError
146
+ skip 'preadv unavailable'
147
+ end
148
+ expect(@fh.pos).to eq(2)
149
+ end
150
+
151
+ example 'validates lengths and offsets' do
152
+ expect { IO.readv(@fh, [-1]) }.to raise_error(ArgumentError)
153
+ expect { IO.readv(@fh, [1, 'x']) }.to raise_error(TypeError)
154
+ expect { IO.readv(@fh, Array.new(IO::IOV_MAX + 1, 0)) }.to raise_error(ArgumentError)
155
+ expect { IO.readv(-1, [1]) }.to raise_error(Errno::EBADF)
156
+ end
157
+
158
+ example 'preserves null bytes and returns binary strings' do
159
+ @fh.rewind
160
+ @fh.write("a\0b")
161
+ @fh.flush
162
+ @fh.rewind
163
+ parts = IO.readv(@fh, [2, 1])
164
+ expect(parts).to eq(["a\0", 'b'])
165
+ expect(parts.map(&:encoding)).to eq([Encoding::BINARY, Encoding::BINARY])
166
+ end
167
+
168
+ example 'returns promptly on a short pipe read' do
169
+ rd, wr = IO.pipe
170
+ wr.write('abc')
171
+ wr.flush
172
+ expect(IO.readv(rd, [2, 4])).to eq(['ab', 'c'])
173
+ ensure
174
+ rd&.close
175
+ wr&.close
176
+ end
177
+ end
178
+
123
179
  context 'writev' do
124
180
  example 'writev' do
125
181
  expect(IO).to respond_to(:writev)
data.tar.gz.sig CHANGED
Binary file
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: io-extra
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.5.0
4
+ version: 1.6.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Daniel J. Berger
@@ -93,7 +93,7 @@ dependencies:
93
93
  - !ruby/object:Gem::Version
94
94
  version: '0'
95
95
  description: |2
96
- Adds the IO.closefrom, IO.fdwalk, IO.pread, IO.pwrite, and IO.writev
96
+ Adds the IO.closefrom, IO.fdwalk, IO.pread, IO.pwrite, IO.readv, and IO.writev
97
97
  singleton methods as well as the IO#directio, IO#directio? and IO#ttyname
98
98
  instance methods (for those platforms that support them).
99
99
  email: djberg96@gmail.com
@@ -116,6 +116,7 @@ files:
116
116
  - doc/io_extra.txt
117
117
  - examples/example_io_extra.rb
118
118
  - examples/example_pread.rb
119
+ - examples/example_writev.rb
119
120
  - examples/writev_benchmark.rb
120
121
  - ext/extconf.rb
121
122
  - ext/extra.bundle.dSYM/Contents/Info.plist
@@ -151,7 +152,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
151
152
  - !ruby/object:Gem::Version
152
153
  version: '0'
153
154
  requirements: []
154
- rubygems_version: 3.6.9
155
+ rubygems_version: 4.0.20
155
156
  specification_version: 4
156
157
  summary: Adds extra methods to the IO class
157
158
  test_files:
metadata.gz.sig CHANGED
Binary file