uringmachine 0.25.0 → 0.27.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +9 -0
- data/TODO.md +17 -19
- data/docs/wroclove.rb.md +52 -0
- data/ext/um/um.c +469 -364
- data/ext/um/um.h +46 -22
- data/ext/um/um_async_op.c +9 -8
- data/ext/um/um_async_op_class.c +3 -3
- data/ext/um/um_class.c +215 -75
- data/ext/um/um_op.c +18 -2
- data/ext/um/um_ssl.c +1 -1
- data/ext/um/um_sync.c +18 -11
- data/ext/um/um_utils.c +14 -4
- data/grant-2025/tasks.md +2 -2
- data/lib/uringmachine/version.rb +1 -1
- data/test/test_async_op.rb +3 -2
- data/test/test_fiber_scheduler.rb +4 -1
- data/test/test_um.rb +410 -27
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 1211434f5b2f4aad1e47d1bd166204cce6792a80e3ef845ae4dee097d8e1ea45
|
|
4
|
+
data.tar.gz: eff370b4564c88c3daf57cbed3d8b165113ddee697ee1b16aa63ad3452a38c02
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: cdb26cf58a41a9f7ad2d025ccc473b0088f4c3e622a65058b1ee6e0aeb95c0eb3775f980cccb60e81b2213ed8fb49263c6500b2c05fbbb71abb156be00dca636
|
|
7
|
+
data.tar.gz: 06f509928387d2812578c143dfb96ed031be10994c7b43908babfef02d72d149b8f2a3932dee7a4d3577150b5512addb8b7c44d5b7ea2e63dddb7f2b427df604
|
data/CHANGELOG.md
CHANGED
data/TODO.md
CHANGED
|
@@ -1,15 +1,25 @@
|
|
|
1
1
|
## immediate
|
|
2
2
|
|
|
3
|
-
- Fix all futex value (Queue, Mutex) to be aligned
|
|
3
|
+
- Fix all futex value (Queue, Mutex) to be properly aligned
|
|
4
4
|
|
|
5
5
|
## Buffer rings - automatic management
|
|
6
6
|
|
|
7
|
+
- Take the buffer_pool branch, rewrite it
|
|
8
|
+
- Allow multiple stream modes:
|
|
9
|
+
- :buffer_pool - uses buffer rings
|
|
10
|
+
- :ssl - read from an SSL connection (`SSLSocket`)
|
|
11
|
+
- :io - read from an `IO`
|
|
12
|
+
|
|
13
|
+
The API will look something like:
|
|
14
|
+
|
|
7
15
|
```ruby
|
|
8
|
-
#
|
|
9
|
-
|
|
16
|
+
# The mode is selected automatically according to the given target
|
|
17
|
+
|
|
18
|
+
stream = UM::Stream.new(fd) # buffer_pool mode
|
|
10
19
|
|
|
11
|
-
|
|
12
|
-
|
|
20
|
+
stream = UM::Stream.new(ssl_sock) # ssl mode
|
|
21
|
+
|
|
22
|
+
stream = UM::Stream.new(conn) # io mode
|
|
13
23
|
```
|
|
14
24
|
|
|
15
25
|
## Balancing I/O with the runqueue
|
|
@@ -54,19 +64,6 @@ machine.read_each(fd, io_buffer: true) { |iobuff, len| ... }
|
|
|
54
64
|
- select on multiple queues (ala Go)
|
|
55
65
|
- select on mixture of queues and fds
|
|
56
66
|
|
|
57
|
-
(see also simplified op management below)
|
|
58
|
-
|
|
59
|
-
## simplified op management
|
|
60
|
-
|
|
61
|
-
Op lifecycle management can be much much simpler
|
|
62
|
-
|
|
63
|
-
- make all ops heap-allocated
|
|
64
|
-
- clear up state transitions:
|
|
65
|
-
|
|
66
|
-
- kernel-side state: unsubmitted, submitted, completed, done (for multishot ops)
|
|
67
|
-
- app-side state: unsubmitted, submitted, ...
|
|
68
|
-
|
|
69
|
-
|
|
70
67
|
## ops
|
|
71
68
|
|
|
72
69
|
- splice / - tee
|
|
@@ -94,7 +91,8 @@ We're still missing:
|
|
|
94
91
|
- ability to supply buffer to `get_line` and `get_string`
|
|
95
92
|
- allow read to eof, maybe with `read_to_eof`
|
|
96
93
|
|
|
97
|
-
For the sake of performance, simplicity and explicitness, we change the API as
|
|
94
|
+
For the sake of performance, simplicity and explicitness, we change the API as
|
|
95
|
+
follows:
|
|
98
96
|
|
|
99
97
|
```ruby
|
|
100
98
|
stream.get_line(buf, limit)
|
data/docs/wroclove.rb.md
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# UringMachine - High Performance Concurrency for Ruby Using io_uring
|
|
2
|
+
|
|
3
|
+
https://www.papercall.io/talks/413880/children/413881
|
|
4
|
+
|
|
5
|
+
## Structure
|
|
6
|
+
|
|
7
|
+
- Introduction: the name (1)
|
|
8
|
+
- What is io_uring (2)
|
|
9
|
+
- How UringMachine works (4)
|
|
10
|
+
- The liburing API
|
|
11
|
+
- Submitting an operation
|
|
12
|
+
- Fiber switching and the runqueue
|
|
13
|
+
- Waiting for and processing CQEs
|
|
14
|
+
- The UringMachine model:
|
|
15
|
+
- exhaust all CPU-bound work, submit I/O work to kernel
|
|
16
|
+
- Wait for and process completions
|
|
17
|
+
- repeat
|
|
18
|
+
- The UringMachine API (7)
|
|
19
|
+
- More or less equivalent to the Unix I/O API
|
|
20
|
+
- Synchronization: `UM::Mutex`, `UM::Queue`
|
|
21
|
+
- Useful abstractions for multishot operations
|
|
22
|
+
- OpenSSL support
|
|
23
|
+
- Custom BIO
|
|
24
|
+
- Streams / Automatic buffer management
|
|
25
|
+
- Utilities and advanced usage: `.inotify`, `.pipe`, `#splice`
|
|
26
|
+
- Timeouts and cancellations (4)
|
|
27
|
+
- Timeout - the basic design
|
|
28
|
+
- The challenges of cancellation
|
|
29
|
+
- double life cycle
|
|
30
|
+
- holding on to buffers
|
|
31
|
+
- ensuring the lifetime of relevant objects when doing an async operation.
|
|
32
|
+
- the TRANSIENT list of ops, for async operations (e.g. `#write_async` -
|
|
33
|
+
how to do we hold on to the write buffer.)
|
|
34
|
+
- how in general we mark the objects involved in async
|
|
35
|
+
- Integration with the Ruby ecosystem (6)
|
|
36
|
+
- The Fiber::Scheduler interface: the good, the bad and the ugly
|
|
37
|
+
- Dealing with CPU-bound workloads
|
|
38
|
+
- SQLite and Extralite in particular (`on_progress` handler)
|
|
39
|
+
- Performance (3)
|
|
40
|
+
- Benchmarks
|
|
41
|
+
- CPU-bound / IO-bound: how UringMachine deals with mixed workloads
|
|
42
|
+
- Applications (3)
|
|
43
|
+
- TP2 / Syntropy
|
|
44
|
+
- Uma - Rack-compatible app server
|
|
45
|
+
- Compare performance of a Rails app on Uma / Falcon / Puma
|
|
46
|
+
- Future directions (3)
|
|
47
|
+
- Further contributions to Ruby:
|
|
48
|
+
- Support for Socket I/O in Fiber::Scheduler
|
|
49
|
+
- More stress testing, prove stability
|
|
50
|
+
- More performance research
|
|
51
|
+
- Make Uma into a first-class app server for Ruby
|
|
52
|
+
- Introduce higher-level
|