lightio 0.1.0.pre → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.travis.yml +14 -1
- data/Gemfile.lock +1 -1
- data/README.md +53 -5
- data/examples/beams.rb +15 -0
- data/examples/echo_server_with_raw_socket.rb +47 -0
- data/lib/lightio.rb +11 -6
- data/lib/lightio/core.rb +11 -0
- data/lib/lightio/{backend → core/backend}/nio.rb +41 -3
- data/lib/lightio/core/beam.rb +133 -0
- data/lib/lightio/core/future.rb +48 -0
- data/lib/lightio/core/ioloop.rb +66 -0
- data/lib/lightio/core/light_fiber.rb +15 -0
- data/lib/lightio/errors.rb +7 -0
- data/lib/lightio/library.rb +14 -0
- data/lib/lightio/library/kernel_ext.rb +18 -0
- data/lib/lightio/library/queue.rb +100 -0
- data/lib/lightio/library/timeout.rb +18 -0
- data/lib/lightio/version.rb +2 -2
- data/lib/lightio/watchers.rb +9 -0
- data/lib/lightio/watchers/io.rb +110 -0
- data/lib/lightio/watchers/schedule.rb +9 -0
- data/lib/lightio/watchers/timer.rb +16 -0
- data/lib/lightio/watchers/watcher.rb +16 -0
- data/lightio.gemspec +4 -4
- metadata +29 -17
- data/lib/lightio/beam.rb +0 -12
- data/lib/lightio/future.rb +0 -35
- data/lib/lightio/ioloop.rb +0 -40
- data/lib/lightio/timer.rb +0 -15
- data/lib/lightio/watcher.rb +0 -14
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 0b2408fef926bf934a659bdd99bb75218c0bb940
|
4
|
+
data.tar.gz: 21c19305f235b572a8131a8d8059b0038701c79e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 3f11ea750958ac0d3c64c4db0ec97c5e1025ada8d1354f7e858f956b0e8e7aacd64fd0983831c9421039dd70c939bc4df466de24888587527962bf095257eb8c
|
7
|
+
data.tar.gz: 00db900dd4891ab01d6dc971e769d8863e64751e2ccf7a70d936413049316ec956c9ca3c5d3db62a30db87f143f0987d36160c8056f589a5e3e39df3484cba91
|
data/.travis.yml
CHANGED
@@ -1,5 +1,18 @@
|
|
1
1
|
sudo: false
|
2
2
|
language: ruby
|
3
3
|
rvm:
|
4
|
+
- jruby-9.1.15.0 # latest stable
|
5
|
+
- 2.2.7
|
6
|
+
- 2.3.4
|
4
7
|
- 2.4.1
|
5
|
-
|
8
|
+
- ruby-head
|
9
|
+
|
10
|
+
env:
|
11
|
+
global:
|
12
|
+
- JRUBY_OPTS="--dev -J-Djruby.launch.inproc=true -J-Xmx1024M"
|
13
|
+
|
14
|
+
matrix:
|
15
|
+
allow_failures:
|
16
|
+
- rvm: ruby-head
|
17
|
+
- rvm: jruby-9.1.15.0
|
18
|
+
fast_finish: true
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -1,8 +1,28 @@
|
|
1
|
-
#
|
1
|
+
# LightIO
|
2
2
|
|
3
|
-
Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/lightio`. To experiment with that code, run `bin/console` for an interactive prompt.
|
4
3
|
|
5
|
-
|
4
|
+
[](http://rubygems.org/gems/lightio)
|
5
|
+
[](https://travis-ci.org/jjyr/lightio)
|
6
|
+
[](https://github.com/jjyr/lightio/blob/master/LICENSE.txt)
|
7
|
+
|
8
|
+
LightIO is a ruby networking library, that combines ruby fiber and IO event loop to provide both simple synchrony library interface and high performance networking IO.
|
9
|
+
|
10
|
+
|
11
|
+
LightIO is heavily inspired by [gevent](http://www.gevent.org/).
|
12
|
+
|
13
|
+
## Current Status
|
14
|
+
|
15
|
+
This library is still **WIP**, *watch* or *star* this repo for further information.
|
16
|
+
Its not recommendation to use LightIO in production now, but you can always give a try. open a issue if you have any question.
|
17
|
+
|
18
|
+
Before we release a full networking framework at the version 1.0.0, there are three targets(milestones).
|
19
|
+
|
20
|
+
1. Provide a bare-bone framework, include fiber based lightweight executor and a way to collebrate with ruby socket library
|
21
|
+
2. Provide lightio networking libraries, that have the same API with ruby stdlib
|
22
|
+
3. Implement ruby stdlib monkey patch, user can apply monkey patch and just write normal ruby code to get the power of lightio
|
23
|
+
|
24
|
+
Thanks to [nio4r](https://github.com/socketry/nio4r), the first target is already achieved.
|
25
|
+
|
6
26
|
|
7
27
|
## Installation
|
8
28
|
|
@@ -22,7 +42,35 @@ Or install it yourself as:
|
|
22
42
|
|
23
43
|
## Usage
|
24
44
|
|
25
|
-
|
45
|
+
``` ruby
|
46
|
+
require 'lightio'
|
47
|
+
|
48
|
+
start = Time.now
|
49
|
+
|
50
|
+
beams = 1000.times.map do
|
51
|
+
# LightIO::Beam is a thread-like executor, use it instead Thread
|
52
|
+
LightIO::Beam.new do
|
53
|
+
# do some io operations in beam
|
54
|
+
LightIO.sleep(1)
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
beams.each(&:join)
|
59
|
+
seconds = Time.now - start
|
60
|
+
puts "1000 beams take #{seconds - 1} seconds to create"
|
61
|
+
```
|
62
|
+
|
63
|
+
View more [examples](/examples).
|
64
|
+
|
65
|
+
## Documentation
|
66
|
+
|
67
|
+
See [wiki](https://github.com/jjyr/lightio/wiki) for more information
|
68
|
+
|
69
|
+
[API Documentation](http://www.rubydoc.info/gems/lightio/frames)
|
70
|
+
|
71
|
+
## Discussion
|
72
|
+
|
73
|
+
[Discussion on Gitter](https://gitter.im/lightio-dev/Lobby?utm_source=share-link&utm_medium=link&utm_campaign=share-link)
|
26
74
|
|
27
75
|
## Development
|
28
76
|
|
@@ -32,7 +80,7 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
|
|
32
80
|
|
33
81
|
## Contributing
|
34
82
|
|
35
|
-
Bug reports and pull requests are welcome on GitHub at https://github.com/
|
83
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/jjyr/lightio. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
|
36
84
|
|
37
85
|
## License
|
38
86
|
|
data/examples/beams.rb
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
require 'lightio'
|
2
|
+
|
3
|
+
start = Time.now
|
4
|
+
|
5
|
+
beams = 1000.times.map do
|
6
|
+
# LightIO::Beam is a thread-like executor, use it instead Thread
|
7
|
+
LightIO::Beam.new do
|
8
|
+
# do some io operations in beam
|
9
|
+
LightIO.sleep(1)
|
10
|
+
end
|
11
|
+
end
|
12
|
+
|
13
|
+
beams.each(&:join)
|
14
|
+
seconds = Time.now - start
|
15
|
+
puts "1000 beams take #{seconds - 1} seconds to create"
|
@@ -0,0 +1,47 @@
|
|
1
|
+
# Example from https://github.com/socketry/nio4r/blob/master/examples/echo_server.rb
|
2
|
+
# rewrite it in lightio for demonstrate
|
3
|
+
# this example demonstrate how to use ruby 'raw' socket with LightIO
|
4
|
+
|
5
|
+
require 'lightio'
|
6
|
+
require 'socket'
|
7
|
+
|
8
|
+
class EchoServer
|
9
|
+
def initialize(host, port)
|
10
|
+
@server = TCPServer.new(host, port)
|
11
|
+
end
|
12
|
+
|
13
|
+
def run
|
14
|
+
# wait server until readable
|
15
|
+
server_watcher = LightIO::Watchers::IO.new(@server, :r)
|
16
|
+
while server_watcher.wait_read
|
17
|
+
socket = @server.accept
|
18
|
+
_, port, host = socket.peeraddr
|
19
|
+
puts "accept connection from #{host}:#{port}"
|
20
|
+
|
21
|
+
# LightIO::Beam is lightweight executor, provide thread-like interface
|
22
|
+
# just start new beam for per socket
|
23
|
+
LightIO::Beam.new(socket) do |socket|
|
24
|
+
socket_watcher = LightIO::Watchers::IO.new(socket, :r)
|
25
|
+
begin
|
26
|
+
while socket_watcher.wait_read
|
27
|
+
echo(socket)
|
28
|
+
end
|
29
|
+
rescue EOFError
|
30
|
+
_, port, host = socket.peeraddr
|
31
|
+
puts "*** #{host}:#{port} disconnected"
|
32
|
+
# remove close socket watcher
|
33
|
+
socket_watcher.close
|
34
|
+
socket.close
|
35
|
+
end
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
39
|
+
|
40
|
+
def echo(socket)
|
41
|
+
data = socket.read_nonblock(4096)
|
42
|
+
socket.write_nonblock(data)
|
43
|
+
end
|
44
|
+
end
|
45
|
+
|
46
|
+
|
47
|
+
EchoServer.new('localhost', 3000).run if __FILE__ == $0
|
data/lib/lightio.rb
CHANGED
@@ -1,10 +1,15 @@
|
|
1
|
+
# LightIO
|
1
2
|
require 'lightio/version'
|
2
|
-
require 'lightio/
|
3
|
-
require 'lightio/
|
4
|
-
require 'lightio/
|
5
|
-
require 'lightio/
|
6
|
-
require 'lightio/timer'
|
7
|
-
require 'lightio/backend/nio'
|
3
|
+
require 'lightio/errors'
|
4
|
+
require 'lightio/core'
|
5
|
+
require 'lightio/watchers'
|
6
|
+
require 'lightio/library'
|
8
7
|
|
8
|
+
# LightIO provide light-weight executor: LightIO::Beam and batch io libraries,
|
9
|
+
# view LightIO::Core::Beam to learn how to concurrent programming with Beam,
|
10
|
+
# view LightIO::Watchers::IO to learn how to manage 'raw' io objects,
|
11
|
+
# Core and Library modules are included under LightIO namespace, so you can use LightIO::Beam for convenient
|
9
12
|
module LightIO
|
13
|
+
include Core
|
14
|
+
include Library
|
10
15
|
end
|
data/lib/lightio/core.rb
ADDED
@@ -0,0 +1,11 @@
|
|
1
|
+
# Core
|
2
|
+
require 'lightio/core/ioloop'
|
3
|
+
require 'lightio/core/light_fiber'
|
4
|
+
require 'lightio/core/future'
|
5
|
+
require 'lightio/core/beam'
|
6
|
+
|
7
|
+
# LightIO::Core include core classes: Beam, IOloop, Future
|
8
|
+
#
|
9
|
+
# view examples for clues
|
10
|
+
module LightIO::Core
|
11
|
+
end
|
@@ -1,6 +1,7 @@
|
|
1
1
|
# use nio4r implement event loop, inspired from eventmachine/pure_ruby implement
|
2
|
+
require 'nio'
|
2
3
|
require 'set'
|
3
|
-
module LightIO
|
4
|
+
module LightIO::Core
|
4
5
|
module Backend
|
5
6
|
|
6
7
|
class Error < RuntimeError
|
@@ -46,12 +47,15 @@ module LightIO
|
|
46
47
|
end
|
47
48
|
end
|
48
49
|
|
50
|
+
# LightIO use NIO as default event-driving backend
|
49
51
|
class NIO
|
50
52
|
def initialize
|
51
53
|
# @selector = NIO::Selector.new
|
52
54
|
@current_loop_time = nil
|
53
55
|
@running = false
|
54
56
|
@timers = Timers.new
|
57
|
+
@callbacks = []
|
58
|
+
@selector = ::NIO::Selector.new
|
55
59
|
end
|
56
60
|
|
57
61
|
def run
|
@@ -60,11 +64,14 @@ module LightIO
|
|
60
64
|
loop do
|
61
65
|
@current_loop_time = Time.now
|
62
66
|
run_timers
|
67
|
+
run_callbacks
|
68
|
+
handle_selectables
|
63
69
|
end
|
64
70
|
end
|
65
71
|
|
66
|
-
|
67
|
-
|
72
|
+
|
73
|
+
def add_callback(&blk)
|
74
|
+
@callbacks << blk
|
68
75
|
end
|
69
76
|
|
70
77
|
def add_timer(timer)
|
@@ -75,11 +82,42 @@ module LightIO
|
|
75
82
|
@timers.cancel_timer(timer)
|
76
83
|
end
|
77
84
|
|
85
|
+
def add_io_wait(io, interests, &blk)
|
86
|
+
monitor = @selector.register(io, interests)
|
87
|
+
monitor.value = blk
|
88
|
+
monitor
|
89
|
+
end
|
90
|
+
|
91
|
+
def cancel_io_wait(io)
|
92
|
+
@selector.deregister(io)
|
93
|
+
end
|
94
|
+
|
78
95
|
def stop
|
79
96
|
return unless @running
|
80
97
|
@running = false
|
81
98
|
raise
|
82
99
|
end
|
100
|
+
|
101
|
+
private
|
102
|
+
|
103
|
+
def run_timers
|
104
|
+
@timers.fire(@current_loop_time)
|
105
|
+
end
|
106
|
+
|
107
|
+
def handle_selectables
|
108
|
+
@selector.select(0) do |monitor|
|
109
|
+
# invoke callback if io is ready
|
110
|
+
if monitor.readiness
|
111
|
+
monitor.value.call(monitor.io)
|
112
|
+
end
|
113
|
+
end
|
114
|
+
end
|
115
|
+
|
116
|
+
def run_callbacks
|
117
|
+
while (callback = @callbacks.shift)
|
118
|
+
callback.call
|
119
|
+
end
|
120
|
+
end
|
83
121
|
end
|
84
122
|
end
|
85
123
|
end
|
@@ -0,0 +1,133 @@
|
|
1
|
+
module LightIO::Core
|
2
|
+
# Beam is light-weight executor, provide thread-like interface
|
3
|
+
#
|
4
|
+
# @Example:
|
5
|
+
# #- initialize with block
|
6
|
+
# b = Beam.new{puts "hello"}
|
7
|
+
# b.join
|
8
|
+
# #output: hello
|
9
|
+
#
|
10
|
+
# b = Beam.new(1,2,3){|one, two, three| puts [one, two, three].join(",") }
|
11
|
+
# b.join
|
12
|
+
# #output: 1,2,3
|
13
|
+
#
|
14
|
+
# #- use join wait beam done
|
15
|
+
# b = Beam.new(){LightIO.sleep 3}
|
16
|
+
# b.join
|
17
|
+
# b.alive? # false
|
18
|
+
class Beam < LightFiber
|
19
|
+
|
20
|
+
# Create a new beam
|
21
|
+
#
|
22
|
+
# Beam is light-weight executor, provide thread-like interface
|
23
|
+
#
|
24
|
+
# Beam.new("hello"){|hello| puts hello }
|
25
|
+
#
|
26
|
+
# @param [Array] args pass arguments to Beam block
|
27
|
+
# @param [Proc] blk block to execute
|
28
|
+
# @return [Beam]
|
29
|
+
def initialize(*args, &blk)
|
30
|
+
raise Error, "must be called with a block" unless blk
|
31
|
+
super() {
|
32
|
+
begin
|
33
|
+
@value = yield(*args)
|
34
|
+
rescue StandardError => e
|
35
|
+
@error = e
|
36
|
+
end
|
37
|
+
# mark as dead
|
38
|
+
dead
|
39
|
+
# transfer back to parent(caller fiber) after schedule
|
40
|
+
parent.transfer
|
41
|
+
}
|
42
|
+
# schedule beam in ioloop
|
43
|
+
ioloop.add_callback {transfer}
|
44
|
+
@alive = true
|
45
|
+
end
|
46
|
+
|
47
|
+
def alive?
|
48
|
+
super && @alive
|
49
|
+
end
|
50
|
+
|
51
|
+
# block and wait beam return a value
|
52
|
+
def value
|
53
|
+
if alive?
|
54
|
+
self.parent = Beam.current
|
55
|
+
ioloop.transfer
|
56
|
+
end
|
57
|
+
check_and_raise_error
|
58
|
+
@value
|
59
|
+
end
|
60
|
+
|
61
|
+
# Block and wait beam dead
|
62
|
+
#
|
63
|
+
# @param [Numeric] limit wait limit seconds if limit > 0, return nil if beam still alive, else return beam self
|
64
|
+
# @return [Beam, nil]
|
65
|
+
def join(limit=0)
|
66
|
+
# try directly get result
|
67
|
+
if !alive? || limit <= 0
|
68
|
+
# call value to raise error
|
69
|
+
value
|
70
|
+
return self
|
71
|
+
end
|
72
|
+
|
73
|
+
# set a transfer back timer
|
74
|
+
parent = Beam.current
|
75
|
+
timer = LightIO::Watchers::Timer.new(limit)
|
76
|
+
timer.set_callback {parent.transfer}
|
77
|
+
ioloop.add_timer(timer)
|
78
|
+
ioloop.transfer
|
79
|
+
|
80
|
+
if alive?
|
81
|
+
nil
|
82
|
+
else
|
83
|
+
check_and_raise_error
|
84
|
+
self
|
85
|
+
end
|
86
|
+
end
|
87
|
+
|
88
|
+
# Kill beam
|
89
|
+
#
|
90
|
+
# @return [Beam]
|
91
|
+
def kill
|
92
|
+
dead
|
93
|
+
parent.transfer if self == Beam.current
|
94
|
+
self
|
95
|
+
end
|
96
|
+
|
97
|
+
class << self
|
98
|
+
|
99
|
+
# Schedule beams
|
100
|
+
#
|
101
|
+
# normally beam should be auto scheduled, use this method to manually trigger a schedule
|
102
|
+
#
|
103
|
+
# @return [nil]
|
104
|
+
def pass
|
105
|
+
schedule = LightIO::Watchers::Schedule.new
|
106
|
+
IOloop.current.wait(schedule)
|
107
|
+
end
|
108
|
+
end
|
109
|
+
|
110
|
+
private
|
111
|
+
|
112
|
+
# mark beam as dead
|
113
|
+
def dead
|
114
|
+
@alive = false
|
115
|
+
end
|
116
|
+
|
117
|
+
# Beam transfer back to parent after schedule
|
118
|
+
# parent is fiber or beam who called value/join methods
|
119
|
+
# if not present a parent, Beam will transfer to ioloop
|
120
|
+
def parent=(parent)
|
121
|
+
@parent = parent
|
122
|
+
end
|
123
|
+
|
124
|
+
# get parent/ioloop to transfer back
|
125
|
+
def parent
|
126
|
+
@parent || ioloop
|
127
|
+
end
|
128
|
+
|
129
|
+
def check_and_raise_error
|
130
|
+
raise @error if @error
|
131
|
+
end
|
132
|
+
end
|
133
|
+
end
|
@@ -0,0 +1,48 @@
|
|
1
|
+
module LightIO::Core
|
2
|
+
# Provide a safe way to transfer beam/fiber control flow.
|
3
|
+
#
|
4
|
+
# @Example:
|
5
|
+
# future = Future.new
|
6
|
+
# # future#value will block current beam
|
7
|
+
# Beam.new{future.value}
|
8
|
+
# # use transfer to set value
|
9
|
+
# future.transfer(1)
|
10
|
+
class Future
|
11
|
+
def initialize
|
12
|
+
@value = nil
|
13
|
+
@ioloop = IOloop.current
|
14
|
+
@state = :init
|
15
|
+
end
|
16
|
+
|
17
|
+
def done?
|
18
|
+
@state == :done
|
19
|
+
end
|
20
|
+
|
21
|
+
# Transfer and set result value
|
22
|
+
#
|
23
|
+
# use this method to set back result
|
24
|
+
def transfer(value=nil)
|
25
|
+
raise Error, "state error" if done?
|
26
|
+
@value = value
|
27
|
+
done!
|
28
|
+
@light_fiber.transfer if @light_fiber
|
29
|
+
end
|
30
|
+
|
31
|
+
# Get value
|
32
|
+
#
|
33
|
+
# this method will block current beam/fiber, until future result is set.
|
34
|
+
def value
|
35
|
+
return @value if done?
|
36
|
+
raise Error, 'already used' if @light_fiber
|
37
|
+
@light_fiber = LightFiber.current
|
38
|
+
@ioloop.transfer
|
39
|
+
@value
|
40
|
+
end
|
41
|
+
|
42
|
+
private
|
43
|
+
|
44
|
+
def done!
|
45
|
+
@state = :done
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
@@ -0,0 +1,66 @@
|
|
1
|
+
require 'lightio/core/backend/nio'
|
2
|
+
module LightIO::Core
|
3
|
+
# IOloop like a per-threaded EventMachine (cause fiber cannot resume cross threads)
|
4
|
+
#
|
5
|
+
# IOloop handle io waiting and schedule beams, user do not supposed to directly use this class
|
6
|
+
class IOloop
|
7
|
+
|
8
|
+
def initialize
|
9
|
+
@fiber = Fiber.new {run}
|
10
|
+
@backend = Backend::NIO.new
|
11
|
+
end
|
12
|
+
|
13
|
+
# should never invoke explicitly
|
14
|
+
def run
|
15
|
+
# start io loop and never return...
|
16
|
+
@backend.run
|
17
|
+
end
|
18
|
+
|
19
|
+
def add_timer(timer)
|
20
|
+
@backend.add_timer(timer)
|
21
|
+
end
|
22
|
+
|
23
|
+
def add_callback(&blk)
|
24
|
+
@backend.add_callback(&blk)
|
25
|
+
end
|
26
|
+
|
27
|
+
def add_io_wait(io, interests, &blk)
|
28
|
+
@backend.add_io_wait(io, interests, &blk)
|
29
|
+
end
|
30
|
+
|
31
|
+
def cancel_io_wait(io)
|
32
|
+
@backend.cancel_io_wait(io)
|
33
|
+
end
|
34
|
+
|
35
|
+
# Wait a watcher, watcher can be a timer or socket.
|
36
|
+
# see LightIO::Watchers module for detail
|
37
|
+
def wait(watcher)
|
38
|
+
future = Future.new
|
39
|
+
# add watcher to loop
|
40
|
+
id = Object.new
|
41
|
+
watcher.set_callback {future.transfer id}
|
42
|
+
watcher.start(self)
|
43
|
+
# trigger a fiber switch
|
44
|
+
# wait until watcher is ok
|
45
|
+
# then do work
|
46
|
+
if (result = future.value) != id
|
47
|
+
raise InvalidTransferError, "expect #{id}, but get #{result}"
|
48
|
+
end
|
49
|
+
end
|
50
|
+
|
51
|
+
def transfer
|
52
|
+
@fiber.transfer
|
53
|
+
end
|
54
|
+
|
55
|
+
class << self
|
56
|
+
# return current ioloop or create new one
|
57
|
+
def current
|
58
|
+
key = :"lightio.ioloop"
|
59
|
+
unless Thread.current.thread_variable?(key)
|
60
|
+
Thread.current.thread_variable_set(key, IOloop.new)
|
61
|
+
end
|
62
|
+
Thread.current.thread_variable_get(key)
|
63
|
+
end
|
64
|
+
end
|
65
|
+
end
|
66
|
+
end
|
@@ -0,0 +1,15 @@
|
|
1
|
+
require 'fiber'
|
2
|
+
|
3
|
+
module LightIO::Core
|
4
|
+
# LightFiber is internal represent, we make slight extend on ruby Fiber to bind fibers to IOLoop
|
5
|
+
#
|
6
|
+
# SHOULD NOT BE USED DIRECTLY
|
7
|
+
class LightFiber < Fiber
|
8
|
+
attr_reader :ioloop
|
9
|
+
|
10
|
+
def initialize(ioloop: IOloop.current, &blk)
|
11
|
+
@ioloop = ioloop
|
12
|
+
super(&blk)
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
@@ -0,0 +1,14 @@
|
|
1
|
+
require 'lightio/library/queue'
|
2
|
+
require 'lightio/library/kernel_ext'
|
3
|
+
require 'lightio/library/timeout'
|
4
|
+
|
5
|
+
module LightIO
|
6
|
+
# Library include modules can cooperative with LightIO::Beam
|
7
|
+
module Library
|
8
|
+
# extend library modules
|
9
|
+
def self.included(base)
|
10
|
+
base.extend KernelExt
|
11
|
+
base.extend Timeout
|
12
|
+
end
|
13
|
+
end
|
14
|
+
end
|
@@ -0,0 +1,18 @@
|
|
1
|
+
module LightIO::Library
|
2
|
+
module KernelExt
|
3
|
+
def sleep(*duration)
|
4
|
+
if duration.size > 1
|
5
|
+
raise ArgumentError, "wrong number of arguments (given #{duration.size}, expected 0..1)"
|
6
|
+
elsif duration.size == 0
|
7
|
+
LightIO::IOloop.current.transfer
|
8
|
+
end
|
9
|
+
duration = duration[0]
|
10
|
+
if duration.zero? && LightIO::Beam.current.respond_to?(:pass)
|
11
|
+
LightIO::Beam.current.pass
|
12
|
+
return
|
13
|
+
end
|
14
|
+
timer = LightIO::Watchers::Timer.new duration
|
15
|
+
LightIO::IOloop.current.wait(timer)
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
@@ -0,0 +1,100 @@
|
|
1
|
+
module LightIO::Library
|
2
|
+
class Queue
|
3
|
+
def initialize
|
4
|
+
@queue = []
|
5
|
+
@waiters = []
|
6
|
+
@close = false
|
7
|
+
end
|
8
|
+
|
9
|
+
def close()
|
10
|
+
#This is a stub, used for indexing
|
11
|
+
@close = true
|
12
|
+
@waiters.each {|w| w.transfer nil}
|
13
|
+
self
|
14
|
+
end
|
15
|
+
|
16
|
+
# closed?
|
17
|
+
#
|
18
|
+
# Returns +true+ if the queue is closed.
|
19
|
+
def closed?()
|
20
|
+
@close
|
21
|
+
end
|
22
|
+
|
23
|
+
# push(object)
|
24
|
+
# enq(object)
|
25
|
+
# <<(object)
|
26
|
+
#
|
27
|
+
# Pushes the given +object+ to the queue.
|
28
|
+
def push(object)
|
29
|
+
raise ClosedQueueError, "queue closed" if @close
|
30
|
+
if @waiters.any?
|
31
|
+
future = LightIO::Future.new
|
32
|
+
LightIO::IOloop.current.add_callback {
|
33
|
+
@waiters.shift.transfer(object)
|
34
|
+
future.transfer
|
35
|
+
}
|
36
|
+
future.value
|
37
|
+
else
|
38
|
+
@queue << object
|
39
|
+
end
|
40
|
+
self
|
41
|
+
end
|
42
|
+
|
43
|
+
alias enq push
|
44
|
+
alias << push
|
45
|
+
# pop(non_block=false)
|
46
|
+
# deq(non_block=false)
|
47
|
+
# shift(non_block=false)
|
48
|
+
#
|
49
|
+
# Retrieves data from the queue.
|
50
|
+
#
|
51
|
+
# If the queue is empty, the calling thread is suspended until data is pushed
|
52
|
+
# onto the queue. If +non_block+ is true, the thread isn't suspended, and an
|
53
|
+
# exception is raised.
|
54
|
+
def pop(non_block=false)
|
55
|
+
if @close
|
56
|
+
return empty? ? nil : @queue.pop
|
57
|
+
end
|
58
|
+
if empty?
|
59
|
+
if non_block
|
60
|
+
raise ThreadError, 'queue empty'
|
61
|
+
else
|
62
|
+
future = LightIO::Future.new
|
63
|
+
@waiters << future
|
64
|
+
future.value
|
65
|
+
end
|
66
|
+
else
|
67
|
+
@queue.pop
|
68
|
+
end
|
69
|
+
end
|
70
|
+
|
71
|
+
alias deq pop
|
72
|
+
alias shift pop
|
73
|
+
# empty?
|
74
|
+
#
|
75
|
+
# Returns +true+ if the queue is empty.
|
76
|
+
def empty?()
|
77
|
+
@queue.empty?
|
78
|
+
end
|
79
|
+
|
80
|
+
# Removes all objects from the queue.
|
81
|
+
def clear()
|
82
|
+
@queue.clear
|
83
|
+
self
|
84
|
+
end
|
85
|
+
|
86
|
+
# length
|
87
|
+
# size
|
88
|
+
#
|
89
|
+
# Returns the length of the queue.
|
90
|
+
def length()
|
91
|
+
@queue.size
|
92
|
+
end
|
93
|
+
|
94
|
+
alias size length
|
95
|
+
# Returns the number of threads waiting on the queue.
|
96
|
+
def num_waiting()
|
97
|
+
@waiters.size
|
98
|
+
end
|
99
|
+
end
|
100
|
+
end
|
@@ -0,0 +1,18 @@
|
|
1
|
+
require 'timeout'
|
2
|
+
module LightIO::Library
|
3
|
+
module Timeout
|
4
|
+
extend self
|
5
|
+
Error = ::Timeout::Error
|
6
|
+
|
7
|
+
def timeout(sec, klass=Error, &blk)
|
8
|
+
return yield(sec) if sec.nil? or sec.zero?
|
9
|
+
beam = LightIO::Beam.new(sec, &blk)
|
10
|
+
message = "execution expired"
|
11
|
+
if beam.join(sec).nil?
|
12
|
+
raise klass, message
|
13
|
+
else
|
14
|
+
beam.value
|
15
|
+
end
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
data/lib/lightio/version.rb
CHANGED
@@ -1,3 +1,3 @@
|
|
1
|
-
module
|
2
|
-
VERSION = "0.1.0
|
1
|
+
module LightIO
|
2
|
+
VERSION = "0.1.0"
|
3
3
|
end
|
@@ -0,0 +1,9 @@
|
|
1
|
+
require 'lightio/watchers/watcher'
|
2
|
+
require 'lightio/watchers/timer'
|
3
|
+
require 'lightio/watchers/schedule'
|
4
|
+
require 'lightio/watchers/io'
|
5
|
+
|
6
|
+
# Watcher is a abstract struct, for libraries to interact with ioloop
|
7
|
+
# see IOloop#wait method
|
8
|
+
module LightIO::Watchers
|
9
|
+
end
|
@@ -0,0 +1,110 @@
|
|
1
|
+
module LightIO::Watchers
|
2
|
+
# LightIO::Watchers::IO provide a NIO::Monitor wrap to manage 'raw' socket / io
|
3
|
+
#
|
4
|
+
# @Example:
|
5
|
+
# #- wait_read for server socket
|
6
|
+
# io_watcher = LightIO::Watchers::IO.new(server_socket, :r)
|
7
|
+
# loop do
|
8
|
+
# io_watcher.wait_read
|
9
|
+
# client_socket = server_socket.accept
|
10
|
+
# # do something
|
11
|
+
# end
|
12
|
+
# io_watcher.close
|
13
|
+
class IO < Watcher
|
14
|
+
# Create a io watcher
|
15
|
+
# @param [Socket] io An IO-able object
|
16
|
+
# @param [Symbol] interests :r, :w, :rw - Is io readable? writeable? or both
|
17
|
+
# @return [LightIO::Watchers::IO]
|
18
|
+
def initialize(io, interests)
|
19
|
+
@io = io
|
20
|
+
@ioloop = LightIO::Core::IOloop.current
|
21
|
+
@waiting = false
|
22
|
+
@wait_for = nil
|
23
|
+
# NIO monitor
|
24
|
+
@monitor = @ioloop.add_io_wait(@io, interests) {callback_on_waiting}
|
25
|
+
end
|
26
|
+
|
27
|
+
def interests
|
28
|
+
@monitor.interests
|
29
|
+
end
|
30
|
+
|
31
|
+
# Replace current interests
|
32
|
+
def interests=(interests)
|
33
|
+
@monitor.interests = interests
|
34
|
+
end
|
35
|
+
|
36
|
+
# Blocking until io interests is satisfied
|
37
|
+
def wait_for(interests)
|
38
|
+
if (self.interests == :w || self.interests == :r) && interests != self.interests
|
39
|
+
raise ArgumentError, "IO interests is #{self.interests}, can't waiting for #{interests}"
|
40
|
+
end
|
41
|
+
@wait_for = interests
|
42
|
+
wait
|
43
|
+
end
|
44
|
+
|
45
|
+
# Blocking until io is readable
|
46
|
+
# @param [Numeric] timeout return nil after timeout seconds, otherwise return self
|
47
|
+
# @return [LightIO::Watchers::IO, nil]
|
48
|
+
def wait_read(timeout=nil)
|
49
|
+
LightIO::Timeout.timeout(timeout) do
|
50
|
+
wait_for :r
|
51
|
+
self
|
52
|
+
end
|
53
|
+
rescue Timeout::Error
|
54
|
+
nil
|
55
|
+
end
|
56
|
+
|
57
|
+
# Blocking until io is writeable
|
58
|
+
# @param [Numeric] timeout return nil after timeout seconds, otherwise return self
|
59
|
+
# @return [LightIO::Watchers::IO, nil]
|
60
|
+
def wait_write(timeout=nil)
|
61
|
+
LightIO::Timeout.timeout(timeout) do
|
62
|
+
wait_for :w
|
63
|
+
self
|
64
|
+
end
|
65
|
+
rescue Timeout::Error
|
66
|
+
nil
|
67
|
+
end
|
68
|
+
|
69
|
+
|
70
|
+
def start(ioloop)
|
71
|
+
# do nothing
|
72
|
+
end
|
73
|
+
|
74
|
+
# stop io listening
|
75
|
+
def close
|
76
|
+
@monitor.close
|
77
|
+
end
|
78
|
+
|
79
|
+
def close?
|
80
|
+
@monitor.close?
|
81
|
+
end
|
82
|
+
|
83
|
+
def wait
|
84
|
+
raise LightIO::Error, "Watchers::IO can't cross threads" if @ioloop != LightIO::Core::IOloop.current
|
85
|
+
raise EOFError, "can't wait closed IO watcher" if @monitor.closed?
|
86
|
+
@waiting = true
|
87
|
+
@ioloop.wait(self)
|
88
|
+
@waiting = false
|
89
|
+
end
|
90
|
+
|
91
|
+
def set_callback(&blk)
|
92
|
+
@callback = blk
|
93
|
+
end
|
94
|
+
|
95
|
+
private
|
96
|
+
|
97
|
+
def callback_on_waiting
|
98
|
+
# only call callback on waiting
|
99
|
+
callback.call if @waiting && io_is_ready?
|
100
|
+
end
|
101
|
+
|
102
|
+
def io_is_ready?
|
103
|
+
if @wait_for == :r
|
104
|
+
@monitor.readable?
|
105
|
+
elsif @wait_for == :w
|
106
|
+
@monitor.writeable?
|
107
|
+
end
|
108
|
+
end
|
109
|
+
end
|
110
|
+
end
|
data/lightio.gemspec
CHANGED
@@ -5,12 +5,12 @@ require "lightio/version"
|
|
5
5
|
|
6
6
|
Gem::Specification.new do |spec|
|
7
7
|
spec.name = "lightio"
|
8
|
-
spec.version =
|
8
|
+
spec.version = LightIO::VERSION
|
9
9
|
spec.authors = ["jjy"]
|
10
10
|
spec.email = ["jjyruby@gmail.com"]
|
11
11
|
|
12
|
-
spec.summary = %q{LightIO is a
|
13
|
-
spec.description = %q{LightIO
|
12
|
+
spec.summary = %q{LightIO is a high performance ruby networking library}
|
13
|
+
spec.description = %q{LightIO combines ruby fiber and IO event loop to provide both simple synchrony library interface and high performance networking IO}
|
14
14
|
spec.homepage = "https://github.com/jjyr/lightio"
|
15
15
|
spec.license = "MIT"
|
16
16
|
|
@@ -21,7 +21,7 @@ Gem::Specification.new do |spec|
|
|
21
21
|
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
22
22
|
spec.require_paths = ["lib"]
|
23
23
|
|
24
|
-
spec.add_runtime_dependency "nio4r"
|
24
|
+
spec.add_runtime_dependency "nio4r", "~> 2.1"
|
25
25
|
spec.add_development_dependency "bundler", "~> 1.16"
|
26
26
|
spec.add_development_dependency "rake", "~> 10.0"
|
27
27
|
spec.add_development_dependency "rspec", "~> 3.0"
|
metadata
CHANGED
@@ -1,29 +1,29 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: lightio
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.0
|
4
|
+
version: 0.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- jjy
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2017-
|
11
|
+
date: 2017-12-10 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: nio4r
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
16
16
|
requirements:
|
17
|
-
- - "
|
17
|
+
- - "~>"
|
18
18
|
- !ruby/object:Gem::Version
|
19
|
-
version: '
|
19
|
+
version: '2.1'
|
20
20
|
type: :runtime
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
|
-
- - "
|
24
|
+
- - "~>"
|
25
25
|
- !ruby/object:Gem::Version
|
26
|
-
version: '
|
26
|
+
version: '2.1'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
28
|
name: bundler
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
@@ -66,8 +66,8 @@ dependencies:
|
|
66
66
|
- - "~>"
|
67
67
|
- !ruby/object:Gem::Version
|
68
68
|
version: '3.0'
|
69
|
-
description: LightIO
|
70
|
-
|
69
|
+
description: LightIO combines ruby fiber and IO event loop to provide both simple
|
70
|
+
synchrony library interface and high performance networking IO
|
71
71
|
email:
|
72
72
|
- jjyruby@gmail.com
|
73
73
|
executables: []
|
@@ -85,14 +85,26 @@ files:
|
|
85
85
|
- Rakefile
|
86
86
|
- bin/console
|
87
87
|
- bin/setup
|
88
|
+
- examples/beams.rb
|
89
|
+
- examples/echo_server_with_raw_socket.rb
|
88
90
|
- lib/lightio.rb
|
89
|
-
- lib/lightio/
|
90
|
-
- lib/lightio/
|
91
|
-
- lib/lightio/
|
92
|
-
- lib/lightio/
|
93
|
-
- lib/lightio/
|
91
|
+
- lib/lightio/core.rb
|
92
|
+
- lib/lightio/core/backend/nio.rb
|
93
|
+
- lib/lightio/core/beam.rb
|
94
|
+
- lib/lightio/core/future.rb
|
95
|
+
- lib/lightio/core/ioloop.rb
|
96
|
+
- lib/lightio/core/light_fiber.rb
|
97
|
+
- lib/lightio/errors.rb
|
98
|
+
- lib/lightio/library.rb
|
99
|
+
- lib/lightio/library/kernel_ext.rb
|
100
|
+
- lib/lightio/library/queue.rb
|
101
|
+
- lib/lightio/library/timeout.rb
|
94
102
|
- lib/lightio/version.rb
|
95
|
-
- lib/lightio/
|
103
|
+
- lib/lightio/watchers.rb
|
104
|
+
- lib/lightio/watchers/io.rb
|
105
|
+
- lib/lightio/watchers/schedule.rb
|
106
|
+
- lib/lightio/watchers/timer.rb
|
107
|
+
- lib/lightio/watchers/watcher.rb
|
96
108
|
- lightio.gemspec
|
97
109
|
homepage: https://github.com/jjyr/lightio
|
98
110
|
licenses:
|
@@ -109,13 +121,13 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
109
121
|
version: '0'
|
110
122
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
111
123
|
requirements:
|
112
|
-
- - "
|
124
|
+
- - ">="
|
113
125
|
- !ruby/object:Gem::Version
|
114
|
-
version:
|
126
|
+
version: '0'
|
115
127
|
requirements: []
|
116
128
|
rubyforge_project:
|
117
129
|
rubygems_version: 2.6.11
|
118
130
|
signing_key:
|
119
131
|
specification_version: 4
|
120
|
-
summary: LightIO is a
|
132
|
+
summary: LightIO is a high performance ruby networking library
|
121
133
|
test_files: []
|
data/lib/lightio/beam.rb
DELETED
data/lib/lightio/future.rb
DELETED
@@ -1,35 +0,0 @@
|
|
1
|
-
# Future, provide another way to operate Beam and Ioloop
|
2
|
-
|
3
|
-
module LightIO
|
4
|
-
class Future
|
5
|
-
def initialize
|
6
|
-
@value = nil
|
7
|
-
@ioloop = IOloop.current
|
8
|
-
@state = :init
|
9
|
-
end
|
10
|
-
|
11
|
-
def done?
|
12
|
-
@state == :done
|
13
|
-
end
|
14
|
-
|
15
|
-
def done!
|
16
|
-
@state = :done
|
17
|
-
end
|
18
|
-
|
19
|
-
# transfer and set value
|
20
|
-
def transfer(value=nil)
|
21
|
-
raise Error, "state error" if done?
|
22
|
-
@value = value
|
23
|
-
done!
|
24
|
-
@beam.transfer if @beam
|
25
|
-
end
|
26
|
-
|
27
|
-
# block current beam/fiber and get value
|
28
|
-
def value
|
29
|
-
return @value if done?
|
30
|
-
@beam = Beam.current
|
31
|
-
@ioloop.transfer
|
32
|
-
@value
|
33
|
-
end
|
34
|
-
end
|
35
|
-
end
|
data/lib/lightio/ioloop.rb
DELETED
@@ -1,40 +0,0 @@
|
|
1
|
-
# IOloop like a per-threaded Eventmachine (cause fiber cannot resume cross threads)
|
2
|
-
# join, 等待 IOloop 结束
|
3
|
-
# 在 loop 中为 waiter (future-like proxy object) 对象返回结果
|
4
|
-
module LightIO
|
5
|
-
class IOloop
|
6
|
-
def initialize
|
7
|
-
@fiber = Fiber.new {run}
|
8
|
-
@backend = Backend::NIO.new
|
9
|
-
end
|
10
|
-
|
11
|
-
# should never invoke explicitly
|
12
|
-
def run
|
13
|
-
# start io loop and never return...
|
14
|
-
@backend.run
|
15
|
-
end
|
16
|
-
|
17
|
-
# wait a watcher, maybe a timer or socket
|
18
|
-
def wait(watcher)
|
19
|
-
future = Future.new
|
20
|
-
# add watcher to loop
|
21
|
-
watcher.set_callback {future.transfer}
|
22
|
-
watcher.register(@backend)
|
23
|
-
# trigger a fiber switch
|
24
|
-
# wait until watcher is ok
|
25
|
-
# then do work
|
26
|
-
future.value
|
27
|
-
end
|
28
|
-
|
29
|
-
def transfer
|
30
|
-
@fiber.transfer
|
31
|
-
end
|
32
|
-
|
33
|
-
class << self
|
34
|
-
# return current ioloop or create new one
|
35
|
-
def current
|
36
|
-
Thread.current[:"lightio.ioloop"] ||= IOloop.new
|
37
|
-
end
|
38
|
-
end
|
39
|
-
end
|
40
|
-
end
|
data/lib/lightio/timer.rb
DELETED