deadly_serious 0.4.0 → 0.4.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +5 -5
- data/lib/deadly_serious/engine/spawner.rb +5 -4
- data/lib/deadly_serious/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 775816edf4860d7516908442da4d8ffaa2793aba
|
4
|
+
data.tar.gz: 0516bbdbccd02bf0102833c1dfcbe2ecc1bbac9f
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 9ddb27632bffcbba2bd7c9f008e0662b27c5dcdcd048ae244cffbe577f7afdcfb19c5c7723891dde7f221f9cf81ccd7681d0aec0ebc9e62d6f25c7cf3820ec7a
|
7
|
+
data.tar.gz: d49e4442d46a296c638d84e6120ae83d25f174d55e353c34e2a63a9741a5116b49f738be2a8642356f84932fa16872884b0a01ec66ca233013a62173adecd609
|
data/README.md
CHANGED
@@ -11,14 +11,14 @@ Unlike [NoFlo](http://noflojs.org), this is not a real engine. It just "orchestr
|
|
11
11
|
Overall, it's slower than a normal ruby program (the pipes add some overhead). However, there are 4 points where this approach is pretty interesting:
|
12
12
|
|
13
13
|
1. High modifiabilty:
|
14
|
-
* The
|
15
|
-
* You can connect ruby process to anything
|
14
|
+
* The interface between each component is tiny and very clear: it's just a stream of characteres. I usually use csv format or json when I need more structure than that.
|
15
|
+
* You can connect ruby process to anything that deals with STDIN, STDOUT or files (which includes shell commands, of course).
|
16
16
|
2. Cheap parallelism and distributed computation:
|
17
|
-
* Each component runs as a separated process. The OS is in charge here (and it does
|
18
|
-
* As any shell command can be
|
17
|
+
* Each component runs as a separated process. The OS is in charge here (and it does an amazing work running things in parallel).
|
18
|
+
* As any shell command can be used as a component, you can use a simple [ncat](http://nmap.org/ncat) (or something similar) to distribute jobs between different boxes.
|
19
19
|
* It's really easy to avoid deadlocks and race conditions with the FBP paradigm.
|
20
20
|
3. Low memory footprint
|
21
|
-
* As each component usually process things as they appear in the pipe, it's easy to crush tons of data
|
21
|
+
* As each component usually process things as they appear in the pipe, it's easy to crush tons of data with very low memory. Notable exceptions as components that needs to accumulate things to process, like "sort".
|
22
22
|
4. Very easy to reason about (personal opinion):
|
23
23
|
* Of course, this is not a merit of this gem, but of Flow Based Programming in general. I dare do say (oh, blasphemy!) that Object Oriented and Functional programming paradigms are good ONLY for tiny systems. They make a huge mess on big ones (#prontofalei).
|
24
24
|
|
@@ -7,13 +7,10 @@ module DeadlySerious
|
|
7
7
|
def initialize(data_dir: './data', pipe_dir: "/tmp/deadly_serious/#{Process.pid}", preserve_pipe_dir: false)
|
8
8
|
@data_dir = data_dir
|
9
9
|
@pipe_dir = pipe_dir
|
10
|
+
@preserve_pipe_dir = preserve_pipe_dir
|
10
11
|
@ids = []
|
11
12
|
|
12
13
|
FileUtils.mkdir_p(pipe_dir) unless File.exist?(pipe_dir)
|
13
|
-
|
14
|
-
unless preserve_pipe_dir
|
15
|
-
at_exit { FileUtils.rm_r(pipe_dir, force: true, secure: true) }
|
16
|
-
end
|
17
14
|
end
|
18
15
|
|
19
16
|
def run
|
@@ -22,6 +19,10 @@ module DeadlySerious
|
|
22
19
|
rescue Exception => e
|
23
20
|
kill_children
|
24
21
|
raise e
|
22
|
+
ensure
|
23
|
+
if !@preserve_pipe_dir && File.exist?(@pipe_dir)
|
24
|
+
FileUtils.rm_r(@pipe_dir, force: true, secure: true)
|
25
|
+
end
|
25
26
|
end
|
26
27
|
|
27
28
|
def spawn_source(a_class, *args, writer: self.class.dasherize(a_class.name))
|