org-converge 0.0.7 → 0.0.8
Sign up to get free protection for your applications and to get access to all the features.
- data/README.org +91 -86
- data/TODO +3 -3
- data/bin/org-spec +29 -0
- data/lib/org-converge/babel_output_buffer.rb +13 -5
- data/lib/org-converge/command.rb +152 -15
- data/lib/org-converge/engine.rb +28 -7
- data/lib/org-converge/version.rb +1 -1
- data/org-converge.gemspec +1 -1
- data/spec/converge_examples/expected_results/spec.org +62 -0
- data/spec/converge_examples/expected_results/spec2.org +62 -0
- data/spec/converge_spec.rb +26 -0
- metadata +19 -13
data/README.org
CHANGED
@@ -2,45 +2,64 @@
|
|
2
2
|
#+STARTUP: showeverything
|
3
3
|
|
4
4
|
* Org Converge
|
5
|
+
|
6
|
+
[[https://secure.travis-ci.org/wallyqs/org-converge.png?branch=master]]
|
7
|
+
|
5
8
|
** Description
|
6
9
|
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
10
|
+
This attempts to be an experiment of using Org mode syntax to
|
11
|
+
create documentable reproducible runs, borrowing some ideas
|
12
|
+
of what is possible to do with tools like =chef-solo=,
|
13
|
+
=rake=, =foreman=, etc...
|
14
|
+
|
15
|
+
** Installing
|
16
|
+
|
17
|
+
: gem install org-converge
|
11
18
|
|
12
19
|
** Motivation
|
13
20
|
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
21
|
+
The Org babel syntax has proven to be flexible enough to writing
|
22
|
+
/reproducible research/ papers. Then, I believe that configuring and setting up
|
23
|
+
a server for example is something that could be also be done using
|
24
|
+
the same syntax, given that /converging/ the configuration is something
|
25
|
+
that one ought to be able to reproduce.
|
19
26
|
|
20
27
|
** Usage
|
21
28
|
|
22
|
-
|
29
|
+
To run the blocks in parallel...
|
23
30
|
|
24
31
|
#+begin_src sh
|
25
|
-
org-converge path/to/setup-file.org
|
32
|
+
org-converge path/to/setup-file.org --runmode=parallel
|
26
33
|
#+end_src
|
27
34
|
|
28
|
-
|
35
|
+
or sequentially...
|
29
36
|
|
30
37
|
#+begin_src sh
|
31
38
|
org-converge path/to/setup-file.org --runmode=sequential
|
32
39
|
#+end_src
|
33
40
|
|
41
|
+
By including ~:after~ or ~:before~ arguments to a block,
|
42
|
+
it is possible to make the blocks depend on one another.
|
43
|
+
|
44
|
+
#+begin_src sh
|
45
|
+
org-converge path/to/setup-file.org --runmode=chained
|
46
|
+
#+end_src
|
47
|
+
|
48
|
+
*** Other commands available
|
49
|
+
|
50
|
+
: org-run # alias for org-converge
|
51
|
+
: org-tangle # just tangles the contents without running the blocks
|
52
|
+
|
34
53
|
** Quick example
|
35
54
|
|
36
|
-
|
37
|
-
|
38
|
-
|
55
|
+
The following is an example of running 3 processes
|
56
|
+
in parallel by defining them as code blocks from
|
57
|
+
an Org mode file:
|
39
58
|
|
40
59
|
#+begin_src sh
|
41
60
|
,#+TITLE: Running Org babel processes in parallel
|
42
61
|
|
43
|
-
|
62
|
+
,* Print with different languages
|
44
63
|
|
45
64
|
,#+name: hello_from_bash
|
46
65
|
,#+begin_src sh :shebang #!/bin/bash
|
@@ -64,96 +83,82 @@ org-converge path/to/setup-file.org --runmode=sequential
|
|
64
83
|
,#+end_src
|
65
84
|
#+end_src
|
66
85
|
|
67
|
-
|
86
|
+
We store this in a file named =hello.org= and then run it as follows:
|
68
87
|
|
69
88
|
#+begin_src sh
|
70
89
|
org-converge hello.org
|
71
90
|
#+end_src
|
72
91
|
|
73
|
-
|
92
|
+
This would produce an output similar to the following:
|
74
93
|
|
75
94
|
#+begin_src sh
|
76
|
-
|
77
|
-
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
|
84
|
-
|
85
|
-
|
86
|
-
|
87
|
-
I, [2014-04-17T23:48:45.792321 #1710] INFO -- : hello_from_ruby (1712) -- hello world from ruby
|
88
|
-
I, [2014-04-17T23:48:45.882020 #1710] INFO -- : hello_from_python (1713) -- hello world from python
|
89
|
-
I, [2014-04-17T23:48:46.787106 #1710] INFO -- : hello_from_bash (1711) -- hello world from bash
|
90
|
-
I, [2014-04-17T23:48:46.793347 #1710] INFO -- : hello_from_ruby (1712) -- hello world from ruby
|
91
|
-
I, [2014-04-17T23:48:46.883511 #1710] INFO -- : hello_from_python (1713) -- hello world from python
|
92
|
-
I, [2014-04-17T23:48:47.794846 #1710] INFO -- : hello_from_ruby (1712) -- hello world from ruby
|
93
|
-
I, [2014-04-17T23:48:47.796387 #1710] INFO -- : hello_from_bash (1711) -- hello world from bash
|
94
|
-
I, [2014-04-17T23:48:47.884196 #1710] INFO -- : hello_from_python (1713) -- hello world from python
|
95
|
+
[2014-05-04T19:23:40 +0900] Tangling 0 files...
|
96
|
+
[2014-05-04T19:23:40 +0900] Tangling succeeded!
|
97
|
+
[2014-05-04T19:23:40 +0900] Tangling 3 scripts within directory: /Users/mariko/repos/org-converge/run...
|
98
|
+
[2014-05-04T19:23:40 +0900] Running code blocks now! (3 runnable blocks found in total)
|
99
|
+
[2014-05-04T19:23:40 +0900] hello_from_bash (4664) -- started with pid 4664
|
100
|
+
[2014-05-04T19:23:40 +0900] hello_from_ruby (4665) -- started with pid 4665
|
101
|
+
[2014-05-04T19:23:40 +0900] hello_from_python (4666) -- started with pid 4666
|
102
|
+
[2014-05-04T19:23:40 +0900] hello_from_bash (4664) -- hello world from bash
|
103
|
+
[2014-05-04T19:23:41 +0900] hello_from_ruby (4665) -- hello world from ruby
|
104
|
+
[2014-05-04T19:23:41 +0900] hello_from_python (4666) -- hello world from python
|
105
|
+
[2014-05-04T19:23:42 +0900] hello_from_ruby (4665) -- hello world from ruby
|
95
106
|
#+end_src
|
96
107
|
|
97
108
|
** How it works
|
98
109
|
|
99
|
-
|
100
|
-
|
101
|
-
|
110
|
+
Org Converge uses an liberally extended version of Org Babel
|
111
|
+
features in order to give support for converging the configuration
|
112
|
+
of a server.
|
113
|
+
|
114
|
+
For example, using Org Babel and macros we can easily spread config
|
115
|
+
files on a server by writing the following on a ~server.org~ file.
|
116
|
+
|
117
|
+
#+begin_src sh
|
118
|
+
,#+begin_src yaml :tangle /etc/component.yml
|
119
|
+
multitenant: false
|
120
|
+
status_port: 10004
|
121
|
+
,#+end_src
|
122
|
+
#+end_src
|
123
|
+
|
124
|
+
And then configure it by running it as follows, (considering we have
|
125
|
+
the correct permissions for tangling at =/etc/component.yml=):
|
102
126
|
|
103
|
-
|
104
|
-
|
127
|
+
#+begin_src sh
|
128
|
+
sudo org-converge server.org
|
129
|
+
#+end_src
|
105
130
|
|
106
|
-
|
107
|
-
|
108
|
-
|
109
|
-
status_port: 10004
|
110
|
-
,#+end_src
|
111
|
-
#+end_src
|
131
|
+
Next, let's say that we no only one want to set the configured templates,
|
132
|
+
but that we also want to install some packages. In that case, we
|
133
|
+
should be able to do the following:
|
112
134
|
|
113
|
-
|
114
|
-
|
135
|
+
#+begin_src sh
|
136
|
+
,* Configuring the component
|
115
137
|
|
116
|
-
|
117
|
-
|
118
|
-
|
138
|
+
,#+begin_src yaml :tangle /etc/component.yml
|
139
|
+
multitenant: false
|
140
|
+
status_port: 10004
|
141
|
+
,#+end_src
|
119
142
|
|
120
|
-
|
121
|
-
but that we also want to install some packages. In that case, we
|
122
|
-
should be able to do the following:
|
143
|
+
,* Installing the dependencies
|
123
144
|
|
124
|
-
|
125
|
-
|
145
|
+
Need the following so that ~bundle install~ can compile
|
146
|
+
the native extensions correctly.
|
126
147
|
|
127
|
-
|
128
|
-
|
129
|
-
|
130
|
-
,#+end_src
|
131
|
-
|
132
|
-
,* Installing the dependencies
|
148
|
+
,#+begin_src sh
|
149
|
+
apt-get install build-essentials -y
|
150
|
+
,#+end_src
|
133
151
|
|
134
|
-
|
135
|
-
the native extensions correctly.
|
152
|
+
Then the following should work:
|
136
153
|
|
137
|
-
|
138
|
-
|
139
|
-
|
140
|
-
|
141
|
-
|
142
|
-
|
143
|
-
,#+begin_src sh
|
144
|
-
cd project_path
|
145
|
-
bundle install
|
146
|
-
,#+end_src
|
147
|
-
#+end_src
|
148
|
-
|
149
|
-
As long as the repo has been already checked out in the directory,
|
150
|
-
the previous example will succeed.
|
151
|
-
|
152
|
-
More practical examples can be found [[https://github.com/wallyqs/org-converge/tree/master/examples][here]], more will be added as
|
153
|
-
long as dogfooding from this goes well.
|
154
|
+
,#+begin_src sh
|
155
|
+
cd project_path
|
156
|
+
bundle install
|
157
|
+
,#+end_src
|
158
|
+
#+end_src
|
154
159
|
|
155
160
|
** Contributing
|
156
161
|
|
157
|
-
|
158
|
-
|
159
|
-
|
162
|
+
The project is in very early development at this moment, but if you
|
163
|
+
feel that it is interesting enough, please create a ticket to start
|
164
|
+
the discussion.
|
data/TODO
CHANGED
@@ -5,7 +5,7 @@
|
|
5
5
|
* [0/7] > 0.0.9 version
|
6
6
|
|
7
7
|
- [ ] Macros can be loaded and applied to the configuration
|
8
|
-
- [ ] Actually support converging and idempotency
|
8
|
+
- [ ] Actually support converging and idempotency (~--runmode=idempotent~)
|
9
9
|
+ Do not do an operation unless it is required
|
10
10
|
+ Abort in case there was a failure in executing the script.
|
11
11
|
+ This would work by using the ~:cache true~ to a block header argument
|
@@ -16,9 +16,9 @@
|
|
16
16
|
- [ ] Use :eval for evaling blocks (off by default)
|
17
17
|
- [ ] Can use :dir for running a process relative to that directory
|
18
18
|
|
19
|
-
* [
|
19
|
+
* [1/1] 0.0.8 version
|
20
20
|
|
21
|
-
- [
|
21
|
+
- [X] ~org-spec~ exploration!
|
22
22
|
|
23
23
|
* [2/2] 0.0.7 version
|
24
24
|
|
data/bin/org-spec
ADDED
@@ -0,0 +1,29 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
# -*- mode: ruby -*-
|
3
|
+
require 'docopt'
|
4
|
+
require 'org-converge'
|
5
|
+
|
6
|
+
doc = <<OPTIONS
|
7
|
+
|
8
|
+
org-spec: Validate whether code blocks match to the contents of the #+RESULTS block
|
9
|
+
|
10
|
+
Usage:
|
11
|
+
org-spec <org_file> [--showfiles] [--log=<logfile>] [--root-dir=<root_dir>] [--name=<block_name>]
|
12
|
+
|
13
|
+
Options:
|
14
|
+
|
15
|
+
-h --help Show this screen.
|
16
|
+
|
17
|
+
OPTIONS
|
18
|
+
|
19
|
+
begin
|
20
|
+
require "pp"
|
21
|
+
cmd = Docopt::docopt(doc)
|
22
|
+
rescue Docopt::Exit => e
|
23
|
+
puts e.message
|
24
|
+
end
|
25
|
+
|
26
|
+
exit 1 unless cmd
|
27
|
+
|
28
|
+
o = OrgConverge::Command.new(cmd.merge({ '--runmode' => 'spec' }))
|
29
|
+
o.execute!
|
@@ -23,9 +23,9 @@ module Orgmode
|
|
23
23
|
# ~@scripts~ are tangled in order and ran
|
24
24
|
# : @scripts = [text, text, ...]
|
25
25
|
@scripts = Hash.new {|h,k| h[k] = {
|
26
|
-
:lines
|
27
|
-
:header
|
28
|
-
:lang
|
26
|
+
:lines => '',
|
27
|
+
:header => {},
|
28
|
+
:lang => ''
|
29
29
|
}
|
30
30
|
}
|
31
31
|
@scripts_counter = 0
|
@@ -77,10 +77,13 @@ module Orgmode
|
|
77
77
|
end
|
78
78
|
|
79
79
|
case
|
80
|
+
when (line.start_of_results_code_block?)
|
81
|
+
@accumulate_results_block = true
|
80
82
|
when (line.assigned_paragraph_type == :code and @current_tangle)
|
81
83
|
# Need to keep track of the current tangle to buffer its lines
|
82
84
|
@tangle[@current_tangle][:lines] << line.output_text << "\n"
|
83
|
-
when (line.assigned_paragraph_type == :code
|
85
|
+
when (line.assigned_paragraph_type == :code or \
|
86
|
+
line.paragraph_type == :inline_example)
|
84
87
|
# When a tangle is not going on, it means that the lines would go
|
85
88
|
# into a runnable script
|
86
89
|
@buffer << line.output_text << "\n"
|
@@ -88,7 +91,12 @@ module Orgmode
|
|
88
91
|
line.assigned_paragraph_type == :code))
|
89
92
|
# Fix indentation and remove fix commas from Org mode before flushing
|
90
93
|
strip_code_block!
|
91
|
-
|
94
|
+
if @accumulate_results_block
|
95
|
+
@scripts[@scripts_counter - 1][:results] = @buffer
|
96
|
+
@accumulate_results_block = false
|
97
|
+
else
|
98
|
+
@scripts[@scripts_counter][:lines] << @buffer
|
99
|
+
end
|
92
100
|
@buffer = ''
|
93
101
|
@scripts_counter += 1
|
94
102
|
end
|
data/lib/org-converge/command.rb
CHANGED
@@ -6,14 +6,21 @@ module OrgConverge
|
|
6
6
|
attr_reader :engine
|
7
7
|
|
8
8
|
def initialize(options)
|
9
|
-
@options
|
10
|
-
@dotorg
|
11
|
-
@root_dir
|
12
|
-
@run_dir
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
9
|
+
@options = options
|
10
|
+
@dotorg = options['<org_file>']
|
11
|
+
@root_dir = options['--root-dir']
|
12
|
+
@run_dir = if @root_dir
|
13
|
+
File.expand_path(File.join(@root_dir, 'run'))
|
14
|
+
else
|
15
|
+
File.expand_path('run')
|
16
|
+
end
|
17
|
+
# The results dir will have a timestamp to avoid having to refresh all the time
|
18
|
+
results_dirname = "results_#{Time.now.strftime("%Y%m%d%H%M%S")}"
|
19
|
+
@results_dir = if @root_dir
|
20
|
+
File.expand_path(File.join(@root_dir, results_dirname))
|
21
|
+
else
|
22
|
+
File.expand_path(results_dirname)
|
23
|
+
end
|
17
24
|
@ob = Orgmode::Parser.new(File.read(dotorg)).babelize
|
18
25
|
@babel = nil
|
19
26
|
@logger = Logger.new(options['--log'] || STDOUT)
|
@@ -40,14 +47,15 @@ module OrgConverge
|
|
40
47
|
|
41
48
|
def converge!
|
42
49
|
tangle!
|
50
|
+
runmode = @options['--runmode'] || ob.in_buffer_settings['RUNMODE']
|
43
51
|
case
|
44
|
-
when @options['--runmode']
|
45
|
-
dispatch_runmode(@options['--runmode'])
|
46
52
|
when @options['--name']
|
47
|
-
|
53
|
+
if runmode == 'sequentially'
|
54
|
+
run_matching_blocks_sequentially!
|
55
|
+
else
|
56
|
+
run_matching_blocks!
|
57
|
+
end
|
48
58
|
else
|
49
|
-
# Try to find one in the buffer
|
50
|
-
runmode = ob.in_buffer_settings['RUNMODE']
|
51
59
|
dispatch_runmode(runmode)
|
52
60
|
end
|
53
61
|
end
|
@@ -60,6 +68,8 @@ module OrgConverge
|
|
60
68
|
run_blocks_sequentially!
|
61
69
|
when 'chained', 'chain', 'tasks'
|
62
70
|
run_blocks_chain!
|
71
|
+
when 'spec'
|
72
|
+
run_against_blocks_results!
|
63
73
|
else # parallel by default
|
64
74
|
run_blocks_in_parallel!
|
65
75
|
end
|
@@ -177,8 +187,135 @@ module OrgConverge
|
|
177
187
|
logger.info "Run has completed successfully.".fg 'green'
|
178
188
|
end
|
179
189
|
|
180
|
-
def
|
181
|
-
|
190
|
+
def run_matching_blocks_sequentially!
|
191
|
+
babel.tangle_runnable_blocks!(@run_dir)
|
192
|
+
|
193
|
+
runlist_stack = []
|
194
|
+
scripts = babel.ob.scripts.select {|k, h| h[:header][:name] =~ Regexp.new(@options['--name']) }
|
195
|
+
scripts.each do |key, script|
|
196
|
+
runlist_stack << [key, script]
|
197
|
+
end
|
198
|
+
|
199
|
+
while not runlist_stack.empty?
|
200
|
+
key, script = runlist_stack.shift
|
201
|
+
|
202
|
+
# Decision: Only run blocks which have a name
|
203
|
+
next unless script[:header][:name]
|
204
|
+
|
205
|
+
display_name = script[:header][:name]
|
206
|
+
with_running_engine do |engine|
|
207
|
+
file = File.expand_path("#{@run_dir}/#{key}")
|
208
|
+
cmd = "#{script[:lang]} #{file}"
|
209
|
+
engine.register display_name, cmd, { :cwd => @root_dir, :logger => logger }
|
210
|
+
end
|
211
|
+
end
|
212
|
+
logger.info "Run has completed successfully.".fg 'green'
|
213
|
+
end
|
214
|
+
|
215
|
+
def run_against_blocks_results!
|
216
|
+
require 'diff/lcs'
|
217
|
+
require 'diff/lcs/hunk'
|
218
|
+
|
219
|
+
succeeded = []
|
220
|
+
failed = []
|
221
|
+
|
222
|
+
logger.info "Runmode: spec"
|
223
|
+
runlist_stack = []
|
224
|
+
scripts = if @options['--name']
|
225
|
+
babel.ob.scripts.select {|k, h| h[:header][:name] =~ Regexp.new(@options['--name']) }
|
226
|
+
else
|
227
|
+
babel.ob.scripts
|
228
|
+
end
|
229
|
+
scripts.each { |key, script| runlist_stack << [key, script] }
|
230
|
+
|
231
|
+
babel.tangle_runnable_blocks!(@run_dir)
|
232
|
+
FileUtils.mkdir_p(@results_dir)
|
233
|
+
|
234
|
+
while not runlist_stack.empty?
|
235
|
+
key, script = runlist_stack.shift
|
236
|
+
|
237
|
+
# Decision: Only run blocks which have a name
|
238
|
+
next unless script[:header][:name]
|
239
|
+
|
240
|
+
display_name = script[:header][:name]
|
241
|
+
script_file = File.expand_path("#{@run_dir}/#{key}")
|
242
|
+
results_file = File.expand_path("#{@results_dir}/#{key}")
|
243
|
+
cmd = "#{script[:lang]} #{script_file}"
|
244
|
+
|
245
|
+
with_running_engine(:runmode => 'spec', :results_dir => @results_dir) \
|
246
|
+
do |engine|
|
247
|
+
engine.register display_name, cmd, {
|
248
|
+
:cwd => @root_dir,
|
249
|
+
:logger => logger,
|
250
|
+
:results => results_file
|
251
|
+
}
|
252
|
+
end
|
253
|
+
|
254
|
+
if scripts[:results]
|
255
|
+
print "Checking results from '#{display_name.fg 'yellow'}' code block:\t"
|
256
|
+
expected_lines = script[:results].split("\n").map! {|e| e.chomp }
|
257
|
+
actual_lines = File.open(results_file).read.split("\n").map! {|e| e.chomp }
|
258
|
+
|
259
|
+
output_diff = diff(expected_lines, actual_lines)
|
260
|
+
if output_diff.empty?
|
261
|
+
puts "OK".fg 'green'
|
262
|
+
succeeded << display_name
|
263
|
+
else
|
264
|
+
puts "DIFF".fg 'red'
|
265
|
+
puts output_diff.fg 'red'
|
266
|
+
failed << display_name
|
267
|
+
end
|
268
|
+
end
|
269
|
+
end
|
270
|
+
|
271
|
+
if failed.count > 0
|
272
|
+
puts ''
|
273
|
+
puts 'Failed code blocks:'.fg 'red'
|
274
|
+
failed.each do |name|
|
275
|
+
puts " - #{name.fg 'yellow'}"
|
276
|
+
end
|
277
|
+
puts ''
|
278
|
+
end
|
279
|
+
|
280
|
+
puts "#{succeeded.count + failed.count} examples, #{failed.count} failures".fg 'green'
|
281
|
+
exit 1 if failed.count > 0
|
282
|
+
end
|
283
|
+
|
284
|
+
def diff(expected_lines, actual_lines)
|
285
|
+
output = ""
|
286
|
+
file_length_difference = 0
|
287
|
+
|
288
|
+
diffs = Diff::LCS.diff(expected_lines, actual_lines)
|
289
|
+
hunks = diffs.map do |piece|
|
290
|
+
Diff::LCS::Hunk.new(
|
291
|
+
expected_lines, actual_lines, piece, 3, 0
|
292
|
+
).tap do |h|
|
293
|
+
file_length_difference = h.file_length_difference
|
294
|
+
end
|
295
|
+
end
|
296
|
+
|
297
|
+
hunks.each_cons(2) do |prev_hunk, current_hunk|
|
298
|
+
begin
|
299
|
+
if current_hunk.overlaps?(prev_hunk)
|
300
|
+
current_hunk.merge(prev_hunk)
|
301
|
+
else
|
302
|
+
output << prev_hunk.diff(:unified).to_s
|
303
|
+
end
|
304
|
+
rescue => e
|
305
|
+
end
|
306
|
+
end
|
307
|
+
|
308
|
+
if hunks.last
|
309
|
+
output << hunks.last.diff(:unified).to_s
|
310
|
+
end
|
311
|
+
|
312
|
+
output
|
313
|
+
end
|
314
|
+
|
315
|
+
def with_running_engine(opts={})
|
316
|
+
default_options = { :logger => @logger, :babel => @babel }
|
317
|
+
options = default_options.merge!(opts)
|
318
|
+
engine = OrgConverge::Engine.new(options)
|
182
319
|
yield engine
|
183
320
|
engine.start
|
184
321
|
end
|
data/lib/org-converge/engine.rb
CHANGED
@@ -16,8 +16,9 @@ module OrgConverge
|
|
16
16
|
|
17
17
|
def initialize(options={})
|
18
18
|
super(options)
|
19
|
-
@logger
|
20
|
-
@babel
|
19
|
+
@logger = options[:logger] || Logger.new(STDOUT)
|
20
|
+
@babel = options[:babel]
|
21
|
+
@runmode = options[:runmode]
|
21
22
|
end
|
22
23
|
|
23
24
|
# We allow other processes to exit with 0 status
|
@@ -37,11 +38,16 @@ module OrgConverge
|
|
37
38
|
@processes.each do |process|
|
38
39
|
reader, writer = create_pipe
|
39
40
|
begin
|
41
|
+
# In case of spec mode, we need to redirect the output to a results file instead
|
42
|
+
writer = File.open(process.options[:results], 'a') if @runmode == 'spec'
|
40
43
|
pid = process.run(:output => writer)
|
41
44
|
@names[process] = "#{@names[process]}.#{pid}"
|
42
|
-
|
45
|
+
|
46
|
+
# NOTE: In spec mode we need to be more strict on what is flushed by the engine
|
47
|
+
# because we will be comparing the output
|
48
|
+
writer.puts "started with pid #{pid}" unless @runmode == 'spec'
|
43
49
|
rescue Errno::ENOENT
|
44
|
-
writer.puts "unknown command: #{process.command}"
|
50
|
+
writer.puts "unknown command: #{process.command}" unless @runmode == 'spec'
|
45
51
|
end
|
46
52
|
@running[pid] = [process]
|
47
53
|
@readers[pid] = reader
|
@@ -63,8 +69,9 @@ module OrgConverge
|
|
63
69
|
output = "#{pad_process_name(ps)}(#{pid})".fg get_color_for_pid(pid.to_i)
|
64
70
|
output += " -- "
|
65
71
|
output += message
|
66
|
-
|
67
|
-
# the name of the process does not appear
|
72
|
+
|
73
|
+
# FIXME: When the process has stopped already, the name of the process does not appear
|
74
|
+
# (which means that this approach is wrong from the beginning probably)
|
68
75
|
logger.info output
|
69
76
|
end
|
70
77
|
rescue Errno::EPIPE
|
@@ -86,7 +93,21 @@ module OrgConverge
|
|
86
93
|
def get_color_for_pid(pid)
|
87
94
|
RAINBOW[pid % 7]
|
88
95
|
end
|
96
|
+
|
97
|
+
def watch_for_termination
|
98
|
+
pid, status = Process.wait2
|
99
|
+
output_with_mutex name_for(pid), termination_message_for(status) unless @runmode == 'spec'
|
100
|
+
@running.delete(pid)
|
101
|
+
yield if block_given?
|
102
|
+
pid
|
103
|
+
rescue Errno::ECHILD
|
104
|
+
end
|
89
105
|
end
|
90
106
|
|
91
|
-
|
107
|
+
# Need to expose the options to make the process be aware
|
108
|
+
# of the possible running mode (specially spec mode)
|
109
|
+
# and where to put the results output
|
110
|
+
class CodeBlockProcess < Foreman::Process
|
111
|
+
attr_reader :options
|
112
|
+
end
|
92
113
|
end
|
data/lib/org-converge/version.rb
CHANGED
data/org-converge.gemspec
CHANGED
@@ -15,7 +15,7 @@ Gem::Specification.new do |gem|
|
|
15
15
|
gem.require_paths = ["lib"]
|
16
16
|
gem.version = OrgConverge::VERSION
|
17
17
|
gem.add_runtime_dependency('docopt', '~> 0.5.0')
|
18
|
-
gem.add_runtime_dependency('org-ruby', '~> 0.9.
|
18
|
+
gem.add_runtime_dependency('org-ruby', '~> 0.9.6')
|
19
19
|
gem.add_runtime_dependency('foreman', '~> 0.63.0')
|
20
20
|
gem.add_runtime_dependency('tco', '~> 0.1.0')
|
21
21
|
gem.add_runtime_dependency('rake', '~> 10.3')
|
@@ -0,0 +1,62 @@
|
|
1
|
+
# -*- mode: org; mode: auto-fill; -*-
|
2
|
+
#+TITLE: Expected results example
|
3
|
+
#+runmode: spec
|
4
|
+
|
5
|
+
Given a literate description of the run along with the expected
|
6
|
+
results, we can try checking whether the contents from the code block
|
7
|
+
are evaling into what is being documented in the results block.
|
8
|
+
|
9
|
+
#+name: hello
|
10
|
+
#+begin_src sh :results output code :exports both
|
11
|
+
for i in `seq 20`; do
|
12
|
+
echo "hello"
|
13
|
+
done
|
14
|
+
#+end_src
|
15
|
+
|
16
|
+
#+RESULTS: hello
|
17
|
+
#+BEGIN_SRC sh
|
18
|
+
hello
|
19
|
+
hello
|
20
|
+
hello
|
21
|
+
hello
|
22
|
+
hello
|
23
|
+
hello
|
24
|
+
hello
|
25
|
+
hello
|
26
|
+
hello
|
27
|
+
hello
|
28
|
+
hello
|
29
|
+
hello
|
30
|
+
hello
|
31
|
+
hello
|
32
|
+
hello
|
33
|
+
hello
|
34
|
+
hello
|
35
|
+
hello
|
36
|
+
hello
|
37
|
+
hello
|
38
|
+
#+END_SRC
|
39
|
+
|
40
|
+
Something else.
|
41
|
+
|
42
|
+
#+name: hola
|
43
|
+
#+begin_src ruby :results output code :exports both
|
44
|
+
10.times do
|
45
|
+
puts "hola"
|
46
|
+
end
|
47
|
+
#+end_src
|
48
|
+
|
49
|
+
#+RESULTS: hola
|
50
|
+
#+BEGIN_SRC ruby
|
51
|
+
hola
|
52
|
+
hola
|
53
|
+
hola
|
54
|
+
hola
|
55
|
+
hola
|
56
|
+
hola
|
57
|
+
hola
|
58
|
+
hola
|
59
|
+
hola
|
60
|
+
hola
|
61
|
+
#+END_SRC
|
62
|
+
|
@@ -0,0 +1,62 @@
|
|
1
|
+
# -*- mode: org; mode: auto-fill; -*-
|
2
|
+
#+TITLE: Expected results example
|
3
|
+
|
4
|
+
#+name: hello_test
|
5
|
+
#+begin_src ruby :results output
|
6
|
+
10.times do
|
7
|
+
puts "hola"
|
8
|
+
end
|
9
|
+
#+end_src
|
10
|
+
|
11
|
+
#+RESULTS: hello_test
|
12
|
+
#+begin_example
|
13
|
+
hola
|
14
|
+
hola
|
15
|
+
hola
|
16
|
+
hola
|
17
|
+
hola
|
18
|
+
hola
|
19
|
+
hola
|
20
|
+
hola
|
21
|
+
hola
|
22
|
+
hola
|
23
|
+
#+end_example
|
24
|
+
|
25
|
+
This one would fail near christmas, outside of Japan,
|
26
|
+
when the site is down, or there is a change in the rss.xml being provided.
|
27
|
+
|
28
|
+
#+name: christmas_check
|
29
|
+
#+begin_src sh :results output
|
30
|
+
curl https://isitchristmas.com/rss.xml 2> /dev/null | grep title
|
31
|
+
# curl isitchristmastyet.com 2> /dev/null
|
32
|
+
#+end_src
|
33
|
+
|
34
|
+
#+RESULTS: christmas_check
|
35
|
+
#+begin_example
|
36
|
+
<title>Is It Christmas?</title>
|
37
|
+
<title>いいえ</title>
|
38
|
+
<title>いいえ</title>
|
39
|
+
<title>いいえ</title>
|
40
|
+
<title>いいえ</title>
|
41
|
+
<title>いいえ</title>
|
42
|
+
<title>いいえ</title>
|
43
|
+
<title>いいえ</title>
|
44
|
+
<title>いいえ</title>
|
45
|
+
<title>いいえ</title>
|
46
|
+
<title>いいえ</title>
|
47
|
+
#+end_example
|
48
|
+
|
49
|
+
#+RESULTS: fetch_google
|
50
|
+
#+begin_example
|
51
|
+
<title>Is It Christmas?</title>
|
52
|
+
<title>いいえ</title>
|
53
|
+
<title>いいえ</title>
|
54
|
+
<title>いいえ</title>
|
55
|
+
<title>いいえ</title>
|
56
|
+
<title>いいえ</title>
|
57
|
+
<title>いいえ</title>
|
58
|
+
<title>いいえ</title>
|
59
|
+
<title>いいえ</title>
|
60
|
+
<title>いいえ</title>
|
61
|
+
<title>いいえ</title>
|
62
|
+
#+end_example
|
data/spec/converge_spec.rb
CHANGED
@@ -105,4 +105,30 @@ describe OrgConverge::Command do
|
|
105
105
|
expected_contents = "init\n0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n10\n11"
|
106
106
|
File.read(File.join(example_dir, 'out.log')).should == expected_contents
|
107
107
|
end
|
108
|
+
|
109
|
+
it "should run 'expected_results' with src blocks" do
|
110
|
+
example_dir = File.join(EXAMPLES_DIR, 'expected_results')
|
111
|
+
setup_file = File.join(example_dir, 'spec.org')
|
112
|
+
|
113
|
+
o = OrgConverge::Command.new({
|
114
|
+
'<org_file>' => setup_file,
|
115
|
+
'--root-dir' => example_dir,
|
116
|
+
'--runmode' => 'spec'
|
117
|
+
})
|
118
|
+
success = o.execute!
|
119
|
+
success.should == true
|
120
|
+
end
|
121
|
+
|
122
|
+
it "should run 'expected_results' with example blocks" do
|
123
|
+
example_dir = File.join(EXAMPLES_DIR, 'expected_results')
|
124
|
+
setup_file = File.join(example_dir, 'spec2.org')
|
125
|
+
|
126
|
+
o = OrgConverge::Command.new({
|
127
|
+
'<org_file>' => setup_file,
|
128
|
+
'--root-dir' => example_dir,
|
129
|
+
'--runmode' => 'spec'
|
130
|
+
})
|
131
|
+
success = o.execute!
|
132
|
+
success.should == true
|
133
|
+
end
|
108
134
|
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: org-converge
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.8
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,11 +9,11 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2014-05-
|
12
|
+
date: 2014-05-05 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: docopt
|
16
|
-
requirement: &
|
16
|
+
requirement: &70238558687960 !ruby/object:Gem::Requirement
|
17
17
|
none: false
|
18
18
|
requirements:
|
19
19
|
- - ~>
|
@@ -21,21 +21,21 @@ dependencies:
|
|
21
21
|
version: 0.5.0
|
22
22
|
type: :runtime
|
23
23
|
prerelease: false
|
24
|
-
version_requirements: *
|
24
|
+
version_requirements: *70238558687960
|
25
25
|
- !ruby/object:Gem::Dependency
|
26
26
|
name: org-ruby
|
27
|
-
requirement: &
|
27
|
+
requirement: &70238558687480 !ruby/object:Gem::Requirement
|
28
28
|
none: false
|
29
29
|
requirements:
|
30
30
|
- - ~>
|
31
31
|
- !ruby/object:Gem::Version
|
32
|
-
version: 0.9.
|
32
|
+
version: 0.9.6
|
33
33
|
type: :runtime
|
34
34
|
prerelease: false
|
35
|
-
version_requirements: *
|
35
|
+
version_requirements: *70238558687480
|
36
36
|
- !ruby/object:Gem::Dependency
|
37
37
|
name: foreman
|
38
|
-
requirement: &
|
38
|
+
requirement: &70238558687020 !ruby/object:Gem::Requirement
|
39
39
|
none: false
|
40
40
|
requirements:
|
41
41
|
- - ~>
|
@@ -43,10 +43,10 @@ dependencies:
|
|
43
43
|
version: 0.63.0
|
44
44
|
type: :runtime
|
45
45
|
prerelease: false
|
46
|
-
version_requirements: *
|
46
|
+
version_requirements: *70238558687020
|
47
47
|
- !ruby/object:Gem::Dependency
|
48
48
|
name: tco
|
49
|
-
requirement: &
|
49
|
+
requirement: &70238558686560 !ruby/object:Gem::Requirement
|
50
50
|
none: false
|
51
51
|
requirements:
|
52
52
|
- - ~>
|
@@ -54,10 +54,10 @@ dependencies:
|
|
54
54
|
version: 0.1.0
|
55
55
|
type: :runtime
|
56
56
|
prerelease: false
|
57
|
-
version_requirements: *
|
57
|
+
version_requirements: *70238558686560
|
58
58
|
- !ruby/object:Gem::Dependency
|
59
59
|
name: rake
|
60
|
-
requirement: &
|
60
|
+
requirement: &70238558686100 !ruby/object:Gem::Requirement
|
61
61
|
none: false
|
62
62
|
requirements:
|
63
63
|
- - ~>
|
@@ -65,13 +65,14 @@ dependencies:
|
|
65
65
|
version: '10.3'
|
66
66
|
type: :runtime
|
67
67
|
prerelease: false
|
68
|
-
version_requirements: *
|
68
|
+
version_requirements: *70238558686100
|
69
69
|
description: A light configuration management tool for Org mode
|
70
70
|
email:
|
71
71
|
- waldemar.quevedo@gmail.com
|
72
72
|
executables:
|
73
73
|
- org-converge
|
74
74
|
- org-run
|
75
|
+
- org-spec
|
75
76
|
- org-tangle
|
76
77
|
extensions: []
|
77
78
|
extra_rdoc_files: []
|
@@ -85,6 +86,7 @@ files:
|
|
85
86
|
- TODO
|
86
87
|
- bin/org-converge
|
87
88
|
- bin/org-run
|
89
|
+
- bin/org-spec
|
88
90
|
- bin/org-tangle
|
89
91
|
- examples/apt-get-install/setup.org
|
90
92
|
- examples/fluentd/setup.org
|
@@ -101,6 +103,8 @@ files:
|
|
101
103
|
- spec/converge_examples/basic_tangle/conf.yml.expected
|
102
104
|
- spec/converge_examples/basic_tangle/setup.org
|
103
105
|
- spec/converge_examples/commented_block/run.org
|
106
|
+
- spec/converge_examples/expected_results/spec.org
|
107
|
+
- spec/converge_examples/expected_results/spec2.org
|
104
108
|
- spec/converge_examples/linked_tasks/tasks.org
|
105
109
|
- spec/converge_examples/runlist_example/setup.org
|
106
110
|
- spec/converge_examples/specified_block/run.org
|
@@ -136,6 +140,8 @@ test_files:
|
|
136
140
|
- spec/converge_examples/basic_tangle/conf.yml.expected
|
137
141
|
- spec/converge_examples/basic_tangle/setup.org
|
138
142
|
- spec/converge_examples/commented_block/run.org
|
143
|
+
- spec/converge_examples/expected_results/spec.org
|
144
|
+
- spec/converge_examples/expected_results/spec2.org
|
139
145
|
- spec/converge_examples/linked_tasks/tasks.org
|
140
146
|
- spec/converge_examples/runlist_example/setup.org
|
141
147
|
- spec/converge_examples/specified_block/run.org
|