bob 0.1 → 0.4.0

Sign up to get free protection for your applications and to get access to all the features.
data/README.rdoc CHANGED
@@ -1,51 +1,6 @@
1
1
  = Bob the Builder
2
2
 
3
- Given a Buildable object with the following public API:
4
-
5
- * <tt>buildable.kind</tt>
6
-
7
- Should return a Symbol with whatever kind of repository the buildable's code is
8
- in (:git, :svn, etc).
9
-
10
- * <tt>buildable.uri</tt>
11
-
12
- Returns a string like "git://github.com/integrity/bob.git", pointing to the code
13
- repository.
14
-
15
- * <tt>buildable.branch</tt>
16
-
17
- What branch of the repository should we build?
18
-
19
- * <tt>buildable.build_script</tt>
20
-
21
- Returns a string containing the build script to be run when "building".
22
-
23
- * <tt>buildable.start_building(commit_id)</tt>
24
-
25
- `commit_id` is a String that contains whatever is appropriate for the repo type,
26
- so it would be a SHA1 hash for git repos, or a numeric id for svn, etc. This is a
27
- callback so the buildable can determine how long it takes to build. It doesn't
28
- need to return anything.
29
-
30
- * <tt>buildable.finish_building(commit_id, build_status, build_output)</tt>
31
-
32
- Callback for when the build finishes. It doesn't need to return anything. It will
33
- receive a string with the commit identifier, a boolean for the build exit status
34
- (true for successful builds, false fore failed ones) and a string with the build
35
- output (both STDOUT and STDERR).
36
-
37
- A successful build is one where the build script returns a zero status code.
38
-
39
- Bob will, when called like:
40
-
41
- Bob.build(buildable, commit_id)
42
-
43
- 1. Checkout the buildable on the specified commit
44
- 2. Call <tt>buildable.start_building</tt>
45
- 3. Run the script provided in <tt>build_script</tt> in the buildable.
46
- 4. When the build process finishes, it will call <tt>finish_building</tt> with
47
- the commit_id, the build status (true if the script returns a status code
48
- of 0, false otherwise), and a string with the build output (both STDOUT and STDERR).
3
+ Bob builds your code.
49
4
 
50
5
  == Do I need this?
51
6
 
@@ -54,5 +9,6 @@ automated CI server, which is what most people need.
54
9
 
55
10
  == Credits
56
11
 
12
+ Thanks a lot to raggi[http://github.com/raggi] for the threaded engine.
57
13
  Authors:: Nicolas Sanguinetti (foca[http://github.com/foca]) and Simon Rozet (sr[http://github.com/sr])
58
14
  License:: MIT (Check LICENSE for details)
data/Rakefile CHANGED
@@ -1,13 +1,14 @@
1
1
  require "rake/testtask"
2
2
 
3
+ rdoc_sources = %w(hanna/rdoctask rdoc/task rake/rdoctask)
3
4
  begin
4
- require "hanna/rdoctask"
5
+ require rdoc_sources.shift
5
6
  rescue LoadError
6
- require "rake/rdoctask"
7
+ retry
7
8
  end
8
9
 
9
10
  begin
10
- require "metric_fu"
11
+ require "metric_fu" if RUBY_VERSION < "1.9"
11
12
  rescue LoadError
12
13
  end
13
14
 
@@ -20,23 +21,14 @@ end
20
21
  desc "Default: run all tests"
21
22
  task :default => :test
22
23
 
23
- SCMs = %w[git svn]
24
-
25
- desc "Run unit tests"
26
- task :test => SCMs.map { |scm| "test:#{scm}" } do
27
- ruby "test/bob_test.rb"
28
- ruby "test/background_engine/threaded_test.rb"
24
+ desc "Run tests"
25
+ Rake::TestTask.new(:test) do |t|
26
+ t.test_files = FileList["test/*_test.rb"]
27
+ t.libs << "test"
29
28
  end
30
29
 
31
- SCMs.each { |scm|
32
- desc "Run unit tests with #{scm}"
33
- task "test:#{scm}" do
34
- ruby "test/scm/#{scm}_test.rb"
35
- end
36
- }
37
-
38
- Rake::RDocTask.new do |rd|
39
- rd.main = "README"
30
+ (defined?(RDoc::Task) ? RDoc::Task : Rake::RDocTask).new do |rd|
31
+ rd.main = "README.rdoc"
40
32
  rd.title = "Documentation for Bob the Builder"
41
33
  rd.rdoc_files.include("README.rdoc", "LICENSE", "lib/**/*.rb")
42
34
  rd.rdoc_dir = "doc"
data/bob.gemspec CHANGED
@@ -1,7 +1,7 @@
1
1
  Gem::Specification.new do |s|
2
2
  s.name = "bob"
3
- s.version = "0.1"
4
- s.date = "2009-04-12"
3
+ s.version = "0.4.0"
4
+ s.date = "2009-10-10"
5
5
 
6
6
  s.description = "Bob the Builder will build your code. Simple."
7
7
  s.summary = "Bob builds!"
@@ -17,30 +17,31 @@ Gem::Specification.new do |s|
17
17
 
18
18
  s.add_dependency "addressable"
19
19
 
20
- if s.respond_to?(:add_development_dependency)
21
- s.add_development_dependency "sr-mg"
22
- s.add_development_dependency "contest"
23
- s.add_development_dependency "redgreen"
24
- s.add_development_dependency "ruby-debug"
25
- end
26
-
27
20
  s.files = %w[
28
21
  .gitignore
29
22
  LICENSE
30
23
  README.rdoc
31
24
  Rakefile
32
25
  bob.gemspec
26
+ deps.rip
33
27
  lib/bob.rb
34
- lib/bob/background_engines.rb
35
- lib/bob/background_engines/foreground.rb
36
28
  lib/bob/builder.rb
29
+ lib/bob/engine.rb
30
+ lib/bob/engine/threaded.rb
37
31
  lib/bob/scm.rb
38
32
  lib/bob/scm/abstract.rb
39
33
  lib/bob/scm/git.rb
40
- lib/core_ext/object.rb
34
+ lib/bob/scm/svn.rb
35
+ lib/bob/test.rb
36
+ lib/bob/test/builder_stub.rb
37
+ lib/bob/test/repo.rb
41
38
  test/bob_test.rb
39
+ test/deps.rip
40
+ test/engine/threaded_test.rb
41
+ test/git_test.rb
42
42
  test/helper.rb
43
- test/helper/git_helper.rb
44
- test/helper/buildable_stub.rb
43
+ test/mixin/scm.rb
44
+ test/svn_test.rb
45
+ test/test_test.rb
45
46
  ]
46
47
  end
data/deps.rip ADDED
@@ -0,0 +1 @@
1
+ addressable 2.1.0
data/lib/bob.rb CHANGED
@@ -1,4 +1,5 @@
1
1
  require "fileutils"
2
+ require "pathname"
2
3
  require "yaml"
3
4
  require "logger"
4
5
  require "time"
@@ -6,31 +7,28 @@ require "addressable/uri"
6
7
 
7
8
  require "bob/builder"
8
9
  require "bob/scm"
9
- require "bob/background_engines"
10
+ require "bob/engine"
10
11
 
11
12
  module Bob
12
- # Builds the specified <tt>buildable</tt>. This object must understand
13
- # the API described in the README.
14
- def self.build(buildable, commit_ids)
15
- Array(commit_ids).each do |commit_id|
16
- Builder.new(buildable, commit_id).build
17
- end
13
+ def self.build(buildable)
14
+ Builder.new(buildable).build
18
15
  end
19
16
 
20
- # Directory where the code for the different buildables will be checked out. Make sure
21
- # the user running Bob is allowed to write to this directory.
17
+ # Directory where the code for the different buildables will be checked out.
18
+ # Make sure the user running Bob is allowed to write to this directory.
22
19
  def self.directory
23
- @directory || "/tmp"
20
+ Pathname(@directory || "/tmp")
24
21
  end
25
22
 
26
- # What will you use to build in background. Must respond to <tt>call</tt> and take a block
27
- # which will be run "in background". The default is to run in foreground.
23
+ # What will you use to build in background. Must respond to <tt>call</tt> and
24
+ # take a block which will be run "in background". The default is to run in
25
+ # foreground.
28
26
  def self.engine
29
- @engine || BackgroundEngines::Foreground
27
+ @engine || Engine::Foreground
30
28
  end
31
29
 
32
- # What to log with (must implement ruby's Logger interface). Logs to STDOUT by
33
- # default.
30
+ # What to log with (must implement ruby's Logger interface). Logs to STDOUT
31
+ # by default.
34
32
  def self.logger
35
33
  @logger || Logger.new(STDOUT)
36
34
  end
data/lib/bob/builder.rb CHANGED
@@ -1,53 +1,42 @@
1
1
  module Bob
2
- # A Builder will take care of building a buildable (wow, you didn't see that coming,
3
- # right?).
4
2
  class Builder
5
- attr_reader :buildable, :commit_id
3
+ attr_reader :buildable
6
4
 
7
- def initialize(buildable, commit_id)
5
+ def initialize(buildable)
8
6
  @buildable = buildable
9
- @commit_id = commit_id
10
7
  end
11
8
 
12
- # This is where the magic happens:
13
- #
14
- # 1. Check out the repo to the appropriate commit.
15
- # 2. Notify the buildable that the build is starting.
16
- # 3. Run the build script on it in the background.
17
- # 4. Reports the build back to the buildable.
18
9
  def build
19
- Bob.logger.info "Building #{commit_id} of the #{buildable.kind} repo at #{buildable.uri}"
20
- in_background do
21
- scm.with_commit(commit_id) do
22
- buildable.start_building(commit_id, scm.info(commit_id))
23
- build_status, build_output = run_build_script
24
- buildable.finish_building(commit_id, build_status, build_output)
25
- end
26
- end
10
+ scm.with_commit(buildable["commit"]) { |commit|
11
+ started(scm.info(commit))
12
+ completed(*run)
13
+ }
27
14
  end
28
15
 
29
- private
16
+ protected
17
+ def started(commit_info)
18
+ end
30
19
 
31
- def run_build_script
32
- build_output = nil
20
+ def completed(status, output)
21
+ end
33
22
 
34
- Bob.logger.debug "Running the build script for #{buildable.uri}"
35
- IO.popen(build_script, "r") { |output| build_output = output.read }
36
- Bob.logger.debug("Ran build script `#{build_script}` and got:\n#{build_output}")
23
+ private
24
+ def run
25
+ output = ""
26
+ status = false
37
27
 
38
- [$?.success?, build_output]
39
- end
28
+ IO.popen(script, "r") { |io| output = io.read }
29
+ status = $?.success?
40
30
 
41
- def build_script
42
- "(cd #{scm.working_dir} && #{buildable.build_script} 2>&1)"
43
- end
31
+ [status, output]
32
+ end
44
33
 
45
- def scm
46
- @scm ||= SCM.new(buildable.kind, buildable.uri, buildable.branch)
47
- end
34
+ def script
35
+ "(cd #{scm.dir_for(buildable["commit"])} && #{buildable["command"]} 2>&1)"
36
+ end
48
37
 
49
- def in_background(&block)
50
- Bob.engine.call(block)
51
- end
38
+ def scm
39
+ @scm ||= SCM.new(buildable["scm"], buildable["uri"], buildable["branch"])
40
+ end
52
41
  end
53
42
  end
@@ -2,8 +2,8 @@ module Bob
2
2
  # Different engines to run code in background. An engine is any object
3
3
  # that responds to #call and takes a Proc object, which should be executed
4
4
  # "in the background". The different engines are:
5
- module BackgroundEngines
6
- autoload :Foreground, "bob/background_engines/foreground"
7
- autoload :Threaded, "bob/background_engines/threaded"
5
+ module Engine
6
+ autoload :Threaded, "bob/engine/threaded"
7
+ Foreground = lambda {|b| b.call }
8
8
  end
9
9
  end
@@ -0,0 +1,145 @@
1
+ require "thread"
2
+
3
+ module Bob
4
+ module Engine
5
+ # A thread pool based build engine. This engine simply adds jobs to an
6
+ # in-memory queue, and processes them as soon as possible.
7
+ class Threaded
8
+ # The optional pool size controls how many threads will be created.
9
+ def initialize(pool_size = 2, logger = Bob.logger)
10
+ @pool = ThreadPool.new(pool_size, logger)
11
+ @logger = logger
12
+ end
13
+
14
+ # Adds a job to the queue.
15
+ def call(job)
16
+ @pool << job
17
+ end
18
+
19
+ # The number of jobs currently in the queue.
20
+ def njobs
21
+ @pool.njobs
22
+ end
23
+
24
+ # This method will not return until #njobs returns 0.
25
+ def wait!
26
+ Thread.pass until @pool.njobs == 0
27
+ end
28
+
29
+ # Manage a pool of threads, allowing for spin up / spin down of the
30
+ # contained threads.
31
+ # Simply processes work added to it's queue via #push.
32
+ # The default size for the pool is 2 threads.
33
+ class ThreadPool
34
+ # A thread safe single value for use as a counter.
35
+ class Incrementor
36
+ # The value passed in will be the initial value.
37
+ def initialize(v = 0)
38
+ @m = Mutex.new
39
+ @v = v
40
+ end
41
+
42
+ # Add the given value to self, default 1.
43
+ def inc(v = 1)
44
+ sync { @v += v }
45
+ end
46
+
47
+ # Subtract the given value to self, default 1.
48
+ def dec(v = 1)
49
+ sync { @v -= v }
50
+ end
51
+
52
+ # Simply shows the value inspect for convenience.
53
+ def inspect
54
+ @v.inspect
55
+ end
56
+
57
+ # Extract the value.
58
+ def to_i
59
+ @v
60
+ end
61
+
62
+ private
63
+
64
+ # Wrap the given block in a mutex.
65
+ def sync(&b)
66
+ @m.synchronize &b
67
+ end
68
+ end
69
+
70
+ # The number of threads in the pool.
71
+ attr_reader :size
72
+
73
+ # The job queue.
74
+ attr_reader :jobs
75
+
76
+ # Set the size of the thread pool. Asynchronously run down threads
77
+ # that are no longer required, and synchronously spawn new required
78
+ # threads.
79
+ def size=(other)
80
+ @size = other
81
+
82
+ if @workers.size > @size
83
+ (@workers.size - @size).times do
84
+ @workers.shift[:run] = false
85
+ end
86
+ else
87
+ (@size - @workers.size).times do
88
+ @workers << spawn
89
+ end
90
+ end
91
+ end
92
+
93
+ # Default pool size is 2 threads.
94
+ def initialize(size = nil, logger = Bob.logger)
95
+ size ||= 2
96
+ @jobs = Queue.new
97
+ @njobs = Incrementor.new
98
+ @workers = Array.new(size) { spawn }
99
+ @logger = logger
100
+ end
101
+
102
+ # Adds a job to the queue, the job can be any number of objects
103
+ # responding to call, and/or a block.
104
+ def add(*jobs, &blk)
105
+ jobs = jobs + Array(blk)
106
+
107
+ jobs.each do |job|
108
+ @jobs << job
109
+ @njobs.inc
110
+ end
111
+ end
112
+
113
+ alias_method :push, :add
114
+ alias_method :<<, :add
115
+
116
+ # A peak at the number of jobs in the queue. N.B. May differ, but
117
+ # should be more accurate than +jobs.size+.
118
+ def njobs
119
+ @njobs.to_i
120
+ end
121
+
122
+ private
123
+
124
+ # Create a new thread and return it. The thread will run until the
125
+ # thread-local value +:run+ is changed to false or nil.
126
+ def spawn
127
+ Thread.new do
128
+ c = Thread.current
129
+ c[:run] = true
130
+
131
+ while c[:run]
132
+ job = @jobs.pop
133
+ begin
134
+ job.call
135
+ rescue Exception => e
136
+ @logger.error("Exception occured during build: #{e.message}")
137
+ end
138
+ @njobs.dec
139
+ end
140
+ end
141
+ end
142
+ end
143
+ end
144
+ end
145
+ end