build-graph 1.0.3 → 1.0.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 2fe664d73ec6b69811e09e9afc08106e751fe154
-  data.tar.gz: 6f9d78571bdd3000fdb171e70855a0da1881cd33
+  metadata.gz: 7dcba8fbc1368e6fd3a2c4117d6a787609b4127e
+  data.tar.gz: 4984ebb6012624af504492ce5ad5597b471f362b
 SHA512:
-  metadata.gz: 04c4b26479fe93ca729f7aa18b996138550f32b9d5214667ea85d7ab29453c847fd65bd703d10ed6b1d4083434eb81ef9f64d42c0cc2b633702d9e6cd4a1ee68
-  data.tar.gz: df5b79fd7d44d134170e78e0b7d032fe8ba2322c83f5b3c3ab07cfc4e05b4636d4af4763510f5f49485cd729dca15ab4d6cdc1341ec2858c555ca19c176380d5
+  metadata.gz: 5e34905fd1150c7d69c77be9292505a7f67f08c8b46924047971a2af18665c8bbf8f9c0d8a8349e0eec2891410b443a45005ce9b9b27ae1e58cb56d37ed81baa
+  data.tar.gz: 471ef8ad632e75fcc1048d497aabbe5959a86e9dd226edf6014ee70c371eee66fbd9592704920b217fe55aa9d4711aaeb9d0e0c0706e142868445baae6561517

data/.travis.yml

@@ -1,9 +1,5 @@
 language: ruby
-compiler: clang
-before_install:
-  - sudo add-apt-repository --yes ppa:ubuntu-toolchain-r/test
-  - sudo apt-get -qq update
-  - sudo apt-get -qq install libstdc++-4.8-dev
+sudo: false
 rvm:
   - 2.0.0
   - 2.1.8
@@ -13,6 +9,13 @@ rvm:
   - rbx-2
 env: COVERAGE=true
 matrix:
+  fast_finish: true
   allow_failures:
+    - rvm: ruby-head
     - rvm: "rbx-2"
-
+addons:
+  apt:
+    sources:
+    - ubuntu-toolchain-r-test
+    packages:
+    - libstdc++-4.8-dev

data/README.md

@@ -2,27 +2,27 @@
 
 Build::Graph is a framework for build systems, with specific functionality for dealing with file based processes.
 
-[![Build Status](https://secure.travis-ci.org/ioquatix/build-graph.png)](http://travis-ci.org/ioquatix/build-graph)
-[![Code Climate](https://codeclimate.com/github/ioquatix/build-graph.png)](https://codeclimate.com/github/ioquatix/build-graph)
+[![Build Status](https://secure.travis-ci.org/ioquatix/build-graph.svg)](http://travis-ci.org/ioquatix/build-graph)
+[![Code Climate](https://codeclimate.com/github/ioquatix/build-graph.svg)](https://codeclimate.com/github/ioquatix/build-graph)
 [![Coverage Status](https://coveralls.io/repos/ioquatix/build-graph/badge.svg)](https://coveralls.io/r/ioquatix/build-graph)
 
 ## Installation
 
 Add this line to your application's Gemfile:
 
-    gem 'build-graph'
+	gem 'build-graph'
 
 And then execute:
 
-    $ bundle
+	$ bundle
 
 Or install it yourself as:
 
-    $ gem install build-graph
+	$ gem install build-graph
 
 ## Usage
 
-A build graph is an abstract set of `[input, process, output]` nodes. A node executes it's proces within the context of a `Task` which represents a specific set of inputs and outputs and is managed within a `Walker` that walks over graph nodes, regenerating tasks where required. If inputs or outputs change (i.e. become dirty), the task is destroyed and regenerated.
+A build graph is an abstract set of `[input, process, output]` nodes. A node executes it's process within the context of a `Task` which represents a specific set of inputs and outputs and is managed within a `Walker` that walks over graph nodes, regenerating tasks where required. If inputs or outputs change (i.e. become dirty), the old task is nullified.
 
 A `Walker` is used to traverse the build graph once. As it walks over the graph it builds a set of `Edge` relationships between nodes and only traverses relationships which are complete `Walker#wait_on_paths`. Parent nodes also wait until all their children are complete `Walker#wait_on_nodes` It also keeps track of failures `Walker#failed?` and fails all dependencies of a node.
 
@@ -36,12 +36,36 @@ Outputs from a node should be all files that are generated directly by the proce
 
 ### Dirty Propagation
 
-A `Node` has a set of `#inputs` and `#outputs` but these are abstract. A `Task`, at the time of execution, captures it's inputs and outputs and these may be monitored for changes in real time. The simplest way to cause a task to regenerate is to simply remove it from the existing graph and it will be regenerated.
+A `Node` has a set of `#inputs` and `#outputs` but these are abstract. For example, `#outputs` could be `:inherit` which means that the node symbolically has all the outputs of all it's direct children. A `Task`, at the time of execution, captures it's inputs and outputs and these may be monitored for changes in real time.
 
 File changes are currently detected using `File::mtime` as this is generally a good trade off between efficiency and accuracy.
 
 When a task is marked as dirty, it also marks all it's outputs as being dirty, which in cause could mark other tasks as dirty. This is the mechanism for which dirtiness propagates through the graph. The walker should only have to traverse the graph once to build it completely. If multiple updates are required (i.e. buidling one part of the graph implicitly dirties another part of the graph), the specification of the graph is incomplete and this may lead to problems within the build graph.
 
+### Example Graph
+
+	target("Library/UnitTest", [] -> :inherit) do
+		library([UnitTest.cpp] -> UnitTest.a) do
+			compile([UnitTest.cpp] -> UnitTest.o)
+			link([UnitTest.o] -> libUnitTest.a)
+		end
+		
+		copy headers: [UnitTest.hpp]
+		
+		# Outputs become libUnitTest.a and UnitTest.hpp
+	end
+
+	target("Executable/UnitTest", [] -> :inherit) do
+		depends("Library/UnitTest")
+		
+		executable(main.cpp -> UnitTest) do
+			compile(main.cpp -> main.o)
+			link([main.o, libUnitTest.a] -> UnitTest)
+		end
+		
+		# Outputs become UnitTest
+	end
+
 ## Contributing
 
 1. Fork it
@@ -54,7 +78,7 @@ When a task is marked as dirty, it also marks all it's outputs as being dirty, w
 
 Released under the MIT license.
 
-Copyright, 2012, 2014, by [Samuel G. D. Williams](http://www.codeotaku.com/samuel-williams).
+Copyright, 2012, 2014, 2016, by [Samuel G. D. Williams](http://www.codeotaku.com/samuel-williams).
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/build-graph.gemspec

@@ -22,8 +22,8 @@ Gem::Specification.new do |spec|
 	
 	spec.required_ruby_version = '>= 2.0'
 	
-	spec.add_dependency "process-group", "~> 1.0.1"
-	spec.add_dependency "build-files", "~> 1.0.2"
+	spec.add_dependency "process-group", "~> 1.1.0"
+	spec.add_dependency "build-files", "~> 1.0.3"
 	
 	spec.add_development_dependency "build-makefile", "~> 1.0.0"
 	

data/lib/build/graph.rb

@@ -18,7 +18,7 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 # THE SOFTWARE.
 
-require_relative 'graph/error'
+require_relative 'graph/task'
 require_relative 'graph/node'
 require_relative 'graph/walker'
 require_relative 'graph/edge'

data/lib/build/graph/call_stack.rb

@@ -0,0 +1,51 @@
+# Copyright, 2014, by Samuel G. D. Williams. <http://www.codeotaku.com>
+# 
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+# 
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+# 
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+module Build
+	module Graph
+		# A call stack contains frames to track state during nested invocations.
+		class CallStack
+			def initialize
+				# Saves state if supplied to #call, which is useful for top level state:
+				@frames = [{}.freeze]
+			end
+			
+			# All stack frames which had state associated with them.
+			attr :frames
+			
+			# Yield with the given state, merged with any prior state.
+			def with(state)
+				if state and !state.empty?
+					@frames << @frames.last.merge(state).freeze
+					yield
+					@frames.pop
+				else
+					yield
+				end
+			end
+			
+			# The current stack frame state.
+			def last
+				@frames.last
+			end
+		end
+	end
+end
+

data/lib/build/graph/edge.rb

@@ -18,8 +18,6 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 # THE SOFTWARE.
 
-require_relative 'error'
-
 require 'fiber'
 
 module Build
@@ -72,6 +70,7 @@ module Build
 				end
 			end
 			
+			# This is called in the case that a parent fails to complete because a child task has failed.
 			def skip!(task)
 				@vertices += 1
 				

data/lib/build/graph/task.rb

@@ -20,6 +20,21 @@
 
 module Build
 	module Graph
+		class TransientError < StandardError
+		end
+		
+		module ChildrenFailed
+			def self.to_s
+				"Children tasks failed!"
+			end
+		end
+		
+		module InputsFailed
+			def self.to_s
+				"Tasks generating inputs failed!"
+			end
+		end
+		
 		class Task
 			def initialize(walker, node)
 				@walker = walker
@@ -69,11 +84,11 @@ module Build
 							fail!(error)
 						end
 					else
-						fail!(:inputs)
+						fail!(InputsFailed)
 					end
 					
 					unless wait_for_children?
-						fail!(:children)
+						fail!(ChildrenFailed)
 					end
 					
 					update_outputs
@@ -148,6 +163,11 @@ module Build
 			end
 			
 			def fail!(error)
+				if logger = @walker.logger
+					logger.error("Task #{self} failed: #{error}")
+					logger.debug(error) if error.kind_of?(Exception)
+				end
+				
 				@error = error
 				@state = :failed
 			end
@@ -155,7 +175,7 @@ module Build
 			# Returns false if any input failed.
 			def wait_for_inputs?
 				# Wait on any inputs, returns whether any inputs failed:
-				@walker.wait_on_paths(@inputs)
+				@walker.wait_on_paths(self, @inputs)
 			end
 			
 			# Returns false if any child failed.

data/lib/build/graph/version.rb

@@ -20,6 +20,6 @@
 
 module Build
 	module Graph
-		VERSION = "1.0.3"
+		VERSION = "1.0.4"
 	end
 end

data/lib/build/graph/walker.rb

@@ -24,7 +24,8 @@ require 'logger'
 require_relative 'task'
 require_relative 'node'
 require_relative 'edge'
-require_relative 'error'
+
+require_relative 'call_stack'
 
 module Build
 	module Graph
@@ -46,9 +47,10 @@ module Build
 				
 				@update = block
 				
+				# A list of paths which are currently being generated by tasks:
 				@outputs = {}
 				
-				@parents = {}
+				@parents = Hash.new{|h,k| h[k] = []}
 				
 				# Failed output paths:
 				@failed_tasks = []
@@ -56,10 +58,17 @@ module Build
 				
 				@logger = logger || Logger.new(nil)
 				@monitor = Files::Monitor.new(logger: @logger)
+				
+				@call_stack = CallStack.new
 			end
 			
-			attr :tasks # {Node => Task}
+			# Primarily for debugging from within Task
+			attr :logger
 			
+			# An Array of all instantiated tasks.
+			attr :tasks
+			
+			# An Array of transient outputs which are currently being generated.
 			attr :outputs
 			
 			attr :failed_tasks
@@ -78,15 +87,17 @@ module Build
 				end
 			end
 			
-			def call(node)
-				# We try to fetch the task if it has already been invoked, otherwise we create a new task.
-				@tasks.fetch(node) do
-					@logger.debug{"Update: #{node}"}
-					
-					@update.call(self, node)
-					
-					# This should now be defined:
-					@tasks[node]
+			def call(node, state = nil)
+				@call_stack.with(state) do
+					# We try to fetch the task if it has already been invoked, otherwise we create a new task.
+					@tasks.fetch(node) do
+						@logger.debug{"Update: #{node}"}
+						
+						@update.call(self, node, @call_stack.last)
+						
+						# This should now be defined:
+						return @tasks[node]
+					end
 				end
 			end
 			
@@ -94,7 +105,7 @@ module Build
 				@failed_tasks.size > 0
 			end
 			
-			def wait_on_paths(paths)
+			def wait_on_paths(task, paths)
 				# If there are no paths, we are done:
 				return true if paths.count == 0
 				
@@ -102,6 +113,7 @@ module Build
 				edge = Edge.new
 				
 				paths = paths.collect(&:to_s)
+				@logger.debug{"Task #{task} is waiting on paths #{paths}"}
 				
 				paths.each do |path|
 					# Is there a task generating this output?
@@ -110,6 +122,11 @@ module Build
 						outputs << edge
 						edge.increment!
 					end
+					
+					# What should we do about paths which haven't been registered as outputs?
+					# Either they exist - or they don't.
+					# If they exist, it means they are probably static inputs of the build graph.
+					# If they don't, it might be an error, or it might be deliberate.
 				end
 				
 				failed = paths.any?{|path| @failed_outputs.include? path}
@@ -125,6 +142,8 @@ module Build
 				# If there are no children like this, then done:
 				return true if children.size == 0
 				
+				@logger.debug{"Task #{parent} is waiting on #{children.count} children"}
+				
 				# Otherwise, construct an edge to track state changes:
 				edge = Edge.new
 				
@@ -135,7 +154,6 @@ module Build
 						# We are waiting for this child to finish:
 						edge.increment!
 						
-						@parents[child.node] ||= []
 						@parents[child.node] << edge
 					end
 				end
@@ -144,12 +162,15 @@ module Build
 			end
 			
 			def enter(task)
-				@logger.debug{"--> #{task.node.process}"}
+				@logger.debug{"Walker entering: #{task.node.process}"}
 				
 				@tasks[task.node] = task
 				
-				# In order to wait on outputs, they must be known before entering the task. This might seem odd, but unless we know outputs are being generated, waiting for them to complete is impossible - unless this was somehow specified ahead of time. The implications of this logic is that all tasks must be sequential in terms of output -> input chaning. This is not a problem in practice.
+				# In order to wait on outputs, they must be known before entering the task. This might seem odd, but unless we know outputs are being generated, waiting for them to complete is impossible - unless this was somehow specified ahead of time. The implications of this logic is that all tasks must be sequential in terms of output -> input chaning. This is by design and is not a problem in practice.
+				
 				if outputs = task.outputs
+					@logger.debug{"Task will generate outputs: #{outputs.to_a.collect(&:to_s).inspect}"}
+					
 					outputs.each do |path|
 						@outputs[path.to_s] = []
 					end
@@ -157,7 +178,7 @@ module Build
 			end
 			
 			def exit(task)
-				@logger.debug{"<-- #{task.node.process}"}
+				@logger.debug{"Walker exiting: #{task.node.process}, task #{task.failed? ? 'failed' : 'succeeded'}"}
 				
 				# Fail outputs if the node failed:
 				if task.failed?
@@ -172,6 +193,14 @@ module Build
 				task.outputs.each do |path|
 					path = path.to_s
 					
+					if logger.debug?
+						if task.failed?
+							@logger.debug "\tFile failed: #{path}"
+						else
+							@logger.debug "\tFile available: #{path}"
+						end
+					end
+					
 					if edges = @outputs.delete(path)
 						edges.each{|edge| edge.traverse(task)}
 					end
@@ -186,7 +215,7 @@ module Build
 			end
 			
 			def delete(node)
-				@logger.debug{">-< #{node.process}"}
+				@logger.debug{"Delete #{node}"}
 
 				if task = @tasks.delete(node)
 					@monitor.delete(task)

data/spec/build/graph/build_test.rb

@@ -57,8 +57,9 @@ viz = Graphviz::Graph.new
 viz.attributes[:rankdir] = 'LR'
 
 walker.run do
-	walker.update(top)
-	group.wait
+	group.wait do
+		walker.update(top)
+	end
 	
 	walker.tasks.each do |node, task|
 		input_nodes = []