gd_bam 0.0.6 → 0.0.7

data/README.md

@@ -35,6 +35,7 @@ You need a working project with API access. You should also have username pass a
 ### Warnings
 The project is currently cloned out of an existing project. That means that you need to have access to it. If you do not (the project PID is i49w4c73c2mh75iiehte3fv3fbos8h2k) ask svarovsky@gooddata.com. Eventually this will be covered by a template so you will not need to do anything special. The template creation is tracked here https://jira.intgdc.com/browse/GD-34641 .
 
+###Let's get to it
 We will spin a goodsales project and load it with data. Prerequisite for this is a functioning Salesforce poject that you can grab at force.com.
 
 `bam scaffold project test --blueprint goodsales`
@@ -58,7 +59,7 @@ We will talk in detail why we split etl for project into downloaders and rest of
 
 You can watch the progress in our CloudConnect console.
 
-Now generate the etl. 
+Now generate the etl.
 
 `bam generate`
 
@@ -68,8 +69,19 @@ This works the same as with downloaders but its default target is clover_project
 
 After it is finished log in to gooddata go into your project and celebrate. You just did project using BAM.
 
+##Painful metadata management
+Key pain that I had with CloudConnect is that I hated the management of metadata. Every project I saw was just pile of metadata definition that has to be constantly changed and tweaked. This is caused by couple of chioces that creators of underlying Clover engine made in the beginning and probably will not be changed easily. While I am trying to make it better I am still bound by these choices and sometimes the wiring stick out - sorry for that.
+
+###Incremental metadata
+Bam is working with something that is called Incremental metadata. Metadata is not defined in each step you just say what you want to change. Picture is probably better than thousand words.
+
+You have a conceptual picture of a simple transformation. You get a Tap that downloads FirstName and LastName somewhere. Obviously you would like to join them together to form a name. Exactly this happens in the second box the transformer. You would like to sink the only field and that is name. So on the next edge what you say is "I am adding Name and removing FirstName and LastName". So far so good. What is elegant about this approach is that how it copes with change. Imagine that the tap gets not only FirstName and LastName but also Age. Now what you need to change? If you would do it the old way You would have to change metadata on both edges, tap transformer and sink. With incremental metadata you need to change tap and sink nothing else. Since I claim that dealing with metadata was the biggest pain this is a lot of work (and errors) that you just saved.
+
+###Types or not?
+Clover engine is built on Java and it shows. It is statically typed and CTL Clover transformation language resembles Java a lot. While it helps speed and many people claim it prevents errors it also causes more work and helps metadata explosion. Sometimes you need to translate an field into another field becuase you need to do something specific or the component needs it. It is not problem per se but it is important to see the tradeoffs and push the functionality into the components that should work for you and not against you. It is also important to do certain tasks at certain phases. If you do this you found out that certain parts are easier to automate or you can easily reuse work that you did somewhere else.
+
 ##Taps
-Taps are sources of data. There are various types (:-)) but right now you can use just salesforce tap.
+Taps are sources of data. Right now you can use just salesforce tap.
 
 ###Common properties
 Every tap has the source and an id. Source tells it where to go to grab data. DB, CSV, SalesForce. Depending on the type the definition might be different. Id is a field that holds a name with which you can reference particular tap. Obviously it needs to be unique.
@@ -155,10 +167,10 @@ Fail early. There is nothing more frustrating than when the ETL fails during exw
 ####Mandatory fields
 Sometimes it is necessary to move fields around in SF. In such case the tap will. If you know this upfront you can tell BAM that this field is not mandatory and it will silently go along filling the missing field with ''
 
-###Sinks
+##Flows
+Flow is an abstraction
 
-
-This tap will download users from sf (you have to provide credentials in params.json). It then runs graph called "process user" (this is part of the distribution). This graph concatenates first name and last name together.
+This flow will download users from sf (you have to provide credentials in params.json). It then runs graph called "process user" (this is part of the distribution). This graph concatenates first name and last name together. It then feeds data to the sink.
 
 	GoodData::CloverGenerator::DSL::flow("user") do |f|
 	  tap(:id => "user")
@@ -175,29 +187,59 @@ This tap will download users from sf (you have to provide credentials in params.
 
 Now you have to provide it the definition of tap which you can do like this.
 
+###When I call external graph? How does it work?
+In the flow you can call external graph by using
 
+	graph('my_graph')
+	
+It goes to 2 places (this will change) and tries to find the graph. First place is your `local_graphs` directory in your project the second place is central reporsitory that is currently inside bam library and this part will probably change. 
 
-Also you need to provide a definition for sink which can look somwhow like this.
+##Sinks
 
-	{
-	   "type"     : "dataset"
-	  ,"id"       : "user"
-	  ,"gd_name"  : "user"
-	  ,"fields"   : [
-	    {
-	      "name" : "Id"
-	    },
-	    {
-	      "name" : "Name"
-	    }
-	  ]
-	}
+Sink is a definition of where data goes to. Currently there is only one sink type and that is gooddata dataset.
 
-For this example to work you need to provide SF and gd credentials. Provide them in params.json. You would need to provide also a project with appropriate project but this is out of scope of this "example" (I am working on tools that would make it easier).
+###GoodData
+	
+    {
+      "type" : "dataset",
+      "id" : "account",
+      "gd_name" : "account",
+      "fields": [
+        {
+          "name" : "id",
+          "type" : "connection_point",
+          "meta" : "Id"
+        },
+        {
+          "type" : "label",
+          "for"  : "id",
+          "name" : "name",
+          "meta" : "Name"
+        },
+        {
+          "type" : "label",
+          "for"  : "id",
+          "name" : "url",
+          "meta" : "Url"
+        },
+        {
+          "name" : "market",
+          "type" : "attribute",
+          "meta" : "Market__c"
+        }
+      ]
+    }
+      
+Gooddat sink is currently just mimicking the CL tool definition + some shortcuts on top of that. If you are familiar with CL tool you should be right at home if I tell you that the only additional thing you have to provide is telling BAM which metadata field is pulled in to a given field.
+
+
+
+<!--For this example to work you need to provide SF and gd credentials. Provide them in params.json. You would need to provide also a project with appropriate project but this is out of scope of this "example" (I am working on tools that would make it easier).
 
 Now run `bam generate` and there will be a folder with the clover project generated. Open it in CC find main.grf and run it. After crunching for a while you should see data in the project.
+-->
 
-### Runtime commands
+##Runtime commands
 Part of the distribution is the bam executable which lets you do several neat things on the commandline
 
 Run `bam` to get the list of commands
@@ -207,7 +249,7 @@ Run `bam help command` to get help about the command
 deploys the directory to the server. You can provide the param of the process as a parameter
 
 ### generate
-Generates the ETL. The default target directory is clover_project (currently cannot be changed). You can provide --only parameter to specify the name of the flow to be processed if you do not need to generate all flows. Currently you can specify only on in only param
+Generates the ETL. The default target directory is clover_project (currently cannot be changed). You can provide --only parameter to specify the name of the flow to be processed if you do not need to generate all flows. Currently you can specify only one flow
 
 ### generate_downloaders
 If you have incremental downloaders in your project it good to deploy them as a separate process. This generates only the downloaders and is meant for exacltly this purpose. If you are interested about why it is a good idea. Take a look here (TBD). The target directory is downloaders_project (currently cannot be changed).
@@ -216,10 +258,11 @@ If you have incremental downloaders in your project it good to deploy them as a
 Investigates what is changed and performs the changes in the target project. Uses CL tool behind the scenes. Needs more work
 
 ### model_sync
-Syncs the model with the definition in sinks. {Todo} Add interactive addition. Sometimes the new field can actually be a typo or something like that. Possible to uncover with validate_datasets
+Syncs the model with the definition in sinks. Sometimes the new field can actually be a typo or something like that. Possible to uncover with validate_datasets
 
 ### run
-TBD
+Runs the project and 
+`bam run clover-project --email me@gooddata.com`
 
 ### scaffold
 Takes an argument and creates a scaffold for you. It can scaffold project, flow, sink and tap.

data/bin/bam

@@ -31,6 +31,15 @@ command :generate do |c|
   end
 end
 
+desc 'Jacks into SF.'
+# arg_name 'Describe arguments to new here'
+command :sf_jack_in do |c|
+
+  c.action do |global_options,options,args|
+    GoodData::CloverGenerator.sf_jack_in
+  end
+end
+
 desc 'Generates clover project for downloaders.'
 # arg_name 'Describe arguments to new here'
 command :generate_downloaders do |c|
@@ -106,7 +115,8 @@ command :project do |c|
       when nil
         fail "Empty project not supported now"
     end
-    GoodData::CloverGenerator.connect_to_gd
+    
+    GoodData::CloverGenerator.connect_to_gd()
     with_users  = options[:with_users]
     
     export = {
@@ -184,7 +194,7 @@ command :scaffold do |c|
 
   c.action do |global_options,options,args|
     command = args.first
-    fail "You did not provide what I should scaffold. I can generate project, tap, flow, sink nothing else" unless ["project", "tap", "flow", "sink"].include?(command) 
+    fail "You did not provide what I should scaffold. I can generate project, tap, flow, sink, graph_template nothing else" unless ["project", "tap", "flow", "sink", "graph_template"].include?(command) 
     case command
     when "project"
       directory = args[1]
@@ -209,6 +219,12 @@ command :scaffold do |c|
       name = args[1]
       fail "Name of the sink has to be provided as an argument. See help" if name.nil?
       GoodData::CloverGenerator.setup_sink(name)
+    when "graph_template"
+      name = args[1]
+      target = args[2]
+      fail "Name of the template has to be provided as an argument. See help" if name.nil?
+      fail "Name of the target has to be provided as an argument. See help" if target.nil?
+      GoodData::CloverGenerator.generate_graph_template(name, target)
     end
   end
 end
@@ -262,13 +278,16 @@ command :run do |c|
 
     verbose = global_options[:v]
 
-    GoodData::CloverGenerator.connect_to_gd
+    logger = Logger.new(STDOUT) if global_options[:l]
+
+
+    GoodData::CloverGenerator.connect_to_gd(:logger => logger)
     options = global_options.merge({:name => "temporary"})
     GoodData::CloverGenerator.deploy(dir, options) do |deploy_response|
       puts HighLine::color("Executing", HighLine::BOLD) if verbose
       GoodData::CloverGenerator.create_email_channel(options) do |channel_response|
-        GoodData::CloverGenerator.subscribe_on_finish(:success, channel_response["channelConfiguration"]["meta"]["uri"], deploy_response["cloverTransformation"]["links"]["self"].split('/').last)
-        GoodData::CloverGenerator.execute_process(deploy_response["cloverTransformation"]["links"]["executions"], dir)
+        GoodData::CloverGenerator.subscribe_on_finish(:success, channel_response["channelConfiguration"]["meta"]["uri"], deploy_response["process"]["links"]["self"].split('/').last)
+        GoodData::CloverGenerator.execute_process(deploy_response["process"]["links"]["executions"], dir)
       end
     end
   end

data/lib/bam/version.rb

@@ -1,3 +1,3 @@
 module Bam
-  VERSION = '0.0.6'
+  VERSION = '0.0.7'
 end

data/lib/graphs/process_forecast.grf

@@ -0,0 +1,20 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<Graph author="fluke" created="Tue Feb 05 15:38:24 PST 2013" guiVersion="3.3.2" id="1360179808937" licenseType="Commercial" modified="Thu May 02 12:37:08 PDT 2013" modifiedBy="gdc-defectivedisplay" name="process_name" revision="1.14" showComponentDetails="true">
+<Global>
+<Metadata fileURL="${PROJECT}/metadata/${FLOW}/${NAME}/1_in.xml" id="Metadata0"/>
+<Metadata fileURL="${PROJECT}/metadata/${FLOW}/${NAME}/1_out.xml" id="Metadata1"/>
+<MetadataGroup id="ComponentGroup0" name="metadata"/>
+<Property fileURL="params.txt" id="GraphParameter0"/>
+<Property fileURL="workspace.prm" id="GraphParameter2"/>
+<Dictionary/>
+</Global>
+<Phase number="0">
+<Node enabled="enabled" fileURL="data/1_in.csv" guiHeight="77" guiName="CSV Reader" guiWidth="128" guiX="37" guiY="169" id="DATA_READER0" quoteCharacter="&quot;" quotedStrings="true" skipRows="1" type="DATA_READER"/>
+<Node enabled="enabled" fileURL="data/out.csv" guiHeight="77" guiName="CSV Writer" guiWidth="128" guiX="609" guiY="169" id="DATA_WRITER0" outputFieldNames="true" quoteCharacter="&quot;" quotedStrings="true" type="DATA_WRITER"/>
+<Node dedupKey="ForecastCategoryName(a)" enabled="enabled" guiHeight="65" guiName="Dedup" guiWidth="128" guiX="368" guiY="160" id="DEDUP0" type="DEDUP"/>
+<Node enabled="enabled" guiHeight="89" guiName="ExtSort" guiWidth="128" guiX="199" guiY="175" id="EXT_SORT0" sortKey="ForecastCategoryName(a)" type="EXT_SORT"/>
+<Edge fromNode="DATA_READER0:0" guiBendpoints="" guiRouter="Manhattan" id="Edge0" inPort="Port 0 (in)" metadata="Metadata0" outPort="Port 0 (output)" toNode="EXT_SORT0:0"/>
+<Edge fromNode="DEDUP0:0" guiBendpoints="" guiRouter="Manhattan" id="Edge1" inPort="Port 0 (in)" metadata="Metadata1" outPort="Port 0 (unique)" toNode="DATA_WRITER0:0"/>
+<Edge fromNode="EXT_SORT0:0" guiBendpoints="" guiRouter="Manhattan" id="Edge2" inPort="Port 0 (in)" metadata="Metadata0" outPort="Port 0 (out)" toNode="DEDUP0:0"/>
+</Phase>
+</Graph>

data/lib/graphs/process_opp_records.grf

@@ -0,0 +1,84 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<Graph author="gdc-defectivedisplay" created="Sat Mar 16 12:07:29 PDT 2013" guiVersion="3.3.2" id="1363571865202" licenseCode="CLP1DGOODD71636137BY" licenseType="Commercial" modified="Thu Apr 11 13:57:16 PDT 2013" modifiedBy="Jagjit" name="xxxx" revision="1.55" showComponentDetails="true">
+<Global>
+<Metadata fileURL="${PROJECT}/metadata/${FLOW}/${NAME}/1_in.xml" id="Metadata0"/>
+<Metadata fileURL="${PROJECT}/metadata/${FLOW}/${NAME}/1_out.xml" id="Metadata1"/>
+<Metadata id="Metadata2">
+<Record fieldDelimiter="|" name="recordName1" recordDelimiter="\n" type="delimited">
+<Field name="field1" type="string"/>
+</Record>
+</Metadata>
+<Property fileURL="params.txt" id="GraphParameter0"/>
+<Property fileURL="workspace.prm" id="GraphParameter2"/>
+<Dictionary/>
+</Global>
+<Phase number="0">
+<Node enabled="enabled" fileURL="data/1_in.csv" guiHeight="77" guiName="CSV Reader" guiWidth="128" guiX="83" guiY="90" id="DATA_READER0" quoteCharacter="&quot;" quotedStrings="true" skipRows="1" type="DATA_READER"/>
+<Node enabled="enabled" guiHeight="65" guiName="Reformat" guiWidth="128" guiX="282" guiY="80" id="REFORMAT1" type="REFORMAT">
+<attr name="transform"><![CDATA[//#CTL2
+
+// Transforms input record into output record.
+function integer transform() {
+	$out.0.* = $in.0.*;
+	
+	if ($in.0.Probability == null) {
+		$out.0.Probability = "0";
+	} else {
+		$out.0.Probability = toString(str2decimal($in.0.Probability)/100D);
+	}
+
+	if ($in.0.Amount == null) {
+		$out.0.Amount = "0";
+	} else {
+		$out.0.Amount = toString(round(str2decimal($in.0.Amount)*100D)/100D);
+	}
+
+	if ($in.0.CloseDate != null && $in.0.CloseDate != "" && left($in.0.CloseDate, 4) < "1950") {
+		$out.0.CloseDate = '1950-01-01';
+	}
+	
+	if ($in.0.CloseDate != null && $in.0.CloseDate != "" && left($in.0.CloseDate, 4) > "2049") {
+		$out.0.CloseDate = '2049-01-01';
+	}
+
+	if ($in.0.CreatedDate != null && $in.0.CreatedDate != "" && left($in.0.CreatedDate, 4) < "1950") {
+		$out.0.CreatedDate = '1950-01-01';
+	}
+	
+	if ($in.0.CreatedDate != null && $in.0.CreatedDate != "" && left($in.0.CreatedDate, 4) > "2049") {
+		$out.0.CreatedDate = '2049-01-01';
+	}
+
+  if ($in.0.CreatedDate != null && $in.0.CreatedDate != "" ) {
+    $out.0.CreatedDate = left($in.0.CreatedDate, 10);
+	}
+
+	return ALL;
+}
+
+// Called during component initialization.
+// function boolean init() {}
+
+// Called during each graph run before the transform is executed. May be used to allocate and initialize resources
+// required by the transform. All resources allocated within this method should be released
+// by the postExecute() method.
+// function void preExecute() {}
+
+// Called only if transform() throws an exception.
+// function integer transformOnError(string errorMessage, string stackTrace) {}
+
+// Called during each graph run after the entire transform was executed. Should be used to free any resources
+// allocated within the preExecute() method.
+// function void postExecute() {}
+
+// Called to return a user-defined error message when an error occurs.
+// function string getMessage() {}
+]]></attr>
+</Node>
+<Edge fromNode="DATA_READER0:0" guiBendpoints="" guiRouter="Manhattan" id="Edge1" inPort="Port 0 (in)" metadata="Metadata0" outPort="Port 0 (output)" toNode="REFORMAT1:0"/>
+<Edge fromNode="REFORMAT1:0" guiBendpoints="" guiRouter="Manhattan" id="Edge0" inPort="Port 0 (in)" metadata="Metadata1" outPort="Port 0 (out)" toNode="DATA_WRITER0:0"/>
+</Phase>
+<Phase number="1">
+<Node enabled="enabled" fileURL="data/out.csv" guiHeight="77" guiName="CSV Writer" guiWidth="128" guiX="517" guiY="74" id="DATA_WRITER0" outputFieldNames="true" quotedStrings="true" type="DATA_WRITER"/>
+</Phase>
+</Graph>

data/lib/runtime.rb

@@ -55,12 +55,15 @@ module GoodData
       end
     end
 
-    def self.setup_clover_project(base)
+    def self.setup_clover_project(base, options={})
+
+      name = options[:name] || PARAMS[:project_name]
+
       [PROJECT_GRAPHS_ROOT, PROJECT_METADATA_ROOT, PROJECT_DATA_ROOT].each do |dir|
         FileUtils::mkdir_p base + dir
       end
       FileUtils::touch(base + 'params.txt')
-      render_template("project.erb", PARAMS.merge(:project_name => "downloader-#{PARAMS[:project_name]}"), :to_file => base + '.project')
+      render_template("project.erb", PARAMS.merge(:project_name => name), :to_file => base + '.project')
       render_template("workspace.prm.erb", PARAMS, :to_file => base + 'workspace.prm')
     end
 
@@ -138,6 +141,11 @@ module GoodData
       pp report
     end
 
+    def self.sf_jack_in
+      client = get_sf_client(PARAMS)
+      binding.pry
+    end
+
     def self.generate_docs
       project = build_project
       sources = project.get_sources
@@ -209,7 +217,7 @@ module GoodData
     end
 
     def self.generate_downloaders(options={})
-      setup_clover_project(CLOVER_DOWNLOADERS_ROOT)
+      setup_clover_project(CLOVER_DOWNLOADERS_ROOT, :name => "downloaders-#{PARAMS[:project_name]}")
       project = build_project
       sources = project.get_sources
       sf_sources = sources.find_all {|tap| tap[:source] == "salesforce" && tap[:incremental] == true}
@@ -226,16 +234,16 @@ module GoodData
 
     def self.execute_process(link, dir)
       result = GoodData.post(link, {
-        :graphExecution => {
+        :execution => {
          :graph => "./#{dir}/graphs/main.grf",
          :params => {}  
         }
       })
-      GoodData.poll(result, "cloverExecutionTask")
+      GoodData.poll(result, "executionTask")
     end
 
-    def self.connect_to_gd
-        GoodData.logger = Logger.new(STDOUT)
+    def self.connect_to_gd(options={})
+        GoodData.logger = options[:logger]
         GoodData.connect(PARAMS[:gd_login], PARAMS[:gd_pass])
         GoodData.project = PARAMS[:project_pid] if !PARAMS[:project_pid].nil? && !PARAMS[:project_pid].empty?
     end
@@ -333,7 +341,7 @@ module GoodData
         process_id = options[:process]
 
         data = {
-            :cloverTransformation => {
+            :process => {
               :name => deploy_name || "#{PARAMS[:project_name]}",
               :path => "/uploads/#{File.basename(temp.path)}"
             }
@@ -355,7 +363,7 @@ module GoodData
           res = deploy_graph(dir, options)
           block.call(res)
         ensure
-          self_link = res["cloverTransformation"]["links"]["self"]
+          self_link = res["process"]["links"]["self"]
           GoodData.delete(self_link)
         end
       else
@@ -363,11 +371,16 @@ module GoodData
       end
     end
 
+    def self.generate_graph_template(name, target)
+      template_name = "#{name}_template.grf.erb"
+      render_template(template_name, PARAMS, :to_file => USER_DEFINED_GRAPHS_ROOT + target)
+    end
+
 
     def self.run(options)
 
       only_flow = options[:only]
-      setup_clover_project(CLOVER_PROJECT_ROOT)
+      setup_clover_project(CLOVER_PROJECT_ROOT, :name => "etl-#{PARAMS[:project_name]}")
       p = build_project
       sources   = p.get_sources
       datasets  = p.get_datasets

data/templates/reformat_template.grf.erb

@@ -0,0 +1,46 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<Graph author="fluke" created="Tue Feb 05 15:38:24 PST 2013" guiVersion="3.3.1" id="1360179808937" licenseType="Commercial" modified="Fri Feb 22 13:36:18 PST 2013" modifiedBy="fluke" name="process_name" revision="1.12" showComponentDetails="true">
+<Global>
+<Metadata fileURL="${PROJECT}/metadata/${FLOW}/${NAME}/1_in.xml" id="Metadata0"/>
+<Metadata fileURL="${PROJECT}/metadata/${FLOW}/${NAME}/1_out.xml" id="Metadata1"/>
+<MetadataGroup id="ComponentGroup0" name="metadata"/>
+<Property fileURL="params.txt" id="GraphParameter0"/>
+<Property fileURL="workspace.prm" id="GraphParameter1"/>
+<Dictionary/>
+</Global>
+<Phase number="0">
+<Node enabled="enabled" fileURL="data/1_in.csv" guiHeight="77" guiName="CSV Reader" guiWidth="128" guiX="124" guiY="169" id="DATA_READER0" quoteCharacter="&quot;" quotedStrings="true" skipRows="1" type="DATA_READER"/>
+<Node enabled="enabled" fileURL="data/out.csv" guiHeight="77" guiName="CSV Writer" guiWidth="128" guiX="609" guiY="169" id="DATA_WRITER0" outputFieldNames="true" quoteCharacter="&quot;" quotedStrings="true" type="DATA_WRITER"/>
+<Node enabled="enabled" guiHeight="65" guiName="Reformat" guiWidth="128" guiX="365" guiY="175" id="REFORMAT0" type="REFORMAT">
+<attr name="transform"><![CDATA[//#CTL2
+
+// Transforms input record into output record.
+function integer transform() {
+	$out.0.* = $in.0.*;
+
+	return ALL;
+}
+
+// Called during component initialization.
+// function boolean init() {}
+
+// Called during each graph run before the transform is executed. May be used to allocate and initialize resources
+// required by the transform. All resources allocated within this method should be released
+// by the postExecute() method.
+// function void preExecute() {}
+
+// Called only if transform() throws an exception.
+// function integer transformOnError(string errorMessage, string stackTrace) {}
+
+// Called during each graph run after the entire transform was executed. Should be used to free any resources
+// allocated within the preExecute() method.
+// function void postExecute() {}
+
+// Called to return a user-defined error message when an error occurs.
+// function string getMessage() {}
+]]></attr>
+</Node>
+<Edge fromNode="DATA_READER0:0" guiBendpoints="" guiRouter="Manhattan" id="Edge0" inPort="Port 0 (in)" metadata="Metadata0" outPort="Port 0 (output)" toNode="REFORMAT0:0"/>
+<Edge fromNode="REFORMAT0:0" guiBendpoints="" guiRouter="Manhattan" id="Edge1" inPort="Port 0 (in)" metadata="Metadata1" outPort="Port 0 (out)" toNode="DATA_WRITER0:0"/>
+</Phase>
+</Graph>

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: gd_bam
 version: !ruby/object:Gem::Version
-  version: 0.0.6
+  version: 0.0.7
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-04-30 00:00:00.000000000 Z
+date: 2013-05-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -258,7 +258,7 @@ dependencies:
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 0.5.10
+        version: 0.5.11
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
@@ -266,7 +266,7 @@ dependencies:
     requirements:
     - - '='
       - !ruby/object:Gem::Version
-        version: 0.5.10
+        version: 0.5.11
 - !ruby/object:Gem::Dependency
   name: highline
   requirement: !ruby/object:Gem::Requirement
@@ -350,6 +350,8 @@ files:
 - lib/graphs/process_activity.grf
 - lib/graphs/process_activity_dim.grf
 - lib/graphs/process_activity_owner.grf
+- lib/graphs/process_forecast.grf
+- lib/graphs/process_opp_records.grf
 - lib/graphs/process_opportunity.grf
 - lib/graphs/process_opportunity_line_item.grf
 - lib/graphs/process_opportunity_snapshot.grf
@@ -368,6 +370,7 @@ files:
 - templates/flow.rb.erb
 - templates/params.json.erb
 - templates/project.erb
+- templates/reformat_template.grf.erb
 - templates/source.json.erb
 - templates/tap.json.erb
 - templates/update_dataset.script.erb
@@ -392,18 +395,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
   - - ! '>='
     - !ruby/object:Gem::Version
       version: '0'
-      segments:
-      - 0
-      hash: 3491844236925090228
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
   - - ! '>='
     - !ruby/object:Gem::Version
       version: '0'
-      segments:
-      - 0
-      hash: 3491844236925090228
 requirements: []
 rubyforge_project: 
 rubygems_version: 1.8.25