rubysync 0.1.0 → 0.1.1

data/HISTORY.txt

@@ -1,3 +1,9 @@
+== 0.1.1 / 2007-10-29
+
+* Some streamlining of the base_pipeline in_handler and out_handler methods
+* Added documentation to the base_connector class so you get a template for building your
+own connector if you run "rubysync connector my_connector -t base"
+
 == 0.1.0 / 2007-09-26
 
 * Dropped the map_client_to_vault and map_vault_to_client methods. in_transform and out_transform now handle that

data/README.txt

@@ -12,11 +12,17 @@ Alternatively, you can run it in one-shot mode and simply sync A with B.
 You can configure RubySync to perform transformations on the data as it
 syncs. RubySync is designed both as a handy utility to pack into your
 directory management toolkit or as a fully-fledged provisioning system
-for your organisation.
+for your organization.
 
 
 == FEATURES/PROBLEMS:
-  
+
+* Event-driven synchronization (if connector supports it) with fall-back to polling
+* Ruby DSL for "configuration" style event processing
+* Clean separation of connector details from data transformation
+* Connectors available for CSV files, XML, LDAP and RDBMS (via ActiveRecord)
+* Easy API for writing your own connectors
+
 
 == SYNOPSIS:
 
@@ -40,7 +46,7 @@ for your organisation.
   You would then edit the file pipelines/my_pipeline.rb to configure the
   policy for synchronizing between the two connectors.
                                         
-  You may then execute the pipeline in one-shot mode (daemon mode is coming):
+  You may then execute the pipeline in one-shot mode:
 
     $ rubysync once my
 

data/bin/rubysync

@@ -91,7 +91,8 @@ class Controller < SimpleConsole::Controller
                      :t => :type,
                      :V => :vault,
                      :C => :client},
-         :int =>{:v => :verbose}
+         :int =>{:v => :verbose,
+	 	:d => :delay}
 
   def default
     #RDoc::usage 'Usage'
@@ -115,6 +116,7 @@ class Controller < SimpleConsole::Controller
   def start
     pipeline_name = params[:id]
     pipeline = pipeline_called pipeline_name
+    pipeline.delay = params[:delay]
     if pipeline
       pipeline.start
     else
@@ -231,21 +233,20 @@ end
 def example
 puts <<"END"
   This sets up the skeleton of a configuration for importing comma delimeted
-  text files into a database. Note, if the application happens to be a Rails
-  app then it can also export changes.
+  text files into an XML file.
 
-    $ rubysync create db_demo
+    $ rubysync create xml_demo
     $ cd db_demo
     $ rubysync connector my_csv -t csv_file
-    $ rubysync connector my_db -t active_record
+    $ rubysync connector my_xml -t xml
   
   You would then edit the files:
 
     connectors/my_csv_connector.rb   ;where to get CSV files, field names, etc
-    connectors/my_db_connector.rb    ;how to connect to your DB or Rails app.
+    connectors/my_xml_connector.rb   ;Set the path to your XML file
 
   And enter:
-    $ rubysync pipeline my -C my_csv -V my_db
+    $ rubysync pipeline my -C my_csv -V my_xml
 
   You would then edit the file pipelines/my_pipeline.rb to configure the
   policy for synchronizing between the two connectors.
@@ -348,13 +349,6 @@ class #{name.to_s.camelize}Pipeline < RubySync::Pipelines::BasePipeline
   # Should evaluate to the path for placing a new record on the client
   # out_place do
   # end
-  
-  
-  # These statements control logging. In log level 2 or higher (specify -v 2),
-  # the event will be dumped to the log at any of the places specified below.
-  # Uncomment and edit to taste. 
-  # dump_before :in_filter, :in_transform, :in_place, :out_filter, :out_place, :out_transform
-  # dump_after :in_filter, :in_transform, :in_place, :out_filter, :out_place, :out_transform
 
 
 end

data/bin/rubysync.rb

@@ -91,7 +91,8 @@ class Controller < SimpleConsole::Controller
                      :t => :type,
                      :V => :vault,
                      :C => :client},
-         :int =>{:v => :verbose}
+         :int =>{:v => :verbose,
+	 	:d => :delay}
 
   def default
     #RDoc::usage 'Usage'
@@ -115,6 +116,7 @@ class Controller < SimpleConsole::Controller
   def start
     pipeline_name = params[:id]
     pipeline = pipeline_called pipeline_name
+    pipeline.delay = params[:delay]
     if pipeline
       pipeline.start
     else
@@ -231,21 +233,20 @@ end
 def example
 puts <<"END"
   This sets up the skeleton of a configuration for importing comma delimeted
-  text files into a database. Note, if the application happens to be a Rails
-  app then it can also export changes.
+  text files into an XML file.
 
-    $ rubysync create db_demo
+    $ rubysync create xml_demo
     $ cd db_demo
     $ rubysync connector my_csv -t csv_file
-    $ rubysync connector my_db -t active_record
+    $ rubysync connector my_xml -t xml
   
   You would then edit the files:
 
     connectors/my_csv_connector.rb   ;where to get CSV files, field names, etc
-    connectors/my_db_connector.rb    ;how to connect to your DB or Rails app.
+    connectors/my_xml_connector.rb   ;Set the path to your XML file
 
   And enter:
-    $ rubysync pipeline my -C my_csv -V my_db
+    $ rubysync pipeline my -C my_csv -V my_xml
 
   You would then edit the file pipelines/my_pipeline.rb to configure the
   policy for synchronizing between the two connectors.
@@ -348,13 +349,6 @@ class #{name.to_s.camelize}Pipeline < RubySync::Pipelines::BasePipeline
   # Should evaluate to the path for placing a new record on the client
   # out_place do
   # end
-  
-  
-  # These statements control logging. In log level 2 or higher (specify -v 2),
-  # the event will be dumped to the log at any of the places specified below.
-  # Uncomment and edit to taste. 
-  # dump_before :in_filter, :in_transform, :in_place, :out_filter, :out_place, :out_transform
-  # dump_after :in_filter, :in_transform, :in_place, :out_filter, :out_place, :out_transform
 
 
 end

data/lib/ruby_sync/connectors/base_connector.rb

@@ -93,12 +93,18 @@ module RubySync::Connectors
         raise "Not implemented"
       end
 
-      # Subclasses must override this to interface with the external system
+      # Subclasses MAY override this to interface with the external system
       # and generate an event for every change that affects items within
       # the scope of this connector.
-      # todo: Make the default behaviour to build a database of the key of
-      # each entry with a hash of the contents of the entry. Then to compare
-      # that against each entry to see if it has changed.
+      #
+      # The default behaviour is to compare a hash of each entry in the
+      # database with a stored hash of its previous value and generate
+      # add, modify and delete events appropriately. This is normally a very
+      # inefficient way to operate so overriding this method is highly
+      # recommended if you can detect changes in a more efficient manner.
+      #
+      # This method will be called repeatedly until the connector is
+      # stopped.
       def each_change
         DBM.open(self.mirror_dbm_filename) do |dbm|
           # scan existing entries to see if any new or modified
@@ -359,8 +365,6 @@ module RubySync::Connectors
       end
 
 
-
-
       # Return an array of possible fields for this connector.
       # Implementations should override this to query the datasource
       # for possible fields.
@@ -379,6 +383,103 @@ module RubySync::Connectors
         '::' + "#{connector_name}_connector".camelize
       end
 
+      def self.sample_config
+          return <<END
+          # The comments in this file should help you to create a custom connector.
+          # We're going to assume that you know how to program in Ruby. If you don't then
+          # quickly pop-off and learn it: http://ruby-lang.org.
+          #
+          # Edit the comments as you go to describe the specifics of your connector.
+          # If you need more information, consult http://rubysync.org/docs/developer/connectors
+          
+          
+          # Call the option class method to declare the options used to configure
+          # your connector.
+          # eg.
+          #
+          #option :filename, :frequency
+          #
+          # Would define an option called filename and one called frequency. You could then follow up with:
+          #
+          # filename 'default.csv'
+          # frequency 10
+          #
+          # And, of course, the same could be done in child classes (aka configuration files)
+          # The value set becomes available as a readable method of the same name in instances
+          # of the class.                   
+          
+          
+          ####### Configuration methods
+          
+          # Return the list of the fields available for this connector. Feel free to print an
+          # informative message if you can't determine the available fields for the datastore.
+          def self.fields
+            puts "The author of #{__FILE__} hasn't got around to implementing the working out the default fields yet :>"
+          end
+          
+          # Return the string that will be inserted as the contents of the subclass created
+          # when "rubysync connnector blah -t your_connector" is run.
+          def self.sample_config
+            return <<-END
+              # This is the default configuration provided by #{__FILE__}
+              #
+              # Kind of sparse. Isn't it?
+              #
+              #
+            END
+          end
+
+          ####### Reading methods
+          
+          # If your datasource supports random access (as would, for example, a database) then
+          # implement the following:
+          #
+          #def [](path)
+          #  #return the entry at location indicated by 'path'
+          #  #An 'entry' is a hash where the key is the attribute name and the value is an
+          #  #array containing the value or values for the the attribute
+          #end               
+          
+          # Subclasses must override this to
+          # interface with the external system and generate entries for every
+          # entry in the scope passing the entry path (id) and its data (as a hash of arrays).
+          def each_entry
+            raise "Not implemented"
+          end
+
+          # Subclasses MAY override this to interface with the external system
+          # and generate an event for every change that affects items within
+          # the scope of this connector.
+          #
+          # The default behaviour is to compare a hash of each entry in the
+          # database with a stored hash of its previous value and generate
+          # add, modify and delete events appropriately. This is normally a very
+          # inefficient way to operate so overriding this method is highly
+          # recommended if you can detect changes in a more efficient manner.
+          #
+          # This method will be called repeatedly until the connector is
+          # stopped.
+          #def each_change
+          #end
+          
+          ######## Writing methods
+
+          
+          # Apply operations to create database a entry at path
+          def add(path, operations)
+          end
+          
+          # Apply operations to alter database entry at path
+          def modify(path, operations)
+          end
+
+
+          # Remove database entry at path
+          def delete(path)
+          end
+END
+      end
+
 private
 
       def self.options options

data/lib/ruby_sync/connectors/connector_event_processing.rb

@@ -38,7 +38,7 @@ module RubySync
         log.info "Adding '#{event.target_path}' to '#{name}'"
         raise Exception.new("#{name}: Entry with path '#{event.target_path}' already exists, add failing.") if self[event.target_path]
         if is_vault? && event.association && path_for_association(event.association)
-          raise Exception.new("#{name}: Association already in use. Add failing.") 
+          raise Exception.new("#{name}: Association (#{event.association.to_s}) already in use. Add failing.") 
         end
         call_if_exists(:target_transform, event)
         if add(event.target_path, event.payload)
@@ -87,7 +87,7 @@ module RubySync
         if entry = self[path]
           DBM.open(self.mirror_dbm_filename) do |dbm|
             dbm[path.to_s] = digest(entry)
-          end
+           end
         end
       end
       

data/lib/ruby_sync/connectors/ldap_connector.rb

@@ -60,7 +60,7 @@ module RubySync::Connectors
 
     def each_entry
       Net::LDAP.open(:host=>host, :port=>port, :auth=>auth) do |ldap|
-        ldap.search :base => search_base, :filter => search_filter do |ldap_entry|
+        ldap.search :base => search_base, :filter => search_filter, :return_result => false do |ldap_entry|
           yield ldap_entry.dn, to_entry(ldap_entry)
         end
       end
@@ -118,6 +118,7 @@ END
       with_ldap do |ldap|
         result = ldap.search :base=>path, :scope=>Net::LDAP::SearchScope_BaseObject, :filter=>'objectclass=*'
         return nil if !result or result.size == 0
+        # todo: See if this can be shortened
         answer = {}
         result[0].attribute_names.each do |name|
           answer[name.to_s] = result[0][name]

data/lib/ruby_sync/connectors/xml_connector.rb

@@ -113,8 +113,6 @@ module RubySync::Connectors
       el
     end
 
-
-
     
     def to_entry entry_element
       entry = {}

data/lib/ruby_sync/event.rb

@@ -46,7 +46,7 @@ module RubySync
     
     include RubySync::Utilities
 
-    attr_accessor :type,        # :delete, :add, :modify ...
+    attr_accessor :type,        # :delete, :add, :modify, :disassociate
                   :source,
                   :target,
                   :payload,
@@ -70,6 +70,12 @@ module RubySync
       self.new(:modify, source, source_path, association, payload)
     end
 
+    # Remove the association between the entry on the source and
+    # the associated entry (if any) on the target.
+    def self.disassociate source, source_path, association=nil, payload=nil
+      self.new(:disassociate, source, source_path, association, payload)
+    end
+
     def initialize type, source, source_path=nil, association=nil, payload=nil
       self.type = type.to_sym
       self.source = source
@@ -125,6 +131,11 @@ module RubySync
       end
     end
           
+
+    def hint
+    "(#{source.name} => #{target.name}) #{source_path}"
+    end
+          
     
     def to_yaml_properties
       %w{ @type @source_path @target_path @association @payload}
@@ -156,7 +167,18 @@ module RubySync
       subjects = subjects.flatten.collect {|s| s.to_s}
       @uncommitted_operations = uncommitted_operations.delete_if {|op| !subjects.include?(op.subject.to_s)}
     end
-    
+   
+   def delete_when_blank
+       @uncommitted_operations = uncommitted_operations.map do |op| 
+         if op.sets_blank?
+	   @type == :modify ? op.same_but_as(:delete) : nil
+	 else
+	   op
+	 end
+       end.compact
+   end    
+       
+     
      # Add a value to a given subject unless it already sets a value
      def add_default field_name, value
        add_value(field_name.to_s, value) unless sets_value? field_name.to_s

data/lib/ruby_sync/operation.rb

@@ -82,6 +82,7 @@ module RubySync
     def same_but_as type
       op = self.dup
       op.type = type
+      op.values = nil if type == :delete
       op
     end
     
@@ -93,6 +94,10 @@ module RubySync
       op
     end
     
+    def sets_blank?
+      [:add, :replace].include? @type and
+      (!@values || as_array(@values).select {|v| v && v != ''}.empty?)
+    end
     
   end
 end

data/lib/ruby_sync/pipelines/base_pipeline.rb

@@ -80,10 +80,12 @@ module RubySync
 
       def self.in_transform(&blk) event_method :in_transform,&blk; end
       def self.out_transform(&blk) event_method :out_transform,&blk; end
-      def self.in_match(&blk) event_method :in_match_if,&blk; end
-      def self.out_match(&blk) event_method :out_match_if,&blk; end
-      def self.in_create_if(&blk) event_method :in_create_if,&blk; end
-      def self.out_create_if(&blk) event_method :out_create_if,&blk; end
+      def self.in_match(&blk) event_method :in_match,&blk; end
+      def self.out_match(&blk) event_method :out_match,&blk; end
+      def self.in_create(&blk) event_method :in_create,&blk; end
+      def self.out_create(&blk) event_method :out_create,&blk; end
+      def self.in_place(&blk) event_method :in_place,&blk; end
+      def self.out_place(&blk) event_method :out_place,&blk; end
 
       def self.event_method name,&blk
         define_method name do |event|
@@ -91,23 +93,17 @@ module RubySync
         end
       end
 
-      def self.in_place(&blk) place :in, &blk; end
-      def self.out_place(&blk) place :out, &blk; end
-
-      def self.place direction, &blk
-        define_method "#{direction}_place" do |event|
-          event.target_path = event.instance_eval &blk
-        end
+      def in_match(event)
+        log.debug "Default matching rule - vault[in_place] exists?"
+        path = in_place(event)
+        vault.respond_to?('[]') and vault[path] and path
       end
       
-      
-      # Override to implement some kind of matching
-      def default_match event
-        log.debug "Default matching rule - source path exists on client?"
-        event.target.respond_to?('[]') and event.target[event.source_path]
+      def out_match(event)
+        log.debug "Default matching rule - client[out_place] exists?"
+        path = out_place(event)
+        client.respond_to?('[]') and client[path] and path
       end
-      alias_method :in_match, :default_match
-      alias_method :out_match, :default_match
       
       # Override to restrict creation on the client
       def default_create event
@@ -117,15 +113,21 @@ module RubySync
       alias_method :in_create, :default_create
       alias_method :out_create, :default_create
       
-      
       # Override to modify the target path for creation on the client
       def default_place(event)
-        log.debug "Default placement rule target_path = source_path"
-        event.target_path = event.source_path
+        log.debug "Default placement: same as source_path"
+        event.source_path
       end
       alias_method :in_place, :default_place
       alias_method :out_place, :default_place
       
+      def in_place_transform(event)
+        event.target_path = in_place(event)
+      end
+      
+      def out_place_transform(event)
+        event.target_path = out_place(event)
+      end
       
       def perform_transform name, event, hint=""
 	      log.info "Performing #{name}"
@@ -197,43 +199,55 @@ module RubySync
       def out_event_filter(event);  true;  end
       
       # Called by the 'in' connector in the 'in' thread to process events generated by the client.
+      # Note: The client can't really know whether the event is an add or a modify because it doesn't store
+      # the association.
       def in_handler(event)
         event.target = @vault
         event.retrieve_association(association_context)
 
-        hint = "(#{client.name} => #{vault.name}) #{event.source_path}"
-        log.info "Processing incoming #{event.type} event "+hint
-        perform_transform :in_filter, event, hint
-        
-        # The client can't really know whether its an add or a modify because it doesn't store
-        # the association.
-        if event.type == :modify
-          unless event.associated? and vault.find_associated(event.association)
-	    log.info "No associated entry in vault for modify event. Converting to add"
-	    event.convert_to_add
-	  end
-        elsif event.type == :add and event.associated? and vault.find_associated(event.association)
-	  log.info "Associated entry in vault for add event. Converting to modify"
-          event.convert_to_modify
+        log.info "Processing incoming #{event.type} event "+event.hint
+        perform_transform :in_filter, event, event.hint
+
+        associated_entry = nil
+        unless event.type == :disassociate
+          associated_entry = vault.find_associated(event.association) if event.associated?
+          unless associated_entry
+            match = in_match(event)
+            if match
+              log.info("Matching entry found for unassociated event: '#{match}'. Creating association.")
+              event.association = Association.new(association_context, event.source_path)
+              vault.associate event.association, match
+              associated_entry = vault[match]
+            end
+          end          
+        end
+            
+        if associated_entry
+          if event.type == :add
+        	  log.info "Associated entry in vault for add event. Converting to modify"
+            event.convert_to_modify
+          end
+        elsif event.type == :modify
+      	    log.info "No associated entry in vault for modify event. Converting to add"
+      	    event.convert_to_add 
         end
 
-        perform_transform :in_transform, event, hint
+        perform_transform :in_transform, event, event.hint
 
-        
-	      # todo: Maybe we should merge any add or modify that is associated or matched
-        if event.type == :add
-          match = in_match(event)
-          if match
-	    log.info "Matching record found in vault. Merging."
-            event.merge(match)
-            log.info "---\n"; return
-          end
-          
+        case event.type
+        when :add
           if in_create(event)
-            perform_transform :in_place, event, hint
-	    log.info "Create on vault allowed. Placing at #{event.target_path}"
+            perform_transform :in_place_transform, event, event.hint
+      	    log.info "Create on vault allowed. Placing at #{event.target_path}"
           else
-	    log.info "Create rule disallowed creation"
+      	    log.info "Create rule disallowed creation"
+            log.info "---\n"; return
+          end
+        when :modify
+          event.merge(associated_entry)
+        else
+          unless event.associated?
+      	    log.info "No associated entry in vault for #{event.type} event. Dropping"
             log.info "---\n"; return
           end
         end
@@ -243,73 +257,133 @@ module RubySync
         
       end
       
-      
-          # Called by the identity-vault connector in the 'out' thread to process events generated
-          # by the identity vault.
-          def out_handler(event)
-            event.target = @client
-            event.retrieve_association(association_context)
-            event.convert_to_modify if event.associated? and event.type == :add
-
-            hint = "(path=#{event.source_path} #{vault.name} => #{client.name})"
-            log.info "Processing out-going #{event.type} event #{hint}"
-            #log.info YAML.dump(event)
-            unless out_event_filter event
-    	  log.info "Disallowed by out_event_filter"
-              log.info "---\n"; return
-    	end
-
-            # Remove unwanted attributes
-            perform_transform :out_filter, event
+      # Called by the 'in' connector in the 'in' thread to process events generated by the client.
+      # Note: The client can't really know whether the event is an add or a modify because it doesn't store
+      # the association.
+      def out_handler(event)
+        event.target = @client
+        event.retrieve_association(association_context)
 
-            unless event.associated?
-    	  log.info "no association"
-              if [:delete, :remove_association].include? event.type
-                log.info "#{name}: No action for #{event.type} of unassociated entry"
-                log.info "---\n"; return
-              end
-            end
+        log.info "Processing outgoing #{event.type} event "+ event.hint
+        perform_transform :out_filter, event, event.hint
 
-            if event.type == :modify
-              unless event.associated? and client.has_entry_for_key?(event.association.key)
-    	  log.info "Can't find associated client record so converting modify to add"
-                event.convert_to_add
-              end
+        associated_entry = nil
+        unless event.type == :disassociate
+          associated_entry = client.entry_for_own_association_key(event.association.key) if event.associated?
+          unless associated_entry
+            match = out_match(event)
+            if match
+              log.info("Matching entry found for unassociated event: '#{match}'. Creating association.")
+              event.association = Association.new(association_context, match)
+              vault.associate event.association, event.source_path
+              associated_entry = client[match]
             end
+          end          
+        end
+            
+        if associated_entry
+          if event.type == :add
+        	  log.info "Associated entry in client for add event. Converting to modify"
+            event.convert_to_modify
+          end
+        elsif event.type == :modify
+      	    log.info "No associated entry in client for modify event. Converting to add"
+      	    event.convert_to_add 
+        end
 
-            if event.type == :add
-              match = out_match(event) 
-              log.info "Attempting to match"
-              if match # exactly one event record on the client matched
-                log.info "Match found, merging"
-                event.merge(match)
-                association = Association.new(self.association_context, match.source_path)
-                vault.associate asssociation, event.source_path
-                log.info "---\n"; return
-              end
-              log.info "No match found, creating"
-              unless out_create(event)
-    	    log.info "Creation denied by create rule"
-                log.info "---\n"; return
-    	  end
-              perform_transform :out_place, event
-    	  log.info "Placing new entry at #{event.target_path}"
-            end
+        perform_transform :out_transform, event, event.hint
 
-            perform_transform :out_transform, event
-            association_key = nil
-            with_rescue("#{client.name}: Processing command") do
-              association_key = client.process(event)
-            end
-            if association_key
-              association = Association.new(association_context, association_key)
-              with_rescue("#{client.name}: Storing association #{association} in vault") do
-                vault.associate(association, event.source_path)
-              end
-    	else
-    	  log.info "Client didn't return an association key"
-            end
+        case event.type
+        when :add
+          if out_create(event)
+            perform_transform :out_place_transform, event, event.hint
+      	    log.info "Create on client allowed. Placing at #{event.target_path}"
+          else
+      	    log.info "Create rule disallowed creation"
+            log.info "---\n"; return
           end
+        when :modify
+          event.merge(associated_entry)
+        else
+          unless event.associated?
+      	    log.info "No associated entry in client for #{event.type} event. Dropping"
+            log.info "---\n"; return
+          end
+        end
+
+        with_rescue("#{client.name}: Processing command") {client.process(event)}
+        log.info "---\n"
+        
+      end
+      
+      
+      # Called by the identity-vault connector in the 'out' thread to process events generated
+      # by the identity vault.
+      #       def out_handler(event)
+      #         event.target = @client
+      #         event.retrieve_association(association_context)
+      #         event.convert_to_modify if event.associated? and event.type == :add
+      # 
+      #         hint = "(path=#{event.source_path} #{vault.name} => #{client.name})"
+      #         log.info "Processing out-going #{event.type} event #{hint}"
+      #         #log.info YAML.dump(event)
+      #         unless out_event_filter event
+      #           log.info "Disallowed by out_event_filter"
+      #           log.info "---\n"; return
+      #         end
+      # 
+      #         # Remove unwanted attributes
+      #         perform_transform :out_filter, event
+      # 
+      #         unless event.associated?
+      #           log.info "no association"
+      #           if [:delete, :disassociate].include? event.type
+      #             log.info "#{name}: No action for #{event.type} of unassociated entry"
+      #             log.info "---\n"; return
+      #           end
+      #         end
+      # 
+      #         if event.type == :modify
+      #           unless event.associated? and client.has_entry_for_key?(event.association.key)
+      #           log.info "Can't find associated client record so converting modify to add"
+      #           event.convert_to_add
+      #         end
+      #       end
+      # 
+      #             if event.type == :add
+      #               match = out_match(event) 
+      #               log.info "Attempting to match"
+      #               if match # exactly one event record on the client matched
+      #                 log.info "Match found, merging"
+      #                 perform_transform :out_place, event
+      #                 event.merge(match)
+      #                 association = Association.new(self.association_context, match.source_path)
+      #                 vault.associate asssociation, event.source_path
+      #                 log.info "---\n"; return
+      #               end
+      #               log.info "No match found, creating"
+      #               unless out_create(event)
+      #     log.info "Creation denied by create rule"
+      #                 log.info "---\n"; return
+      #   end
+      #               perform_transform :out_place_transform, event
+      #       log.info "Placing new entry at #{event.target_path}"
+      #             end
+      # 
+      #             perform_transform :out_transform, event
+      #             association_key = nil
+      #             with_rescue("#{client.name}: Processing command") do
+      #               association_key = client.process(event)
+      #             end
+      #             if association_key
+      #               association = Association.new(association_context, association_key)
+      #               with_rescue("#{client.name}: Storing association #{association} in vault") do
+      #                 vault.associate(association, event.source_path)
+      #               end
+      # else
+      #   log.info "Client didn't return an association key"
+      #             end
+      #           end
       
       
       
@@ -337,6 +411,15 @@ module RubySync
           op.subject = map[op.subject] || op.subject if op.subject
         end
       end
+        
+      
+      
+      
+      # Override to perform whatever transformation on the event is required
+      #def in_transform(event); event; end
+      
+      # Convert fields in the incoming event to those used by the identity vault
+      #def in_map_schema(event); end
       
       # Specify which fields will be allowed through the incoming filter
       # If nil (the default), all fields are allowed. 

data/lib/ruby_sync.rb

@@ -26,7 +26,7 @@ require 'ruby_sync/event'
 
 
 module RubySync
-  VERSION = '0.1.0'
+  VERSION = '0.1.1'
   module Connectors
   end
   module Pipelines

data/test/tc_transformation.rb

@@ -47,10 +47,14 @@ class TransformationTestPipeline < RubySync::Pipelines::BasePipeline
    # Constant string
    map(:note) {"Created by RubySync"}
    map(:shopping) {%w/fish milk bread/}
+   # Conditional mapping
+   map(:password) {value_for(:givenName)} if type == :add
   end
   
   in_place { "#{self.source_path}/path/in/vault"}
   out_place { "#{self.source_path}".split('/')[0] }
+  
+  dump_after :in_transform
 end
 
 class TcTransformation < Test::Unit::TestCase
@@ -67,13 +71,22 @@ class TcTransformation < Test::Unit::TestCase
   
   def test_transform
     @client[client_path] = @bob_details
+    assert_nil @vault.path_for_association(RubySync::Association.new(@pipeline.association_context, client_path)), "Association appears on vault before sync. Strange."
     @pipeline.run_once
+    assert_not_nil @vault.path_for_association(RubySync::Association.new(@pipeline.association_context, client_path)), "Association doesn't appear to have been created on vault"
     assert_not_nil @vault[vault_path],"Bob wasn't created on the vault"
     assert_equal @bob_details['givenName'], @vault[vault_path]['first_name']
     assert_equal @bob_details['sn'], @vault[vault_path]['last_name']
     assert_equal "Created by RubySync", @vault[vault_path]['note'][0]
     assert_equal "music:makeup", @vault[vault_path]['hobbies'][0]
     assert_equal %w/fish milk bread/, @vault[vault_path]['shopping']
+    assert_equal @bob_details['givenName'], @vault[vault_path]['password']
+    @vault[vault_path]['password'] = new_password = 'myNewPassword'
+    assert_equal new_password, @vault[vault_path]['password']
+    @client[client_path]['givenName'] = new_given_name = 'Mary'
+    @pipeline.run_once
+    assert_equal [new_given_name], @vault[vault_path]['first_name']
+    assert_equal new_password, @vault[vault_path]['password']
   end
   
 end

metadata

@@ -1,17 +1,17 @@
 --- !ruby/object:Gem::Specification 
-rubygems_version: 0.9.0
+rubygems_version: 0.9.4
 specification_version: 1
 name: rubysync
 version: !ruby/object:Gem::Version 
-  version: 0.1.0
-date: 2007-09-26 00:00:00 +08:00
+  version: 0.1.1
+date: 2007-10-29 00:00:00 +09:00
 summary: Event driven identity synchronization engine
 require_paths: 
 - lib
 email: ritchiey@gmail.com
 homepage: "    by Ritchie Young"
 rubyforge_project: rubysync
-description: "You can configure RubySync to perform transformations on the data as it syncs. RubySync is designed both as a handy utility to pack into your directory management toolkit or as a fully-fledged provisioning system for your organisation.  == FEATURES/PROBLEMS:   == SYNOPSIS:  This sets up the skeleton of a configuration for importing comma delimited text files into a database. Note, if the application happens to be a Rails app then it can also export changes."
+description: "You can configure RubySync to perform transformations on the data as it syncs. RubySync is designed both as a handy utility to pack into your directory management toolkit or as a fully-fledged provisioning system for your organization.  == FEATURES/PROBLEMS:  * Event-driven synchronization (if connector supports it) with fall-back to polling * Ruby DSL for \"configuration\" style event processing * Clean separation of connector details from data transformation * Connectors available for CSV files, XML, LDAP and RDBMS (via ActiveRecord) * Easy API for writing your own connectors  == SYNOPSIS:"
 autorequire: 
 default_executable: 
 bindir: bin
@@ -25,6 +25,28 @@ required_ruby_version: !ruby/object:Gem::Version::Requirement
 platform: ruby
 signing_key: 
 cert_chain: 
+- |
+  -----BEGIN CERTIFICATE-----
+  MIIDMjCCAhqgAwIBAgIBADANBgkqhkiG9w0BAQUFADA/MREwDwYDVQQDDAhyaXRj
+  aGlleTEVMBMGCgmSJomT8ixkARkWBWdtYWlsMRMwEQYKCZImiZPyLGQBGRYDY29t
+  MB4XDTA3MTAyOTA0MTI0MloXDTA4MTAyODA0MTI0MlowPzERMA8GA1UEAwwIcml0
+  Y2hpZXkxFTATBgoJkiaJk/IsZAEZFgVnbWFpbDETMBEGCgmSJomT8ixkARkWA2Nv
+  bTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAK7xOC2DjOT9aUE/I0MM
+  71CXMM2vMldkLLT0gZo/85I1sTSEBFua4up/ad9j+kr0ymtdhWC/ulKnnub/GtEO
+  OFRxTDKUGKeoYDQgOZ+UqcWuyjW2DUn6mOxQrQClUa/s7REg4cQyCBusQ6rMxFn8
+  zhOq7IR1ZPkbPB4HcSrXh+3/S2iy2LkfQndt3051cEDX5AilcLPy6KcG60VS84N5
+  E5JBDZIj/y56Ve2WdRwvbL8Od5uGYUftW17K6yjz3stXG/Li4I93t7WU4Bp34Rkd
+  Mfx90RY+smkRp34zPf7rtlY77dnQP/3HRxghft5dnfWpHjJpMkEe2r6x0wAAF4L2
+  MvUCAwEAAaM5MDcwCQYDVR0TBAIwADALBgNVHQ8EBAMCBLAwHQYDVR0OBBYEFK42
+  S/x8c9AqJK9Ak8nb9C7fsh9EMA0GCSqGSIb3DQEBBQUAA4IBAQApFEzful/Wtetr
+  pfFP5+VENNzNpXMP0NDP/FRYvEkf10kF3g3FRbT9wnrRAqwupBRNP+LRMywE6pXb
+  CGP1uhA2ElYCPjKl/HgmHqRk7+dGpcXg/wlC/RpFGf1k9AB//akBBrAwGcvMeOv7
+  CDBOYma/WXx6kL/nyYQAWqJlHB3aBUjD9OXbKbSuHV6v7G23YugwtubMJbOCCIr9
+  5wkZH0Ih6BfsOnuGCd6Gfada2L8nxV3c4XMiOjg53EdpauP5RD/bxbSlr0XDN7Bh
+  moz1RHd78JYoT3J7s8pd1KdyhAhtgRJRuMPT5/Lk2edSgLIP/XOhFqQjs8GTKSG0
+  +lEskbtv
+  -----END CERTIFICATE-----
+
 post_install_message: 
 authors: 
 - Ritchie Young