caruby-tissue 1.5.4 → 1.5.5

data/History.md

@@ -1,6 +1,10 @@
 This history lists major release themes. See the GitHub Commits (https://github.com/caruby/tissue)
 for change details.
 
+1.5.5 / 2011-09-22
+------------------
+* Use migration value filter option.
+
 1.5.4 / 2011-08-16
 ------------------
 * Fixed case sensitive DB bug. Bug # 7.

data/examples/galena/README.md

@@ -39,14 +39,6 @@ Each example has a field mapping configuration in the `conf/migration` directory
 For example, the `simple.csv` input file is migrated into caTissue using the
 `simple_migration.yaml` configuration file.
 
-You can browse the source repository [example files](https://github.com/caruby/tissue/tree/master/examples/galena), where:
-
-* `conf/migration` has the mapping configuration files
-
-* `data` has the sample data
-
-* `lib/galena/tissue/migration` has the shim code
-
 Migrate the Galena `simple` example as follows:
 
 1. Open a console in the copied Galena example location.
@@ -81,9 +73,9 @@ features of the caRuby Migration utility as follows:
 
   `crtmigrate --target TissueSpecimen --mapping conf/migration/general_fields.yaml data/general.csv`
 
-* <tt>filter</tt> - a smattering of custom Ruby shim code to convert input values to caTissue values and reject an incomplete migration
+* <tt>filter</tt> - custom default, value filter, and shim code to convert input values to caTissue values and reject an incomplete migration
 
-  `crtmigrate --target TissueSpecimen --mapping conf/migration/filter_fields.yaml --shims lib/galena/tissue/migration/filter_shims.rb --bad bad.csv data/filter.csv`
+  `crtmigrate --target TissueSpecimen --mapping conf/migration/filter_fields.yaml --defaults conf/migration/filter_defaults.yaml --filters conf/migration/filter_values.yaml --shims lib/galena/tissue/migration/filter_shims.rb --bad bad.csv data/filter.csv`
 
 * <tt>frozen</tt> - storage locations
 
@@ -119,4 +111,3 @@ The following input fields are included in the examples:
 * <tt>Gleason</tt>: Specimen primary gleason score
 * <tt>Grade</tt>: Specimen histologic WHO grade
 
-The fields listed are those used in the examples. caTissue supports any caTissue domain object.

data/examples/galena/conf/migration/filter_defaults.yaml

@@ -0,0 +1 @@
+TissueSpecimen.specimen_type: Fixed Tissue

data/examples/galena/conf/migration/filter_values.yaml

@@ -0,0 +1,13 @@
+# This value filter configuration file demonstrates how to transform an input field value
+# to a caTissue value. The specimen type input field is a string which can be TRUE, FALSE
+# or missing. If the input value matches the case-insensitive regular expression /TRUE/i,
+# then the input value is transformed to the caTissue value 'Frozen Tissue'. A missing
+# input value is set to 'Fixed Tissue' by the filter_defaults.yaml configuration.
+#
+# A straight-forward value => value mapping would be expressed as:
+#   'TRUE' : 'Frozen Tissue'
+#   'FALSE' : 'Fixed Tissue'
+#
+TissueSpecimen.specimen_type:
+  !ruby/regexp '/TRUE/i' : 'Frozen Tissue'
+  !ruby/regexp '/FALSE/i' : 'Fixed Tissue'

data/examples/galena/lib/galena/tissue/migration/filter_shims.rb

@@ -1,40 +1,28 @@
 require 'uom'
 
 module CaTissue
-  # Declare the classes modified for migration.
+  # Declares the classes modified for migration.
   shims Participant, TissueSpecimen, SpecimenCollectionGroup
 
   class Participant
     # Extracts the Participant first name from the +Initials+ input field.
     def migrate_first_name(value, row)
-      # The first initial is the first "name".
       value[0, 1]
     end
 
     # Extracts the Participant last name from the +Initials+ input field.
-    #
-    # @param [String] value the input initials field
-    # @param @param [{Symbol => Object}] row the input row field => value hash
     def migrate_last_name(value, row)
-      # The last initial is the last "name".
       value[-1, 1]
     end
   end
 
   class TissueSpecimen
     # Transforms the +Frozen?+ flag input field to the caTissue specimen type +Frozen Tissue+ value.
-    #
-    # @param [String] value the input specimen type field
-    # @param @param [{Symbol => Object}] row the input row field => value hash
     def migrate_specimen_type(value, row)
       value =~ /TRUE/i ? 'Frozen Tissue' : 'Fixed Tissue'
     end
-
     # Parses the source field as a UOM::Measurement if it is a string.
     # Otherwises, returns the source value.
-    #
-    # @param [String, Numeric] value the input quantity field
-    # @param @param [{Symbol => Object}] row the input row field => value hash
     def migrate_initial_quantity(value, row)
       # if value is not a string, then use it as is
       return value unless value.is_a?(String)
@@ -45,9 +33,9 @@ module CaTissue
   end
 
   class SpecimenCollectionGroup
-    # @return [Boolean] whether this SCG has a SPN
+    # Returns whether this SCG has a SPN.
     def migration_valid?
-      not surgical_pathology_number.nil?
+      surgical_pathology_number
     end
   end
 end

data/examples/galena/lib/galena/tissue/migration/frozen_shims.rb

@@ -2,18 +2,15 @@
 require File.join(File.dirname(__FILE__), '..', 'seed', 'defaults')
 
 module CaTissue
-  # Declare the classes modified for migration.
+  # Declares the classes modified for migration.
   shims TissueSpecimen, CollectionProtocolRegistration, StorageContainer 
   
   class StorageContainer
     # Creates the migrated box in the database, if necessary.
     #
-    # @param [{Symbol => Object}] row the input row field => value hash
-    # @param [<Resource>] migrated the migrated instances
+    # @param (see CaRuby::Migratable#migrate)
     def migrate(row, migrated)
       super
-      # Fetch the box from the database, if it exists.
-      # Otherwise, create the box.
       find or create_galena_box
     end
     
@@ -34,7 +31,7 @@ module CaTissue
       # Add the box to the first open slot in the first unfilled rack in the freezer.
       frz << self
       logger.debug { "Placed the tissue box #{self} in freezer #{frz}." }
-      logger.debug { "Creating the tissue box #{self} in the database..." }
+      logger.debug { "Creating the tissue box #{self}..." }
       create
     end
   end

data/examples/galena/lib/galena/tissue/seed/defaults.rb

@@ -16,8 +16,8 @@ module Galena
   
     # Pre-defined Galena example administrative objects. If the Galena example is already set up
     # in the caTissue database, then the default object secondary key attributes can be used as a
-    # +CaRuby::Persistable.find+ template to retrieve the existing objects. Otherwise, the Defaults
-    # attributes can be created by calling +CaRuby::Persistable.create+.
+    # {CaRuby::Persistable#find} template to retrieve the existing objects. Otherwise, the Defaults
+    # attributes can be created by calling {CaRuby::Persistable#create}.
     #
     # In a real-world use case, the administrative objects are typically built in the UI before-hand.
     # In that case, it is only necessary to define the object secondary key rather than content, e.g.:

data/lib/catissue/annotation/annotatable_class.rb

@@ -11,12 +11,12 @@ module CaTissue
     #   annotation objects, or nil if this is not a primary annotation class
     attr_reader :entity_id
       
-    # @return [Class] the {Annotation::DEIntegration} proxy class (nil for 1.1 caTissue)
+    # @return [Class] the {DEIntegration} proxy class (nil for 1.1 caTissue)
     def de_integration_proxy_class
       @de_integration_proxy_class or (superclass.de_integration_proxy_class if superclass < Annotatable)
     end
     
-    # Adds +CaRuby::Domain::Metadata+ and {AnnotatableClass} functionality to the given class.
+    # Adds {CaRuby::Domain::Metadata} and {AnnotatableClass} functionality to the given class.
     #
     # @param [Class] the domain class to extend
     def self.extend_class(klass)
@@ -61,7 +61,7 @@ module CaTissue
       annotation_defined?(symbol)
     end
 
-    # Refines the +CaRuby::Domain::Attributes.toxic_attributes+ to exclude annotation attributes.
+    # Refines the {CaRuby::Domain::Attributes#toxic_attributes} to exclude annotation attributes.
     #
     # @return [<Symbol>] the non-annotation unfetched attributes
     def toxic_attributes
@@ -90,7 +90,7 @@ module CaTissue
       end
     end
 
-    # Filters +CaRuby::Domain::Attributes.loadable_attributes+ to exclude the {#annotation_attributes}
+    # Filters {CaRuby::Domain::Attributes#loadable_attributes} to exclude the {#annotation_attributes}
     # since annotation lazy-loading is not supported.
     #
     # @return (see CaRuby::Domain::Attributes#loadable_attributes)
@@ -164,17 +164,19 @@ module CaTissue
     #
     # @param [String] name the name of the annotation module
     # @param [{Symbol => Object}] opts the annotation options
-    # @option opts [String] :package the package name (default the decapitalized camelized name)
-    # @option opts [String] :service the service name (default the decapitalized underscore name)
+    # @option opts [String] :package the package name (default is the lower-case underscore name)
+    # @option opts [String] :service the service name (default is the lower-case underscore name)
+    # @option opts [String] :group the DE group short name (default is the package)
     def add_annotation(name, opts={})
       # the module symbol
       mod_sym = name.camelize.to_sym
       # the module spec defaults
-      pkg = opts[:package] ||= name.camelize(:lower)
+      pkg = opts[:package] ||= name.underscore
       svc = opts[:service] ||= name.underscore
+      grp = opts[:group] ||= pkg
       # add the annotation entry
       @ann_spec_hash[mod_sym] = opts
-      logger.info("Added #{qp} annotation #{name} with module #{mod_sym}, package #{pkg} and service #{svc}.")
+      logger.info("Added #{qp} annotation #{name} with module #{mod_sym}, package #{pkg}, service #{svc} and group #{grp}.")
     end
     
     # @return [Boolean] whether this annotatable class's annotations are loaded

data/lib/catissue/annotation/annotation_class.rb

@@ -13,6 +13,8 @@ module CaTissue
     # @return [Intger, nil] the container id, or nil if this is not a primary annotation
     attr_reader :container_id
     
+    # @param [Class] klass the {Annotation} class to enable
+    # @param [Module] mod the {AnnotationModule} which scopes the class
     def self.extend_class(klass, mod)
       # Enable the class meta-data.
       klass.extend(CaRuby::Domain::Metadata)
@@ -32,8 +34,9 @@ module CaTissue
     
     # Adds metadata to this annotation class.
     #
-    # @param [Module] mod the annotation module
+    # @param (see AnnotationClass.extend_class)
     def add_annotation_metadata(mod)
+      @domain_module = mod
       alias_attribute(:annotation_module, :domain_module)
       efcd = Annotation::EntityFacade.instance
       @entity_id = efcd.annotation_entity_id(self, false)
@@ -52,7 +55,7 @@ module CaTissue
       @ann_attrs ||= domain_attributes.compose { |attr_md| attr_md.type < Annotation }
     end
 
-    # Filters +CaRuby::Domain::Attributes.loadable_attributes+ to exclude all references,
+    # Filters {CaRuby::Domain::Attributes#loadable_attributes} to exclude all references,
     # since annotation lazy-loading is not supported.
     #
     # @return [Array] an empty array
@@ -73,7 +76,7 @@ module CaTissue
       not ref.nil?
     end
     
-    # @return [Boolean] whether this annotation is neither a {#primary?} nor a {#secondary?} annotation
+    # @return [Boolean] whether this annotation is neither a {#primary?} nor a #{secondary} annotation
     def tertiary_annotation
       not (primary? or secondary?)
     end

data/lib/catissue/annotation/annotation_module.rb

@@ -11,11 +11,14 @@ module CaTissue
 
     # @return [ProxyClass] the annotation proxy class 
     attr_reader :proxy
+    
+    # @return [String] the group short name
+    attr_reader :group
       
     # @return [Class] the hook-annotation association class, or nil for 1.1.x caTissue
     attr_reader :record_entry_class
       
-    # @return [Symbol] the {Annotation::ProxyClass} hook writer method, or nil for 1.1.x caTissue
+    # @return [Symbol] the {#de_integration_proxy_class} hook writer method, or nil for 1.1.x caTissue
     attr_reader :record_entry_hook_writer
 
     # @param [AnnotationModule] mod the annotation module to build
@@ -23,6 +26,7 @@ module CaTissue
     # @param [{Symbol => Object}] the options
     # @option opts [String] :package the DE package name
     # @option opts [String] :service the DE service name
+    # @option opts [String] :group the DE group short name
     # @option opts [String] :record_entry the record entry name class for post-1.1.x caTissue
     def self.extend_module(mod, hook, opts)
       mod.extend(self)
@@ -37,6 +41,7 @@ module CaTissue
       logger.debug { "Building #{hook.qp} annotation #{qp}..." }
       pkg = opts[:package]
       @svc_nm = opts[:service]
+      @group = opts[:group]
       # Enable the resource metadata aspect.
       md_proc = Proc.new { |klass| AnnotationClass.extend_class(klass, self) }
       CaRuby::Domain::Importer.extend_module(self, :mixin => Annotation, :metadata => md_proc, :package => pkg)
@@ -75,7 +80,7 @@ module CaTissue
     # a record entry class.
     #
     # @param [String] the record entry class name specified in the
-    #   {CaTissue::AnntatableClass#add_annotation_attribute} +:record_entry+ option
+    #   {CaTissue::AnnotatableClass#add_annotation} +:record_entry+ option
     def import_record_entry_class(klass, hook)
       @record_entry_class = const_get(klass.name.demodulize.to_sym)
       @record_entry_hook_writer = "#{hook.name.demodulize.underscore}=".to_sym

data/lib/catissue/cli/command.rb

@@ -6,7 +6,7 @@ begin
 rescue Exception => e
   logger.error("caTissue client load was unsuccessful - #{e}:\n#{e.backtrace.qp}")
   puts "caTissue client load was unsuccessful - #{e}."
-  puts "See the log at #+CaRuby::Log.instance.file+ for more information."
+  puts "See the log at #{CaRuby::Log.instance.file} for more information."
   exit 1
 end
 
@@ -15,9 +15,16 @@ require 'catissue/version'
 
 module CaTissue
   module CLI
-    # Augments +CaRuby::CLI::Command+ with caTissue-specific command line option handlers.
+    # Augments {CaRuby::CLI::Command} with caTissue-specific command line option handlers.
     class Command < CaRuby::CLI::Command
-      # @see +CaRuby::CLI::Command.initialize+
+      # 
+      # Built-in options include those specified in +CaRuby::CLI::Command.initialize+
+      # as well as the following:
+      # * --version : print the version of caRuby Tissue as well as the supported caTissue releases and exit
+      #
+      # @param [(<Symbol>, <String, Class>)] specs the arguments and options
+      #   described in +CaRuby::CLI::Command.initialize+
+      # @see {CaRuby::CLI::Command#initialize}
       def initialize(specs=[])
         specs << VERSION_OPT
         super
@@ -29,7 +36,7 @@ module CaTissue
 
       # If the version option is set, then prints the version and exits.
       # Otherwise, extracts the connection command line options, adds them
-      # to {CaTissue.access_properties}, and yields to the executor block.
+      # to the CaTissue access properties, and yields to the executor block.
       #
       # @param [{Symbol => Object}] opts the command line argument and option symbol => value hash
       def handle_options(opts)

data/lib/catissue/cli/migrate.rb

@@ -15,18 +15,18 @@ module CaTissue
       SPECS = [
         [:input, "input", "Source file to migrate"],
         [:target, "-t", "--target CLASS", "Migration target class"],
-        [:mapping, "-m", "--mapping FILE", "The input field => caTissue attribute mapping file"],
-        [:defaults, "-d", "--defaults FILE", "The caTissue attribute => default value mapping file"],
-        [:shims, "-s", "--shims FILE[,FILE...]", Array, "Migration customization shim files to load"],
+        [:mapping, "-m", "--mapping FILE[,FILE...]", Array, "The input field => caTissue attribute mapping file(s)"],
+        [:filters, "--filters FILE[,FILE...]", Array, "The input value => caTissue value mapping file(s)"],
+        [:defaults, "-d", "--defaults FILE[,FILE...]", Array, "The caTissue attribute default value file(s)"],
+        [:shims, "-s", "--shims FILE[,FILE...]", Array, "Migration customization shim file(s) to load"],
         [:bad, "-b", "--bad FILE", "Write each invalid record to the given file and continue migration"],
         [:unique, "-u", "--unique", "Make the migrated objects unique for testing"],
         [:offset, "-o", "--offset N", Integer, "Number of input records to skip before starting the migration"]
       ]
   
-      # Creates a {CaTissue::CLI::Migrate} command with the given standard command line specifications
+      # Creates a {Migrate} command with the given standard command line specifications
       # as well as the {SPECS} command line specifications.
       #
-      # @param (see CaRuby::CLI::Command#initialize)
       # @yield [opts] optional migrator factory
       # @yieldparam [{Symbol => Object}] the {CaTissue::Migrator#initialize} creation options
       # @see CaRuby::Command#run
@@ -46,17 +46,6 @@ module CaTissue
         migrator = block_given? ? yield(opts) : CaTissue::Migrator.new(opts)
         migrator.migrate_to_database
       end
-      
-      def validate(opts)
-        tgt = opts[:target]
-        if tgt.nil? then raise ArgumentError.new("Missing required migration target class option") end
-        begin
-          opts[:target] = CaTissue.const_get(tgt)
-        rescue Exception
-          logger.fatal("Could not load CaTissue class #{tgt} - #{$!}.\n#{$@.qp}")
-          raise MigrationError.new("Could not load migration target class #{tgt}")
-        end
-      end
     end
   end
 end

data/lib/catissue/database/annotation/annotation_service.rb

@@ -19,7 +19,7 @@ module CaTissue
         @intgtr = integrator
       end
 
-      # Augments the +CaRuby::PersistenceService+ create method to handle caTissue annotation
+      # Augments the {CaRuby::PersistenceService} create method to handle caTissue annotation
       # service peculiarities, e.g.:
       # * assigns the identifier, since assignment is not done automatically as is the case with the
       #   default application service
@@ -27,7 +27,7 @@ module CaTissue
       # * Save all referenced annotation objects
       #
       # This method can only be called on primary annotation objects. A _primary_ annotation
-      # is a top-level annotation which has a reference to the domain hook entity which is
+      # is a top-level annotation which has a reference to the {Annotation#hook} which is
       # being annotated.
       # 
       # @param [Annotation] annotation the annotation object to create

data/lib/catissue/database/annotation/entity_facade.rb

@@ -75,7 +75,7 @@ module CaTissue
       #   classes. Use alternative SQL instead.
       #
       # @param [Integer] eid the referencing entity id
-      # @param [String] eid the association property name
+      # @param [String] name the association property name
       # @return [Integer] the referenced {Annotation} class entity id
       def associated_entity_id(eid, name)
         # The caTissue role is capitalized.
@@ -145,7 +145,7 @@ module CaTissue
       
       private
       
-      CORE_PKG_REGEX = /^edu.wustl.catissuecore.domain/
+      CORE_PKG = 'edu.wustl.catissuecore.domain'
       
       CORE_GROUP = 'caTissueCore'
       
@@ -160,7 +160,7 @@ module CaTissue
       # @param (see #recursive_annotation_entity_id)
       # @return (see #recursive_annotation_entity_id)
       def nonrecursive_annotation_entity_id(klass)
-        # The entity group and entity name.
+        # The entity group and name.
         grp, name = split_annotation_entity_class_name(klass)
         # Dive into some obscure SQL.
         result = @executor.execute { |dbh| dbh.select_one(ANN_ENTITY_ID_SQL, grp, name) }
@@ -175,13 +175,8 @@ module CaTissue
         # the Java package and base class name
         pkg, base = Java.split_class_name(jname)
         # A wustl domain class is in the core group.
-        if pkg =~ CORE_PKG_REGEX then
-          [CORE_GROUP, jname]
-        elsif pkg.nil? or pkg['.'] then
-          raise AnnotationError.new("Entity group for Java class #{jname} could not be determined.")
-        else
-          [pkg, base]
-        end
+        # Others are in an annotation module with a designated group.
+        pkg == CORE_PKG ? [CORE_GROUP, jname] : [klass.annotation_module.group, base]
       end
       
       # @param (see #annotation_entity_id)
@@ -213,7 +208,7 @@ module CaTissue
       end
       
       # @param (see #recursive_associated_entity_id)
-      # @return [Integer, nil] the directly associated entity id, if any
+      # @return @return [Integer, nil] the directly associated entity id, if any
       def nonrecursive_associated_entity_id(eid, role)
         logger.debug { "Finding entity id #{eid} #{role} associated entity id..." }
         result = @executor.execute { |dbh| dbh.select_one(ASSN_ENTITY_ID_SQL, eid, role) }

data/lib/catissue/database/annotation/record_entry_integrator.rb

@@ -16,8 +16,9 @@ module CaTissue
       #   by creating a record entry hook proxy. Even though the record entry class
       #   includes Java properties for each primary annotation in the annotation
       #   Java package, the record entry instance only references a single annotation.
-      #   The record entry database table +dyextn_abstract_record_entry+ has foreign
-      #   key to a ABSTRACT_FORM_CONTEXT_ID  a separate record entry is created for each annotation.
+      #   The record entry database table +dyextn_abstract_record_entry+ has a foreign
+      #   key reference to the +ABSTRACT_FORM_CONTEXT_ID+. A separate record entry is
+      #   created for each annotation.
       #
       # @param [Annotatable] hook the hook entity 
       # @param [Annotation] annotation the annotation entity 
@@ -29,8 +30,7 @@ module CaTissue
       private
       
       # @param (see #associate)
-      # @return [Object] yet another association record which associates the hook to the
-      #   annotation in the {REC_ENTRY_PKG}
+      # @return [Object] yet another association record which associates the hook to the annotation
       def create_record_entry(hook, annotation)
         # the DE integration record entry class
         klass = hook.class.de_integration_proxy_class