activefacts 1.5.3 → 1.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: d19d8c379e49d5413f9cf2bfe695b07726923c26
-  data.tar.gz: 1636d7659661aa52e48d677a52b8a0212f7ecced
+  metadata.gz: b1e7f3153a8b937d166b295d8b8340f3294888b7
+  data.tar.gz: 74a1f5ddccb7c51b36c6d8e19791ef12841b5c44
 SHA512:
-  metadata.gz: 83c44c1d896a190aa020533f36de861a15b743db2801ba52d878a738e8c835f2d883ad189fe569e7d5d8e33c5aed2dd44078e94da1f45ac5d77d5511f400e99a
-  data.tar.gz: 099220d45e4b6d0f7539bde2aae1da1d4a0965c8754f318fa1e3836e8cd3656b53389bea62cc99cdc9c01e151dd8c24d04315c140279edb176a21db34c1b18d6
+  metadata.gz: 82841996c8072aa90e40075d787e66fa16726b1048a804301c38cd0e076d02ad963955c6d7bf255bbfbb5b12b83921db132413f6d91ac982db14e28e1528b5c9
+  data.tar.gz: 69fbf20eb22d9ec6bbddb5cc9d9366c0d2e0dced990f140542dfec12409ac550441f15acf48c972c09ece0eb52633eea1cc49cd6b42281ad9f9d5001d17b90d8

data/Manifest.txt

@@ -96,6 +96,8 @@ lib/activefacts/generate/topics.rb
 lib/activefacts/generate/traits/oo.rb
 lib/activefacts/generate/traits/ordered.rb
 lib/activefacts/generate/traits/ruby.rb
+lib/activefacts/generate/traits/datavault.rb
+lib/activefacts/generate/transform/datavault.rb
 lib/activefacts/generate/transform/surrogate.rb
 lib/activefacts/generate/version.rb
 lib/activefacts/input/cql.rb

data/README.rdoc

@@ -4,54 +4,63 @@
 
 == DESCRIPTION
 
-ActiveFacts provides a semantic modeling language, the Constellation
-Query Language (CQL).  CQL combines natural language verbalisation and
-formal logic, producing a formal language that reads like plain
-English. ActiveFacts converts semantic models from CQL to relational
-and object models in SQL, Ruby and other languages.
-
-The generated models are guaranteed congruent, which eliminates the
-object-relational impedance mismatch.  Semantic models are much more
+ActiveFacts provides a fact-based semantic modeling language, the
+Constellation Query Language (CQL).  CQL combines natural language
+verbalisation and formal logic, producing a formal language that
+reads like plain English. ActiveFacts converts semantic models from
+CQL to relational and object models in SQL, Ruby and other languages.
+
+The generated models are guaranteed congruent, which can eliminate the
+object-relational impedance mismatch.  Fact based models are much more
 stable under evolving requirements than either relational or
 object-oriented models, because they directly express the underlying
-elementary facts, so are not susceptible to ramifying change in the
-way those attribute-oriented approaches are.
+conceptual structure as elementary facts, so are not susceptible to
+ramifying change in the way those attribute-oriented approaches are.
 
 Semantic modeling is a refinement of fact-based modeling techniques
 such as ORM2, NIAM and others. ActiveFacts can convert ORM2 files from
 NORMA to CQL. Fact-based modeling is closely related to relational
-modeling in the sixth normal form, but doesn't suffer from 6NF
+modeling in the sixth normal form (as Codd intended it!), but the
+generated relation schemas are in 5NF, so they don't suffer from 6NF
 inefficiency.  The relational models it derives are highly efficient.
 
 == SYNOPSIS:
 
+  afgen --help
   afgen --sql/server myfile.cql
   afgen --ruby myfile.cql
   afgen --cql myfile.orm
+  afgen --transform/surrogate --rails/schema myfile.cql
+  afgen --transform/datavault --sql/server myfile.cql
+  cql (command-line interpreter, including a query evaluator)
 
 == INSTALL:
 
 * sudo gem install activefacts
 
-== UNIMPLEMENTED FEATURES
+== STATUS
 
- * Queries are parsed, but not yet generated to SQL.
+ * The definition language is complete and the main generators are usable.
 
- * The Constellation API lacks SQL persistence (but is already useful;
-   the CQL compiler uses the generated Ruby code extensively)
+ * Arithmetic and aggregate operations in queries are recognised but not compiled.
 
- * Validation of semantic models is incomplete
+ * Queries and derived fact types not yet generated to SQL queries or views..
+
+ * The Constellation API lacks SQL persistence; it has no Object-Relational Mapper
+   (but it's already useful; the CQL compiler uses the generated Ruby code extensively)
+
+ * Advanced constraint types are mostly ignored by the generators.
 
 == REQUIREMENTS:
 
 * Treetop parser generator
 
 * NORMA (see <http://www.ormfoundation.org/files/>), if you want to
-  use ORM (needs Visual Studio Pro edition)
+  use ORM (needs Visual Studio Pro or Community  edition)
 
 == LICENSE:
 
-Copyright (c) 2008 Clifford Heath.
+Copyright (c) 2008-2015 Clifford Heath.
 
 This software is provided 'as-is', without any express or implied warranty.
 In no event will the authors be held liable for any damages arising from the

data/examples/CQL/OilSupply.cql

@@ -16,7 +16,7 @@ Year Nr is written as Signed Integer(32);
 /*
  * Entity Types
  */
-Month is identified by its Nr restricted to {1..12};
+Month [static] is identified by its Nr restricted to {1..12};
 Month is in one Season;
 
 Product is independent identified by its Name;
@@ -36,7 +36,7 @@ Acceptable Substitution is where
 	Product may be substituted by alternate-Product in Season [acyclic, intransitive],
 	alternate-Product is an acceptable substitute for Product in Season;
 
-Supply Period is identified by Year and Month where
+Supply Period [separate, static] is identified by Year and Month where
 	Supply Period is in one Year,
 	Supply Period is in one Month;
 

data/lib/activefacts/cql/FactTypes.treetop

@@ -283,9 +283,24 @@ module ActiveFacts
 	}
       end
 
+      rule role_quantifier
+	quantifier mapping_pragmas enforcement cn:context_note?
+	{
+	  def ast
+	    Compiler::Quantifier.new(
+	      quantifier.value[0],
+	      quantifier.value[1],
+	      enforcement.ast,
+	      cn.empty? ? nil : cn.ast,
+	      mapping_pragmas.value
+	    )
+	  end
+	}
+      end
+
       # This is the rule that causes most back-tracking. I think you can see why.
       rule simple_role
-	q:( quantifier enforcement cn:context_note? )?
+	q:role_quantifier?
         player:derived_variable
         lr:(
 	  literal u:unit?
@@ -296,12 +311,7 @@ module ActiveFacts
         {
 	  def ast
 	    if !q.empty? && q.quantifier.value
-	      quantifier = Compiler::Quantifier.new(
-		q.quantifier.value[0],
-		q.quantifier.value[1],
-		q.enforcement.ast,
-		q.cn.empty? ? nil : q.cn.ast
-	      )
+	      quantifier = q.ast
 	    end
 	    if !lr.empty?
 	      if lr.respond_to?(:literal)

data/lib/activefacts/cql/ObjectTypes.treetop

@@ -167,25 +167,31 @@ module ActiveFacts
 
       rule mapping_pragmas
         '[' s h:mapping_pragma t:(s ',' s mapping_pragma)* s ']' s
-          { def value; t.elements.inject([h.value]) { |a, e| a << e.mapping_pragma.value }; end }
+          {
+	    def value
+	      t.elements.inject([h.value*' ']) do |a, e|
+		a << e.mapping_pragma.value*' '
+	      end
+	    end
+	  }
         /
         s
           { def value; []; end }
       end
 
+      # Each mapping_pragma returns an array of words
       rule mapping_pragma
 	was s names:(id s)+
 	{   # Old or previous name of an object type:
 	  def value
-	    [ was.text_value, names.elements.map{|n|n.text_value} ]
+	    [ was.text_value ] + names.elements.map{|n|n.text_value}
 	  end
 	}
 	/
 	head:id tail:(s id)*
 	{   # A sequence of one or more words denoting a pragma:
 	  def value
-	    ([head]+tail.elements.map(&:id)).
-	      map(&:text_value)*' '
+	    ([head]+tail.elements.map(&:id)).map(&:text_value)
 	  end
 	}
       end

data/lib/activefacts/cql/compiler/clause.rb

@@ -1079,6 +1079,11 @@ module ActiveFacts
                   :max_frequency => @quantifier.max,
                   :min_frequency => @quantifier.min
                 )
+	      if @quantifier.pragmas
+		@quantifier.pragmas.each do |p|
+		  constellation.ConceptAnnotation(:concept => constraint.concept, :mapping_annotation => p)
+		end
+	      end
               trace :constraint, "Made new embedded PC GUID=#{constraint.concept.guid} min=#{@quantifier.min.inspect} max=#{@quantifier.max.inspect} over #{(e = fact_type.entity_type) ? e.name : role_sequence.describe} in #{fact_type.describe}"
               @quantifier.enforcement.compile(constellation, constraint) if @quantifier.enforcement
               @embedded_presence_constraint = constraint
@@ -1097,13 +1102,15 @@ module ActiveFacts
       class Quantifier
         attr_accessor :enforcement
         attr_accessor :context_note
+        attr_accessor :pragmas
         attr_reader :min, :max
 
-        def initialize min, max, enforcement = nil, context_note = nil
+        def initialize min, max, enforcement = nil, context_note = nil, pragmas = nil
           @min = min
           @max = max
           @enforcement = enforcement
           @context_note = context_note
+	  @pragmas = pragmas
         end
 
         def is_unique

data/lib/activefacts/cql/compiler/constraint.rb

@@ -233,6 +233,11 @@ module ActiveFacts
               :is_preferred_identifier => false,
               :is_mandatory => @quantifier.min && @quantifier.min > 0
             )
+	  if @quantifier.pragmas
+	    @quantifier.pragmas.each do |p|
+	      @constellation.ConceptAnnotation(:concept => @constraint.concept, :mapping_annotation => p)
+	    end
+	  end
           @enforcement.compile(@constellation, @constraint) if @enforcement
 	  trace :constraint, "Made new PC GUID=#{@constraint.concept.guid} min=#{@quantifier.min.inspect} max=#{@quantifier.max.inspect} over #{role_sequence.describe}"
           super
@@ -385,6 +390,11 @@ module ActiveFacts
             :vocabulary => @vocabulary,
             :is_mandatory => @quantifier.min == 1
           )
+	  if @quantifier.pragmas
+	    @quantifier.pragmas.each do |p|
+	      @constellation.ConceptAnnotation(:concept => @constraint.concept, :mapping_annotation => p)
+	    end
+	  end
           @enforcement.compile(@constellation, @constraint) if @enforcement
           role_sequences.each_with_index do |role_sequence, i|
             @constellation.SetComparisonRoles(@constraint, i, :role_sequence => role_sequence)

data/lib/activefacts/cql/compiler/fact_type.rb

@@ -171,7 +171,7 @@ module ActiveFacts
             end
           end
 	  @pragmas.each do |p|
-	    @constellation.ConceptAnnotation(:concept => @fact_type.concept, :mapping_annotation => p)
+	    @constellation.ConceptAnnotation(:concept => (@entity_type||@fact_type).concept, :mapping_annotation => p)
 	  end if @pragmas
 
           @clauses.each do |clause|

data/lib/activefacts/generate/traits/datavault.rb

@@ -0,0 +1,241 @@
+#
+#	ActiveFacts Schema Transform
+#	Transform a loaded ActiveFacts vocabulary to suit ActiveRecord
+#
+# Copyright (c) 2009 Clifford Heath. Read the LICENSE file.
+#
+require 'activefacts/generate/helpers/inject'
+
+module ActiveFacts
+  module Generate
+    module DataVaultTraits
+
+      module ObjectType
+
+	def dv_add_surrogate type_name = 'Auto Counter', suffix = 'ID'
+	  # Find or assert the surrogate value type
+	  auto_counter = vocabulary.valid_value_type_name(type_name) ||
+	    constellation.ValueType(:vocabulary => vocabulary, :name => type_name, :concept => :new)
+
+	  # Create a subtype to identify this entity type:
+	  vt_name = self.name + ' '+suffix
+	  my_id = @vocabulary.valid_value_type_name(vt_name) ||
+	    constellation.ValueType(:vocabulary => vocabulary, :name => vt_name, :concept => :new, :supertype => auto_counter)
+
+	  # Create a fact type
+	  identifying_fact_type = constellation.FactType(:concept => :new)
+	  my_role = constellation.Role(:concept => :new, :fact_type => identifying_fact_type, :ordinal => 0, :object_type => self)
+	  self.injected_surrogate_role = my_role
+	  id_role = constellation.Role(:concept => :new, :fact_type => identifying_fact_type, :ordinal => 1, :object_type => my_id)
+
+	  # Create a reading (which needs a RoleSequence)
+	  reading = constellation.Reading(
+	    :fact_type => identifying_fact_type,
+	    :ordinal => 0,
+	    :role_sequence => [:new],
+	    :text => "{0} has {1}"
+	  )
+	  constellation.RoleRef(:role_sequence => reading.role_sequence, :ordinal => 0, :role => my_role)
+	  constellation.RoleRef(:role_sequence => reading.role_sequence, :ordinal => 1, :role => id_role)
+
+	  # Create two uniqueness constraints for the one-to-one. Each needs a RoleSequence (two RoleRefs)
+	  one_id = constellation.PresenceConstraint(
+	      :concept => :new,
+	      :vocabulary => vocabulary,
+	      :name => self.name+'HasOne'+suffix,
+	      :role_sequence => [:new],
+	      :is_mandatory => true,
+	      :min_frequency => 1,
+	      :max_frequency => 1,
+	      :is_preferred_identifier => false
+	    )
+	  @constellation.RoleRef(:role_sequence => one_id.role_sequence, :ordinal => 0, :role => my_role)
+
+	  one_me = constellation.PresenceConstraint(
+	      :concept => :new,
+	      :vocabulary => vocabulary,
+	      :name => self.name+suffix+'IsOfOne'+self.name,
+	      :role_sequence => [:new],
+	      :is_mandatory => false,
+	      :min_frequency => 0,
+	      :max_frequency => 1,
+	      :is_preferred_identifier => true
+	    )
+	  @constellation.RoleRef(:role_sequence => one_me.role_sequence, :ordinal => 0, :role => id_role)
+	end
+      end
+
+      module ValueType
+	def dv_needs_surrogate
+	  !is_auto_assigned
+	end
+
+	def dv_inject_surrogate
+	  trace :transform_surrogate, "Adding surrogate ID to Value Type #{name}"
+	  add_surrogate('Auto Counter', 'ID')
+	end
+      end
+
+      module EntityType
+	def dv_identifying_refs_from
+	  pi = preferred_identifier
+	  rrs = pi.role_sequence.all_role_ref
+
+  #	  REVISIT: This is actually a ref to us, not from
+  #	  if absorbed_via
+  #	    return [absorbed_via]
+  #	  end
+
+	  rrs.map do |rr|
+	    r = references_from.detect{|ref| rr.role == ref.to_role }
+	    unless r
+	      debugger
+	      raise "failed to find #{name} identifying reference for #{rr.role.object_type.name} in #{references_from.inspect}"
+	    end
+	    r
+	  end
+	end
+
+	def dv_needs_surrogate
+
+	  # A recursive proc to replace any reference to an Entity Type by its identifying references:
+	  trace :transform_surrogate_expansion, "Expanding key for #{name}"
+	  substitute_identifying_refs = proc do |object|
+	    if ref = object.absorbed_via
+	      # This shouldn't be necessary, but see the absorbed_via comment above.
+	      absorbed_into = ref.from
+	      trace :transform_surrogate_expansion, "recursing to handle absorption of #{object.name} into #{absorbed_into.name}"
+	      [substitute_identifying_refs.call(absorbed_into)]
+	    else
+	      irf = object.dv_identifying_refs_from
+	      trace :transform_surrogate_expansion, "Iterating for #{object.name} over #{irf.inspect}" do
+		irf.each_with_index do |ref, i|
+		  next if ref.is_unary
+		  next if ref.to_role.object_type.kind_of?(ActiveFacts::Metamodel::ValueType)
+		  recurse_to = ref.to_role.object_type
+
+		  trace :transform_surrogate_expansion, "#{i}: recursing to expand #{recurse_to.name} key in #{ref}" do
+		    irf[i] = substitute_identifying_refs.call(recurse_to)
+		  end
+		end
+	      end
+	      irf
+	    end
+	  end
+	  irf = substitute_identifying_refs.call(self)
+
+	  trace :transform_surrogate, "Does #{name} need a surrogate? it's identified by #{irf.inspect}" do
+
+	    pk_fks = dv_identifying_refs_from.map do |ref|
+	      ref.to && ref.to.is_table ? ref.to : nil
+	    end
+
+	    irf.flatten!
+
+	    # Multi-part identifiers are only allowed if:
+	    # * each part is a foreign key (i.e. it's a join table),
+	    # * there are no other columns (that might require updating) and
+	    # * the object is not the target of a foreign key:
+	    if irf.size >= 2
+	      if pk_fks.include?(nil)
+		trace :transform_surrogate, "#{self.name} needs a surrogate because its multi-part key contains a non-table"
+		return true
+	      elsif references_to.size != 0
+		trace :transform_surrogate, "#{self.name} is a join table between #{pk_fks.map(&:name).inspect} but is also an FK target"
+		return true
+	      elsif (references_from-dv_identifying_refs_from).size > 0
+		# There are other attributes to worry about
+		return true
+	      else
+		trace :transform_surrogate, "#{self.name} is a join table between #{pk_fks.map(&:name).inspect}"
+		return false
+	      end
+	      return true
+	    end
+
+	    # Single-part key. It must be an Auto Counter, or we will add a surrogate
+
+	    identifying_type = irf[0].to
+	    if identifying_type.dv_needs_surrogate
+	      trace :transform_surrogate, "#{self.name} needs a surrogate because #{irf[0].to.name} is not an AutoCounter, but #{identifying_type.supertypes_transitive.map(&:name).inspect}"
+	      return true
+	    end
+
+	    false
+	  end
+	end
+
+	def dv_inject_surrogate
+	  trace :transform_surrogate, "Injecting a surrogate key into #{self.name}"
+
+	  # Disable the preferred identifier:
+	  pi = preferred_identifier
+	  trace :transform_surrogate, "pi for #{name} was '#{pi.describe}'"
+	  pi.is_preferred_identifier = false
+	  @preferred_identifier = nil	# Kill the cache
+
+	  dv_add_surrogate
+
+	  trace :transform_surrogate, "pi for #{name} is now '#{preferred_identifier.describe}'"
+	end
+
+	def dv_add_surrogate type_name = 'Auto Counter', suffix = 'ID'
+	 # Find or assert the surrogate value type
+	 auto_counter = vocabulary.valid_value_type_name(type_name) ||
+	   constellation.ValueType(:vocabulary => vocabulary, :name => type_name, :concept => :new)
+
+	 # Create a subtype to identify this entity type:
+	 vt_name = self.name + ' '+suffix
+	 my_id = @vocabulary.valid_value_type_name(vt_name) ||
+	   constellation.ValueType(:vocabulary => vocabulary, :name => vt_name, :concept => :new, :supertype => auto_counter)
+
+	 # Create a fact type
+	 identifying_fact_type = constellation.FactType(:concept => :new)
+	 my_role = constellation.Role(:concept => :new, :fact_type => identifying_fact_type, :ordinal => 0, :object_type => self)
+	 @injected_surrogate_role = my_role
+	 id_role = constellation.Role(:concept => :new, :fact_type => identifying_fact_type, :ordinal => 1, :object_type => my_id)
+
+	 # Create a reading (which needs a RoleSequence)
+	 reading = constellation.Reading(
+	   :fact_type => identifying_fact_type,
+	   :ordinal => 0,
+	   :role_sequence => [:new],
+	   :text => "{0} has {1}"
+	 )
+	 constellation.RoleRef(:role_sequence => reading.role_sequence, :ordinal => 0, :role => my_role)
+	 constellation.RoleRef(:role_sequence => reading.role_sequence, :ordinal => 1, :role => id_role)
+
+	 # Create two uniqueness constraints for the one-to-one. Each needs a RoleSequence (two RoleRefs)
+	 one_id = constellation.PresenceConstraint(
+	     :concept => :new,
+	     :vocabulary => vocabulary,
+	     :name => self.name+'HasOne'+suffix,
+	     :role_sequence => [:new],
+	     :is_mandatory => true,
+	     :min_frequency => 1,
+	     :max_frequency => 1,
+	     :is_preferred_identifier => false
+	   )
+	 @constellation.RoleRef(:role_sequence => one_id.role_sequence, :ordinal => 0, :role => my_role)
+
+	 one_me = constellation.PresenceConstraint(
+	     :concept => :new,
+	     :vocabulary => vocabulary,
+	     :name => self.name+suffix+'IsOfOne'+self.name,
+	     :role_sequence => [:new],
+	     :is_mandatory => false,
+	     :min_frequency => 0,
+	     :max_frequency => 1,
+	     :is_preferred_identifier => true
+	   )
+	 @constellation.RoleRef(:role_sequence => one_me.role_sequence, :ordinal => 0, :role => id_role)
+
+	 return my_id
+	end
+
+      end
+
+      include ActiveFacts::TraitInjector	# Must be last in this module, after all submodules have been defined
+    end
+  end
+end

data/lib/activefacts/generate/transform/datavault.rb

@@ -0,0 +1,266 @@
+#
+#       Data Vault Transform
+#       Transform a loaded ActiveFacts vocabulary to suit Data Vault
+#
+# Copyright (c) 2015 Infinuendo. Read the LICENSE file.
+#
+require 'activefacts/vocabulary'
+require 'activefacts/persistence'
+
+require 'activefacts/generate/traits/datavault'
+
+module ActiveFacts
+
+  module Generate #:nodoc:
+    module Transform #:nodoc:
+      class DataVault
+	def initialize(vocabulary, *options)
+	  @vocabulary = vocabulary
+	  @constellation = vocabulary.constellation
+	end
+
+	def classify_tables
+	  initial_tables = @vocabulary.tables
+	  non_reference_tables = initial_tables.reject do |table|
+	    table.concept.all_concept_annotation.detect{|ca| ca.mapping_annotation == 'static'} or
+	      !table.is_a?(ActiveFacts::Metamodel::EntityType)
+	  end
+	  @reference_tables = initial_tables-non_reference_tables
+
+	  @link_tables, @hub_tables = non_reference_tables.partition do |table|
+	    identifying_references = table.identifier_columns.map{|c| c.references.first}.uniq
+	    # Which identifying_references are played by other tables?
+	    ir_tables =
+	      identifying_references.select do |r|
+		table_referred_to = r.to
+		# I have no examples of multi-level absorption, but it's possible, so loop
+		while av = table_referred_to.absorbed_via
+		  table_referred_to = av.from
+		end
+		table_referred_to.is_table
+	      end
+	    ir_tables.size > 1
+	  end
+	  trace_table_classifications
+	end
+
+	def trace_table_classifications
+	  # Trace the decisions about table types:
+	  if trace :datavault
+	    [@reference_tables, @hub_tables, @link_tables].zip(['Reference', 'Hub', 'Link']).each do |tables, kind|
+	      trace :datavault, kind+' tables: ' do
+		tables.each do |table|
+		  identifying_references = table.identifier_columns.map{|c| c.references.first}.uniq
+		  trace :datavault, "#{table.name}(#{identifying_references.map{|r| (t = r.to) && t.name || 'self'}*', '})"
+		end
+	      end
+	    end
+	  end
+	end
+
+	def detect_required_surrogates
+	  trace :datavault, "Detecting required surrogates" do
+	    @required_surrogates =
+	      (@hub_tables+@link_tables).select do |table|
+		table.dv_needs_surrogate
+	      end
+	  end
+	end
+
+	def inject_required_surrogates
+	  trace :datavault, "Injecting any required surrogates" do
+	    trace :datavault, "Need to inject surrogates into #{@required_surrogates.map(&:name)*', '}"
+	    @required_surrogates.each do |table|
+	      table.dv_inject_surrogate
+	    end
+	  end
+	end
+
+	def classify_satellite_references table
+	  identifying_references = table.identifier_columns.map{|c| c.references.first}.uniq
+	  non_identifying_references = table.columns.map{|c| c.references[0]}.uniq - identifying_references
+
+	  # Skip this table if no satellite data is needed
+	  # REVISIT: Needed anyway for a link?
+	  if non_identifying_references.size == 0
+	    return nil
+	  end
+
+	  satellites = non_identifying_references.inject({}) do |hash, ref|
+	      # Extract the declared satellite name, or use just "satellite"
+	      satellite_subname =
+		ref.fact_type.internal_presence_constraints.map do |pc|
+		  next if !pc.max_frequency || pc.max_frequency > 1 # Not a Uniqueness Constraint
+		  next if pc.role_sequence.all_role_ref.size > 1    # Covers more than one role
+		  next if pc.role_sequence.all_role_ref.single.role.object_type != table  # Not a unique attribute
+		  pc.concept.all_concept_annotation.map do |ca|
+		    if ca.mapping_annotation =~ /^satellite */
+		      ca.mapping_annotation.sub(/^satellite +/, '')
+		    else
+		      nil
+		    end
+		  end
+		end.flatten.compact.uniq[0] || "satellite"
+	      satellite_name = "#{satellite_subname}"
+	      (hash[satellite_name] ||= []) << ref
+	      hash
+	    end
+	  trace :datavault, "#{table.name} satellites are #{satellites.inspect}"
+	  satellites
+	end
+
+	def create_one_to_many(one, many, predicate_1 = 'has', predicate_2 = 'is of', one_adj = nil)
+	  # Create a fact type
+	  fact_type = @constellation.FactType(:concept => :new)
+	  one_role = @constellation.Role(:concept => :new, :fact_type => fact_type, :ordinal => 0, :object_type => one)
+	  many_role = @constellation.Role(:concept => :new, :fact_type => fact_type, :ordinal => 1, :object_type => many)
+
+	  # Create two readings
+	  reading2 = @constellation.Reading(:fact_type => fact_type, :ordinal => 0, :role_sequence => [:new], :text => "{0} #{predicate_2} {1}")
+	  @constellation.RoleRef(:role_sequence => reading2.role_sequence, :ordinal => 0, :role => many_role)
+	  @constellation.RoleRef(:role_sequence => reading2.role_sequence, :ordinal => 1, :role => one_role, :leading_adjective => one_adj)
+
+	  reading1 = @constellation.Reading(:fact_type => fact_type, :ordinal => 1, :role_sequence => [:new], :text => "{0} #{predicate_1} {1}")
+	  @constellation.RoleRef(:role_sequence => reading1.role_sequence, :ordinal => 0, :role => one_role, :leading_adjective => one_adj)
+	  @constellation.RoleRef(:role_sequence => reading1.role_sequence, :ordinal => 1, :role => many_role)
+
+	  one_id = @constellation.PresenceConstraint(
+	      :concept => :new,
+	      :vocabulary => @vocabulary,
+	      :name => one.name+'HasOne'+many.name,
+	      :role_sequence => [:new],
+	      :is_mandatory => true,
+	      :min_frequency => 1,
+	      :max_frequency => 1,
+	      :is_preferred_identifier => false
+	    )
+	  @constellation.RoleRef(:role_sequence => one_id.role_sequence, :ordinal => 0, :role => many_role)
+	  one_role
+	end
+
+	def assert_value_type name, supertype = nil
+	  @vocabulary.valid_value_type_name(name) ||
+	    @constellation.ValueType(:vocabulary => @vocabulary, :name => name, :supertype => supertype, :concept => :new)
+	end
+
+	def assert_record_source
+	  assert_value_type('Record Source', assert_value_type('String'))
+	end
+
+	def assert_date_time
+	  assert_value_type('Date Time')
+	end
+
+	# Create a PresenceConstraint with two roles, marked as preferred_identifier
+	def create_two_role_identifier(r1, r2)
+	  pc = @constellation.PresenceConstraint(
+	      :concept => :new,
+	      :vocabulary => @vocabulary,
+	      :name => r1.object_type.name+' '+r1.object_type.name+'PK',
+	      :role_sequence => [:new],
+	      :is_mandatory => true,
+	      :min_frequency => 1,
+	      :max_frequency => 1,
+	      :is_preferred_identifier => true
+	    )
+	  @constellation.RoleRef(:role_sequence => pc.role_sequence, :ordinal => 0, :role => r1)
+	  @constellation.RoleRef(:role_sequence => pc.role_sequence, :ordinal => 1, :role => r2)
+	end
+
+	def create_satellite(table, satellite_name, references)
+	  satellite_name = satellite_name.words.titlewords*' '
+	  trace :datavault, "Creating #{satellite_name} for #{table.name} with #{references.size} references" do
+	    # Create a new entity type with record-date fields in its identifier
+
+	    satellite = @constellation.EntityType(:vocabulary => @vocabulary, :name => "#{table.name} #{satellite_name}", :concept => [:new, :implication_rule => "datavault"])
+	    satellite.definitely_table
+
+	    table_role = create_one_to_many(table, satellite)
+
+	    date_time = assert_date_time
+	    date_time_role = create_one_to_many(date_time, satellite, 'is of', 'was loaded at', 'load')
+	    create_two_role_identifier(table_role, date_time_role)
+
+	    record_source = assert_record_source
+	    record_source.length = 64
+	    record_source_role = create_one_to_many(record_source, satellite, 'is of', 'was loaded from')
+
+	    # Move all roles across to it from the parent table.
+	    references.each do |ref|
+	      trace :datavault, "Moving #{ref} across to #{table.name}_#{satellite_name}" do
+		table_role = ref.fact_type.all_role.detect{|r| r.object_type == table}
+		# Reassign the role player to the satellite:
+		if table_role
+		  table_role.object_type = satellite
+		else
+		  #debugger  # Bum, the crappy Reference object bites again.
+		  $stderr.puts "REVISIT: Can't move the role for #{ref.inspect} without mangling the Reference"
+		end
+	      end
+	    end
+	  end
+	end
+
+	def generate(out = $stdout)
+	  @out = out
+
+	  # Strategy:
+	  # Determine list of ER tables
+	  # Partition tables into reference tables (annotated), link tables (two+ FKs in PK), and hub tables
+	  # For each hub and link table
+	  #   Apply a surrogate key if needed (all links, hubs lacking a simple surrogate)
+	  #   Detect references (fact types) leading to all attributes (non-identifying columns)
+	  #   Group attribute facts into satellites (use the satellite annotation if present)
+	  #   For each satellite
+	  #     Create a new entity type with a (hub-key, record-date key)
+	  #     Make new one->many fact type between hub and satellite
+	  #     Modify all attribute facts in this group to attach to the satellite
+	  # Compute a gresh relational mapping
+	  # Exclude reference tables and disable enforcement to them
+
+	  classify_tables
+
+	  detect_required_surrogates
+
+	  trace :datavault, "Creating satellites" do
+	    (@hub_tables+@link_tables).each do |table|
+	      satellites = classify_satellite_references table
+	      next unless satellites
+
+	      trace :datavault, "Creating #{satellites.size} satellites for #{table.name}" do
+		satellites.each do |satellite_name, references|
+		  create_satellite(table, satellite_name, references)
+		end
+	      end
+	    end
+	  end
+
+	  inject_required_surrogates
+
+	  trace :datavault, "Adding standard fields to hubs and links" do
+	    (@hub_tables+@link_tables).each do |table|
+	      date_time = assert_date_time
+	      date_time_role = create_one_to_many(date_time, table, 'is of', 'was loaded at', 'load')
+
+	      record_source = assert_record_source
+	      record_source_role = create_one_to_many(record_source, table, 'is of', 'was loaded from')
+	    end
+	  end
+
+	  # Now, redo the E-R mapping using the revised schema:
+	  @vocabulary.decide_tables
+
+	  # Before departing, ensure we don't emit the reference tables!
+	  @reference_tables.each do |table|
+	    table.definitely_not_table
+	    @vocabulary.tables.delete(table)
+	  end
+
+	end # generate
+
+      end
+    end
+  end
+end
+
+ActiveFacts::Registry.generator('transform/datavault', ActiveFacts::Generate::Transform::DataVault)

data/lib/activefacts/persistence/columns.rb

@@ -256,6 +256,10 @@ module ActiveFacts
         @columns =
           all_columns({})
       end
+
+      def wipe_columns
+	@columns = nil
+      end
     end
 
     # The ValueType class is defined in the metamodel; full documentation is not generated.

data/lib/activefacts/persistence/tables.rb

@@ -150,6 +150,7 @@ module ActiveFacts
                 (rr = c.role_sequence.all_role_ref.single) and
                 rr.role == self
             end
+	# REVISIT: check mapping pragmas, e.g. by to_1.concept.all_concept_annotation.detect{|ca| ca.mapping_annotation == 'separate'}
 
         if fact_type.entity_type
           # This is a role in an objectified fact type
@@ -190,6 +191,7 @@ module ActiveFacts
       def wipe_existing_mapping
         all_object_type.each do |object_type|
           object_type.clear_references
+          object_type.wipe_columns
           object_type.is_table = nil      # Undecided; force an attempt to decide
           object_type.tentative = true    # Uncertain
         end

data/lib/activefacts/version.rb

@@ -7,8 +7,8 @@
 module ActiveFacts
   module Version
     MAJOR = 1
-    MINOR = 5
-    PATCH = 3
+    MINOR = 6
+    PATCH = 0
 
     STRING = [MAJOR, MINOR, PATCH].compact.join('.')
   end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: activefacts
 version: !ruby/object:Gem::Version
-  version: 1.5.3
+  version: 1.6.0
 platform: ruby
 authors:
 - Clifford Heath
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-08-05 00:00:00.000000000 Z
+date: 2015-09-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activefacts-api
@@ -310,9 +310,11 @@ files:
 - lib/activefacts/generate/stats.rb
 - lib/activefacts/generate/text.rb
 - lib/activefacts/generate/topics.rb
+- lib/activefacts/generate/traits/datavault.rb
 - lib/activefacts/generate/traits/oo.rb
 - lib/activefacts/generate/traits/ordered.rb
 - lib/activefacts/generate/traits/ruby.rb
+- lib/activefacts/generate/transform/datavault.rb
 - lib/activefacts/generate/transform/surrogate.rb
 - lib/activefacts/generate/version.rb
 - lib/activefacts/input/cql.rb