activefacts 1.5.3 → 1.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: d19d8c379e49d5413f9cf2bfe695b07726923c26
4
- data.tar.gz: 1636d7659661aa52e48d677a52b8a0212f7ecced
3
+ metadata.gz: b1e7f3153a8b937d166b295d8b8340f3294888b7
4
+ data.tar.gz: 74a1f5ddccb7c51b36c6d8e19791ef12841b5c44
5
5
  SHA512:
6
- metadata.gz: 83c44c1d896a190aa020533f36de861a15b743db2801ba52d878a738e8c835f2d883ad189fe569e7d5d8e33c5aed2dd44078e94da1f45ac5d77d5511f400e99a
7
- data.tar.gz: 099220d45e4b6d0f7539bde2aae1da1d4a0965c8754f318fa1e3836e8cd3656b53389bea62cc99cdc9c01e151dd8c24d04315c140279edb176a21db34c1b18d6
6
+ metadata.gz: 82841996c8072aa90e40075d787e66fa16726b1048a804301c38cd0e076d02ad963955c6d7bf255bbfbb5b12b83921db132413f6d91ac982db14e28e1528b5c9
7
+ data.tar.gz: 69fbf20eb22d9ec6bbddb5cc9d9366c0d2e0dced990f140542dfec12409ac550441f15acf48c972c09ece0eb52633eea1cc49cd6b42281ad9f9d5001d17b90d8
data/Manifest.txt CHANGED
@@ -96,6 +96,8 @@ lib/activefacts/generate/topics.rb
96
96
  lib/activefacts/generate/traits/oo.rb
97
97
  lib/activefacts/generate/traits/ordered.rb
98
98
  lib/activefacts/generate/traits/ruby.rb
99
+ lib/activefacts/generate/traits/datavault.rb
100
+ lib/activefacts/generate/transform/datavault.rb
99
101
  lib/activefacts/generate/transform/surrogate.rb
100
102
  lib/activefacts/generate/version.rb
101
103
  lib/activefacts/input/cql.rb
data/README.rdoc CHANGED
@@ -4,54 +4,63 @@
4
4
 
5
5
  == DESCRIPTION
6
6
 
7
- ActiveFacts provides a semantic modeling language, the Constellation
8
- Query Language (CQL). CQL combines natural language verbalisation and
9
- formal logic, producing a formal language that reads like plain
10
- English. ActiveFacts converts semantic models from CQL to relational
11
- and object models in SQL, Ruby and other languages.
12
-
13
- The generated models are guaranteed congruent, which eliminates the
14
- object-relational impedance mismatch. Semantic models are much more
7
+ ActiveFacts provides a fact-based semantic modeling language, the
8
+ Constellation Query Language (CQL). CQL combines natural language
9
+ verbalisation and formal logic, producing a formal language that
10
+ reads like plain English. ActiveFacts converts semantic models from
11
+ CQL to relational and object models in SQL, Ruby and other languages.
12
+
13
+ The generated models are guaranteed congruent, which can eliminate the
14
+ object-relational impedance mismatch. Fact based models are much more
15
15
  stable under evolving requirements than either relational or
16
16
  object-oriented models, because they directly express the underlying
17
- elementary facts, so are not susceptible to ramifying change in the
18
- way those attribute-oriented approaches are.
17
+ conceptual structure as elementary facts, so are not susceptible to
18
+ ramifying change in the way those attribute-oriented approaches are.
19
19
 
20
20
  Semantic modeling is a refinement of fact-based modeling techniques
21
21
  such as ORM2, NIAM and others. ActiveFacts can convert ORM2 files from
22
22
  NORMA to CQL. Fact-based modeling is closely related to relational
23
- modeling in the sixth normal form, but doesn't suffer from 6NF
23
+ modeling in the sixth normal form (as Codd intended it!), but the
24
+ generated relation schemas are in 5NF, so they don't suffer from 6NF
24
25
  inefficiency. The relational models it derives are highly efficient.
25
26
 
26
27
  == SYNOPSIS:
27
28
 
29
+ afgen --help
28
30
  afgen --sql/server myfile.cql
29
31
  afgen --ruby myfile.cql
30
32
  afgen --cql myfile.orm
33
+ afgen --transform/surrogate --rails/schema myfile.cql
34
+ afgen --transform/datavault --sql/server myfile.cql
35
+ cql (command-line interpreter, including a query evaluator)
31
36
 
32
37
  == INSTALL:
33
38
 
34
39
  * sudo gem install activefacts
35
40
 
36
- == UNIMPLEMENTED FEATURES
41
+ == STATUS
37
42
 
38
- * Queries are parsed, but not yet generated to SQL.
43
+ * The definition language is complete and the main generators are usable.
39
44
 
40
- * The Constellation API lacks SQL persistence (but is already useful;
41
- the CQL compiler uses the generated Ruby code extensively)
45
+ * Arithmetic and aggregate operations in queries are recognised but not compiled.
42
46
 
43
- * Validation of semantic models is incomplete
47
+ * Queries and derived fact types not yet generated to SQL queries or views..
48
+
49
+ * The Constellation API lacks SQL persistence; it has no Object-Relational Mapper
50
+ (but it's already useful; the CQL compiler uses the generated Ruby code extensively)
51
+
52
+ * Advanced constraint types are mostly ignored by the generators.
44
53
 
45
54
  == REQUIREMENTS:
46
55
 
47
56
  * Treetop parser generator
48
57
 
49
58
  * NORMA (see <http://www.ormfoundation.org/files/>), if you want to
50
- use ORM (needs Visual Studio Pro edition)
59
+ use ORM (needs Visual Studio Pro or Community edition)
51
60
 
52
61
  == LICENSE:
53
62
 
54
- Copyright (c) 2008 Clifford Heath.
63
+ Copyright (c) 2008-2015 Clifford Heath.
55
64
 
56
65
  This software is provided 'as-is', without any express or implied warranty.
57
66
  In no event will the authors be held liable for any damages arising from the
@@ -16,7 +16,7 @@ Year Nr is written as Signed Integer(32);
16
16
  /*
17
17
  * Entity Types
18
18
  */
19
- Month is identified by its Nr restricted to {1..12};
19
+ Month [static] is identified by its Nr restricted to {1..12};
20
20
  Month is in one Season;
21
21
 
22
22
  Product is independent identified by its Name;
@@ -36,7 +36,7 @@ Acceptable Substitution is where
36
36
  Product may be substituted by alternate-Product in Season [acyclic, intransitive],
37
37
  alternate-Product is an acceptable substitute for Product in Season;
38
38
 
39
- Supply Period is identified by Year and Month where
39
+ Supply Period [separate, static] is identified by Year and Month where
40
40
  Supply Period is in one Year,
41
41
  Supply Period is in one Month;
42
42
 
@@ -283,9 +283,24 @@ module ActiveFacts
283
283
  }
284
284
  end
285
285
 
286
+ rule role_quantifier
287
+ quantifier mapping_pragmas enforcement cn:context_note?
288
+ {
289
+ def ast
290
+ Compiler::Quantifier.new(
291
+ quantifier.value[0],
292
+ quantifier.value[1],
293
+ enforcement.ast,
294
+ cn.empty? ? nil : cn.ast,
295
+ mapping_pragmas.value
296
+ )
297
+ end
298
+ }
299
+ end
300
+
286
301
  # This is the rule that causes most back-tracking. I think you can see why.
287
302
  rule simple_role
288
- q:( quantifier enforcement cn:context_note? )?
303
+ q:role_quantifier?
289
304
  player:derived_variable
290
305
  lr:(
291
306
  literal u:unit?
@@ -296,12 +311,7 @@ module ActiveFacts
296
311
  {
297
312
  def ast
298
313
  if !q.empty? && q.quantifier.value
299
- quantifier = Compiler::Quantifier.new(
300
- q.quantifier.value[0],
301
- q.quantifier.value[1],
302
- q.enforcement.ast,
303
- q.cn.empty? ? nil : q.cn.ast
304
- )
314
+ quantifier = q.ast
305
315
  end
306
316
  if !lr.empty?
307
317
  if lr.respond_to?(:literal)
@@ -167,25 +167,31 @@ module ActiveFacts
167
167
 
168
168
  rule mapping_pragmas
169
169
  '[' s h:mapping_pragma t:(s ',' s mapping_pragma)* s ']' s
170
- { def value; t.elements.inject([h.value]) { |a, e| a << e.mapping_pragma.value }; end }
170
+ {
171
+ def value
172
+ t.elements.inject([h.value*' ']) do |a, e|
173
+ a << e.mapping_pragma.value*' '
174
+ end
175
+ end
176
+ }
171
177
  /
172
178
  s
173
179
  { def value; []; end }
174
180
  end
175
181
 
182
+ # Each mapping_pragma returns an array of words
176
183
  rule mapping_pragma
177
184
  was s names:(id s)+
178
185
  { # Old or previous name of an object type:
179
186
  def value
180
- [ was.text_value, names.elements.map{|n|n.text_value} ]
187
+ [ was.text_value ] + names.elements.map{|n|n.text_value}
181
188
  end
182
189
  }
183
190
  /
184
191
  head:id tail:(s id)*
185
192
  { # A sequence of one or more words denoting a pragma:
186
193
  def value
187
- ([head]+tail.elements.map(&:id)).
188
- map(&:text_value)*' '
194
+ ([head]+tail.elements.map(&:id)).map(&:text_value)
189
195
  end
190
196
  }
191
197
  end
@@ -1079,6 +1079,11 @@ module ActiveFacts
1079
1079
  :max_frequency => @quantifier.max,
1080
1080
  :min_frequency => @quantifier.min
1081
1081
  )
1082
+ if @quantifier.pragmas
1083
+ @quantifier.pragmas.each do |p|
1084
+ constellation.ConceptAnnotation(:concept => constraint.concept, :mapping_annotation => p)
1085
+ end
1086
+ end
1082
1087
  trace :constraint, "Made new embedded PC GUID=#{constraint.concept.guid} min=#{@quantifier.min.inspect} max=#{@quantifier.max.inspect} over #{(e = fact_type.entity_type) ? e.name : role_sequence.describe} in #{fact_type.describe}"
1083
1088
  @quantifier.enforcement.compile(constellation, constraint) if @quantifier.enforcement
1084
1089
  @embedded_presence_constraint = constraint
@@ -1097,13 +1102,15 @@ module ActiveFacts
1097
1102
  class Quantifier
1098
1103
  attr_accessor :enforcement
1099
1104
  attr_accessor :context_note
1105
+ attr_accessor :pragmas
1100
1106
  attr_reader :min, :max
1101
1107
 
1102
- def initialize min, max, enforcement = nil, context_note = nil
1108
+ def initialize min, max, enforcement = nil, context_note = nil, pragmas = nil
1103
1109
  @min = min
1104
1110
  @max = max
1105
1111
  @enforcement = enforcement
1106
1112
  @context_note = context_note
1113
+ @pragmas = pragmas
1107
1114
  end
1108
1115
 
1109
1116
  def is_unique
@@ -233,6 +233,11 @@ module ActiveFacts
233
233
  :is_preferred_identifier => false,
234
234
  :is_mandatory => @quantifier.min && @quantifier.min > 0
235
235
  )
236
+ if @quantifier.pragmas
237
+ @quantifier.pragmas.each do |p|
238
+ @constellation.ConceptAnnotation(:concept => @constraint.concept, :mapping_annotation => p)
239
+ end
240
+ end
236
241
  @enforcement.compile(@constellation, @constraint) if @enforcement
237
242
  trace :constraint, "Made new PC GUID=#{@constraint.concept.guid} min=#{@quantifier.min.inspect} max=#{@quantifier.max.inspect} over #{role_sequence.describe}"
238
243
  super
@@ -385,6 +390,11 @@ module ActiveFacts
385
390
  :vocabulary => @vocabulary,
386
391
  :is_mandatory => @quantifier.min == 1
387
392
  )
393
+ if @quantifier.pragmas
394
+ @quantifier.pragmas.each do |p|
395
+ @constellation.ConceptAnnotation(:concept => @constraint.concept, :mapping_annotation => p)
396
+ end
397
+ end
388
398
  @enforcement.compile(@constellation, @constraint) if @enforcement
389
399
  role_sequences.each_with_index do |role_sequence, i|
390
400
  @constellation.SetComparisonRoles(@constraint, i, :role_sequence => role_sequence)
@@ -171,7 +171,7 @@ module ActiveFacts
171
171
  end
172
172
  end
173
173
  @pragmas.each do |p|
174
- @constellation.ConceptAnnotation(:concept => @fact_type.concept, :mapping_annotation => p)
174
+ @constellation.ConceptAnnotation(:concept => (@entity_type||@fact_type).concept, :mapping_annotation => p)
175
175
  end if @pragmas
176
176
 
177
177
  @clauses.each do |clause|
@@ -0,0 +1,241 @@
1
+ #
2
+ # ActiveFacts Schema Transform
3
+ # Transform a loaded ActiveFacts vocabulary to suit ActiveRecord
4
+ #
5
+ # Copyright (c) 2009 Clifford Heath. Read the LICENSE file.
6
+ #
7
+ require 'activefacts/generate/helpers/inject'
8
+
9
+ module ActiveFacts
10
+ module Generate
11
+ module DataVaultTraits
12
+
13
+ module ObjectType
14
+
15
+ def dv_add_surrogate type_name = 'Auto Counter', suffix = 'ID'
16
+ # Find or assert the surrogate value type
17
+ auto_counter = vocabulary.valid_value_type_name(type_name) ||
18
+ constellation.ValueType(:vocabulary => vocabulary, :name => type_name, :concept => :new)
19
+
20
+ # Create a subtype to identify this entity type:
21
+ vt_name = self.name + ' '+suffix
22
+ my_id = @vocabulary.valid_value_type_name(vt_name) ||
23
+ constellation.ValueType(:vocabulary => vocabulary, :name => vt_name, :concept => :new, :supertype => auto_counter)
24
+
25
+ # Create a fact type
26
+ identifying_fact_type = constellation.FactType(:concept => :new)
27
+ my_role = constellation.Role(:concept => :new, :fact_type => identifying_fact_type, :ordinal => 0, :object_type => self)
28
+ self.injected_surrogate_role = my_role
29
+ id_role = constellation.Role(:concept => :new, :fact_type => identifying_fact_type, :ordinal => 1, :object_type => my_id)
30
+
31
+ # Create a reading (which needs a RoleSequence)
32
+ reading = constellation.Reading(
33
+ :fact_type => identifying_fact_type,
34
+ :ordinal => 0,
35
+ :role_sequence => [:new],
36
+ :text => "{0} has {1}"
37
+ )
38
+ constellation.RoleRef(:role_sequence => reading.role_sequence, :ordinal => 0, :role => my_role)
39
+ constellation.RoleRef(:role_sequence => reading.role_sequence, :ordinal => 1, :role => id_role)
40
+
41
+ # Create two uniqueness constraints for the one-to-one. Each needs a RoleSequence (two RoleRefs)
42
+ one_id = constellation.PresenceConstraint(
43
+ :concept => :new,
44
+ :vocabulary => vocabulary,
45
+ :name => self.name+'HasOne'+suffix,
46
+ :role_sequence => [:new],
47
+ :is_mandatory => true,
48
+ :min_frequency => 1,
49
+ :max_frequency => 1,
50
+ :is_preferred_identifier => false
51
+ )
52
+ @constellation.RoleRef(:role_sequence => one_id.role_sequence, :ordinal => 0, :role => my_role)
53
+
54
+ one_me = constellation.PresenceConstraint(
55
+ :concept => :new,
56
+ :vocabulary => vocabulary,
57
+ :name => self.name+suffix+'IsOfOne'+self.name,
58
+ :role_sequence => [:new],
59
+ :is_mandatory => false,
60
+ :min_frequency => 0,
61
+ :max_frequency => 1,
62
+ :is_preferred_identifier => true
63
+ )
64
+ @constellation.RoleRef(:role_sequence => one_me.role_sequence, :ordinal => 0, :role => id_role)
65
+ end
66
+ end
67
+
68
+ module ValueType
69
+ def dv_needs_surrogate
70
+ !is_auto_assigned
71
+ end
72
+
73
+ def dv_inject_surrogate
74
+ trace :transform_surrogate, "Adding surrogate ID to Value Type #{name}"
75
+ add_surrogate('Auto Counter', 'ID')
76
+ end
77
+ end
78
+
79
+ module EntityType
80
+ def dv_identifying_refs_from
81
+ pi = preferred_identifier
82
+ rrs = pi.role_sequence.all_role_ref
83
+
84
+ # REVISIT: This is actually a ref to us, not from
85
+ # if absorbed_via
86
+ # return [absorbed_via]
87
+ # end
88
+
89
+ rrs.map do |rr|
90
+ r = references_from.detect{|ref| rr.role == ref.to_role }
91
+ unless r
92
+ debugger
93
+ raise "failed to find #{name} identifying reference for #{rr.role.object_type.name} in #{references_from.inspect}"
94
+ end
95
+ r
96
+ end
97
+ end
98
+
99
+ def dv_needs_surrogate
100
+
101
+ # A recursive proc to replace any reference to an Entity Type by its identifying references:
102
+ trace :transform_surrogate_expansion, "Expanding key for #{name}"
103
+ substitute_identifying_refs = proc do |object|
104
+ if ref = object.absorbed_via
105
+ # This shouldn't be necessary, but see the absorbed_via comment above.
106
+ absorbed_into = ref.from
107
+ trace :transform_surrogate_expansion, "recursing to handle absorption of #{object.name} into #{absorbed_into.name}"
108
+ [substitute_identifying_refs.call(absorbed_into)]
109
+ else
110
+ irf = object.dv_identifying_refs_from
111
+ trace :transform_surrogate_expansion, "Iterating for #{object.name} over #{irf.inspect}" do
112
+ irf.each_with_index do |ref, i|
113
+ next if ref.is_unary
114
+ next if ref.to_role.object_type.kind_of?(ActiveFacts::Metamodel::ValueType)
115
+ recurse_to = ref.to_role.object_type
116
+
117
+ trace :transform_surrogate_expansion, "#{i}: recursing to expand #{recurse_to.name} key in #{ref}" do
118
+ irf[i] = substitute_identifying_refs.call(recurse_to)
119
+ end
120
+ end
121
+ end
122
+ irf
123
+ end
124
+ end
125
+ irf = substitute_identifying_refs.call(self)
126
+
127
+ trace :transform_surrogate, "Does #{name} need a surrogate? it's identified by #{irf.inspect}" do
128
+
129
+ pk_fks = dv_identifying_refs_from.map do |ref|
130
+ ref.to && ref.to.is_table ? ref.to : nil
131
+ end
132
+
133
+ irf.flatten!
134
+
135
+ # Multi-part identifiers are only allowed if:
136
+ # * each part is a foreign key (i.e. it's a join table),
137
+ # * there are no other columns (that might require updating) and
138
+ # * the object is not the target of a foreign key:
139
+ if irf.size >= 2
140
+ if pk_fks.include?(nil)
141
+ trace :transform_surrogate, "#{self.name} needs a surrogate because its multi-part key contains a non-table"
142
+ return true
143
+ elsif references_to.size != 0
144
+ trace :transform_surrogate, "#{self.name} is a join table between #{pk_fks.map(&:name).inspect} but is also an FK target"
145
+ return true
146
+ elsif (references_from-dv_identifying_refs_from).size > 0
147
+ # There are other attributes to worry about
148
+ return true
149
+ else
150
+ trace :transform_surrogate, "#{self.name} is a join table between #{pk_fks.map(&:name).inspect}"
151
+ return false
152
+ end
153
+ return true
154
+ end
155
+
156
+ # Single-part key. It must be an Auto Counter, or we will add a surrogate
157
+
158
+ identifying_type = irf[0].to
159
+ if identifying_type.dv_needs_surrogate
160
+ trace :transform_surrogate, "#{self.name} needs a surrogate because #{irf[0].to.name} is not an AutoCounter, but #{identifying_type.supertypes_transitive.map(&:name).inspect}"
161
+ return true
162
+ end
163
+
164
+ false
165
+ end
166
+ end
167
+
168
+ def dv_inject_surrogate
169
+ trace :transform_surrogate, "Injecting a surrogate key into #{self.name}"
170
+
171
+ # Disable the preferred identifier:
172
+ pi = preferred_identifier
173
+ trace :transform_surrogate, "pi for #{name} was '#{pi.describe}'"
174
+ pi.is_preferred_identifier = false
175
+ @preferred_identifier = nil # Kill the cache
176
+
177
+ dv_add_surrogate
178
+
179
+ trace :transform_surrogate, "pi for #{name} is now '#{preferred_identifier.describe}'"
180
+ end
181
+
182
+ def dv_add_surrogate type_name = 'Auto Counter', suffix = 'ID'
183
+ # Find or assert the surrogate value type
184
+ auto_counter = vocabulary.valid_value_type_name(type_name) ||
185
+ constellation.ValueType(:vocabulary => vocabulary, :name => type_name, :concept => :new)
186
+
187
+ # Create a subtype to identify this entity type:
188
+ vt_name = self.name + ' '+suffix
189
+ my_id = @vocabulary.valid_value_type_name(vt_name) ||
190
+ constellation.ValueType(:vocabulary => vocabulary, :name => vt_name, :concept => :new, :supertype => auto_counter)
191
+
192
+ # Create a fact type
193
+ identifying_fact_type = constellation.FactType(:concept => :new)
194
+ my_role = constellation.Role(:concept => :new, :fact_type => identifying_fact_type, :ordinal => 0, :object_type => self)
195
+ @injected_surrogate_role = my_role
196
+ id_role = constellation.Role(:concept => :new, :fact_type => identifying_fact_type, :ordinal => 1, :object_type => my_id)
197
+
198
+ # Create a reading (which needs a RoleSequence)
199
+ reading = constellation.Reading(
200
+ :fact_type => identifying_fact_type,
201
+ :ordinal => 0,
202
+ :role_sequence => [:new],
203
+ :text => "{0} has {1}"
204
+ )
205
+ constellation.RoleRef(:role_sequence => reading.role_sequence, :ordinal => 0, :role => my_role)
206
+ constellation.RoleRef(:role_sequence => reading.role_sequence, :ordinal => 1, :role => id_role)
207
+
208
+ # Create two uniqueness constraints for the one-to-one. Each needs a RoleSequence (two RoleRefs)
209
+ one_id = constellation.PresenceConstraint(
210
+ :concept => :new,
211
+ :vocabulary => vocabulary,
212
+ :name => self.name+'HasOne'+suffix,
213
+ :role_sequence => [:new],
214
+ :is_mandatory => true,
215
+ :min_frequency => 1,
216
+ :max_frequency => 1,
217
+ :is_preferred_identifier => false
218
+ )
219
+ @constellation.RoleRef(:role_sequence => one_id.role_sequence, :ordinal => 0, :role => my_role)
220
+
221
+ one_me = constellation.PresenceConstraint(
222
+ :concept => :new,
223
+ :vocabulary => vocabulary,
224
+ :name => self.name+suffix+'IsOfOne'+self.name,
225
+ :role_sequence => [:new],
226
+ :is_mandatory => false,
227
+ :min_frequency => 0,
228
+ :max_frequency => 1,
229
+ :is_preferred_identifier => true
230
+ )
231
+ @constellation.RoleRef(:role_sequence => one_me.role_sequence, :ordinal => 0, :role => id_role)
232
+
233
+ return my_id
234
+ end
235
+
236
+ end
237
+
238
+ include ActiveFacts::TraitInjector # Must be last in this module, after all submodules have been defined
239
+ end
240
+ end
241
+ end
@@ -0,0 +1,266 @@
1
+ #
2
+ # Data Vault Transform
3
+ # Transform a loaded ActiveFacts vocabulary to suit Data Vault
4
+ #
5
+ # Copyright (c) 2015 Infinuendo. Read the LICENSE file.
6
+ #
7
+ require 'activefacts/vocabulary'
8
+ require 'activefacts/persistence'
9
+
10
+ require 'activefacts/generate/traits/datavault'
11
+
12
+ module ActiveFacts
13
+
14
+ module Generate #:nodoc:
15
+ module Transform #:nodoc:
16
+ class DataVault
17
+ def initialize(vocabulary, *options)
18
+ @vocabulary = vocabulary
19
+ @constellation = vocabulary.constellation
20
+ end
21
+
22
+ def classify_tables
23
+ initial_tables = @vocabulary.tables
24
+ non_reference_tables = initial_tables.reject do |table|
25
+ table.concept.all_concept_annotation.detect{|ca| ca.mapping_annotation == 'static'} or
26
+ !table.is_a?(ActiveFacts::Metamodel::EntityType)
27
+ end
28
+ @reference_tables = initial_tables-non_reference_tables
29
+
30
+ @link_tables, @hub_tables = non_reference_tables.partition do |table|
31
+ identifying_references = table.identifier_columns.map{|c| c.references.first}.uniq
32
+ # Which identifying_references are played by other tables?
33
+ ir_tables =
34
+ identifying_references.select do |r|
35
+ table_referred_to = r.to
36
+ # I have no examples of multi-level absorption, but it's possible, so loop
37
+ while av = table_referred_to.absorbed_via
38
+ table_referred_to = av.from
39
+ end
40
+ table_referred_to.is_table
41
+ end
42
+ ir_tables.size > 1
43
+ end
44
+ trace_table_classifications
45
+ end
46
+
47
+ def trace_table_classifications
48
+ # Trace the decisions about table types:
49
+ if trace :datavault
50
+ [@reference_tables, @hub_tables, @link_tables].zip(['Reference', 'Hub', 'Link']).each do |tables, kind|
51
+ trace :datavault, kind+' tables: ' do
52
+ tables.each do |table|
53
+ identifying_references = table.identifier_columns.map{|c| c.references.first}.uniq
54
+ trace :datavault, "#{table.name}(#{identifying_references.map{|r| (t = r.to) && t.name || 'self'}*', '})"
55
+ end
56
+ end
57
+ end
58
+ end
59
+ end
60
+
61
+ def detect_required_surrogates
62
+ trace :datavault, "Detecting required surrogates" do
63
+ @required_surrogates =
64
+ (@hub_tables+@link_tables).select do |table|
65
+ table.dv_needs_surrogate
66
+ end
67
+ end
68
+ end
69
+
70
+ def inject_required_surrogates
71
+ trace :datavault, "Injecting any required surrogates" do
72
+ trace :datavault, "Need to inject surrogates into #{@required_surrogates.map(&:name)*', '}"
73
+ @required_surrogates.each do |table|
74
+ table.dv_inject_surrogate
75
+ end
76
+ end
77
+ end
78
+
79
+ def classify_satellite_references table
80
+ identifying_references = table.identifier_columns.map{|c| c.references.first}.uniq
81
+ non_identifying_references = table.columns.map{|c| c.references[0]}.uniq - identifying_references
82
+
83
+ # Skip this table if no satellite data is needed
84
+ # REVISIT: Needed anyway for a link?
85
+ if non_identifying_references.size == 0
86
+ return nil
87
+ end
88
+
89
+ satellites = non_identifying_references.inject({}) do |hash, ref|
90
+ # Extract the declared satellite name, or use just "satellite"
91
+ satellite_subname =
92
+ ref.fact_type.internal_presence_constraints.map do |pc|
93
+ next if !pc.max_frequency || pc.max_frequency > 1 # Not a Uniqueness Constraint
94
+ next if pc.role_sequence.all_role_ref.size > 1 # Covers more than one role
95
+ next if pc.role_sequence.all_role_ref.single.role.object_type != table # Not a unique attribute
96
+ pc.concept.all_concept_annotation.map do |ca|
97
+ if ca.mapping_annotation =~ /^satellite */
98
+ ca.mapping_annotation.sub(/^satellite +/, '')
99
+ else
100
+ nil
101
+ end
102
+ end
103
+ end.flatten.compact.uniq[0] || "satellite"
104
+ satellite_name = "#{satellite_subname}"
105
+ (hash[satellite_name] ||= []) << ref
106
+ hash
107
+ end
108
+ trace :datavault, "#{table.name} satellites are #{satellites.inspect}"
109
+ satellites
110
+ end
111
+
112
+ def create_one_to_many(one, many, predicate_1 = 'has', predicate_2 = 'is of', one_adj = nil)
113
+ # Create a fact type
114
+ fact_type = @constellation.FactType(:concept => :new)
115
+ one_role = @constellation.Role(:concept => :new, :fact_type => fact_type, :ordinal => 0, :object_type => one)
116
+ many_role = @constellation.Role(:concept => :new, :fact_type => fact_type, :ordinal => 1, :object_type => many)
117
+
118
+ # Create two readings
119
+ reading2 = @constellation.Reading(:fact_type => fact_type, :ordinal => 0, :role_sequence => [:new], :text => "{0} #{predicate_2} {1}")
120
+ @constellation.RoleRef(:role_sequence => reading2.role_sequence, :ordinal => 0, :role => many_role)
121
+ @constellation.RoleRef(:role_sequence => reading2.role_sequence, :ordinal => 1, :role => one_role, :leading_adjective => one_adj)
122
+
123
+ reading1 = @constellation.Reading(:fact_type => fact_type, :ordinal => 1, :role_sequence => [:new], :text => "{0} #{predicate_1} {1}")
124
+ @constellation.RoleRef(:role_sequence => reading1.role_sequence, :ordinal => 0, :role => one_role, :leading_adjective => one_adj)
125
+ @constellation.RoleRef(:role_sequence => reading1.role_sequence, :ordinal => 1, :role => many_role)
126
+
127
+ one_id = @constellation.PresenceConstraint(
128
+ :concept => :new,
129
+ :vocabulary => @vocabulary,
130
+ :name => one.name+'HasOne'+many.name,
131
+ :role_sequence => [:new],
132
+ :is_mandatory => true,
133
+ :min_frequency => 1,
134
+ :max_frequency => 1,
135
+ :is_preferred_identifier => false
136
+ )
137
+ @constellation.RoleRef(:role_sequence => one_id.role_sequence, :ordinal => 0, :role => many_role)
138
+ one_role
139
+ end
140
+
141
+ def assert_value_type name, supertype = nil
142
+ @vocabulary.valid_value_type_name(name) ||
143
+ @constellation.ValueType(:vocabulary => @vocabulary, :name => name, :supertype => supertype, :concept => :new)
144
+ end
145
+
146
+ def assert_record_source
147
+ assert_value_type('Record Source', assert_value_type('String'))
148
+ end
149
+
150
+ def assert_date_time
151
+ assert_value_type('Date Time')
152
+ end
153
+
154
+ # Create a PresenceConstraint with two roles, marked as preferred_identifier
155
+ def create_two_role_identifier(r1, r2)
156
+ pc = @constellation.PresenceConstraint(
157
+ :concept => :new,
158
+ :vocabulary => @vocabulary,
159
+ :name => r1.object_type.name+' '+r1.object_type.name+'PK',
160
+ :role_sequence => [:new],
161
+ :is_mandatory => true,
162
+ :min_frequency => 1,
163
+ :max_frequency => 1,
164
+ :is_preferred_identifier => true
165
+ )
166
+ @constellation.RoleRef(:role_sequence => pc.role_sequence, :ordinal => 0, :role => r1)
167
+ @constellation.RoleRef(:role_sequence => pc.role_sequence, :ordinal => 1, :role => r2)
168
+ end
169
+
170
+ def create_satellite(table, satellite_name, references)
171
+ satellite_name = satellite_name.words.titlewords*' '
172
+ trace :datavault, "Creating #{satellite_name} for #{table.name} with #{references.size} references" do
173
+ # Create a new entity type with record-date fields in its identifier
174
+
175
+ satellite = @constellation.EntityType(:vocabulary => @vocabulary, :name => "#{table.name} #{satellite_name}", :concept => [:new, :implication_rule => "datavault"])
176
+ satellite.definitely_table
177
+
178
+ table_role = create_one_to_many(table, satellite)
179
+
180
+ date_time = assert_date_time
181
+ date_time_role = create_one_to_many(date_time, satellite, 'is of', 'was loaded at', 'load')
182
+ create_two_role_identifier(table_role, date_time_role)
183
+
184
+ record_source = assert_record_source
185
+ record_source.length = 64
186
+ record_source_role = create_one_to_many(record_source, satellite, 'is of', 'was loaded from')
187
+
188
+ # Move all roles across to it from the parent table.
189
+ references.each do |ref|
190
+ trace :datavault, "Moving #{ref} across to #{table.name}_#{satellite_name}" do
191
+ table_role = ref.fact_type.all_role.detect{|r| r.object_type == table}
192
+ # Reassign the role player to the satellite:
193
+ if table_role
194
+ table_role.object_type = satellite
195
+ else
196
+ #debugger # Bum, the crappy Reference object bites again.
197
+ $stderr.puts "REVISIT: Can't move the role for #{ref.inspect} without mangling the Reference"
198
+ end
199
+ end
200
+ end
201
+ end
202
+ end
203
+
204
+ def generate(out = $stdout)
205
+ @out = out
206
+
207
+ # Strategy:
208
+ # Determine list of ER tables
209
+ # Partition tables into reference tables (annotated), link tables (two+ FKs in PK), and hub tables
210
+ # For each hub and link table
211
+ # Apply a surrogate key if needed (all links, hubs lacking a simple surrogate)
212
+ # Detect references (fact types) leading to all attributes (non-identifying columns)
213
+ # Group attribute facts into satellites (use the satellite annotation if present)
214
+ # For each satellite
215
+ # Create a new entity type with a (hub-key, record-date key)
216
+ # Make new one->many fact type between hub and satellite
217
+ # Modify all attribute facts in this group to attach to the satellite
218
+ # Compute a gresh relational mapping
219
+ # Exclude reference tables and disable enforcement to them
220
+
221
+ classify_tables
222
+
223
+ detect_required_surrogates
224
+
225
+ trace :datavault, "Creating satellites" do
226
+ (@hub_tables+@link_tables).each do |table|
227
+ satellites = classify_satellite_references table
228
+ next unless satellites
229
+
230
+ trace :datavault, "Creating #{satellites.size} satellites for #{table.name}" do
231
+ satellites.each do |satellite_name, references|
232
+ create_satellite(table, satellite_name, references)
233
+ end
234
+ end
235
+ end
236
+ end
237
+
238
+ inject_required_surrogates
239
+
240
+ trace :datavault, "Adding standard fields to hubs and links" do
241
+ (@hub_tables+@link_tables).each do |table|
242
+ date_time = assert_date_time
243
+ date_time_role = create_one_to_many(date_time, table, 'is of', 'was loaded at', 'load')
244
+
245
+ record_source = assert_record_source
246
+ record_source_role = create_one_to_many(record_source, table, 'is of', 'was loaded from')
247
+ end
248
+ end
249
+
250
+ # Now, redo the E-R mapping using the revised schema:
251
+ @vocabulary.decide_tables
252
+
253
+ # Before departing, ensure we don't emit the reference tables!
254
+ @reference_tables.each do |table|
255
+ table.definitely_not_table
256
+ @vocabulary.tables.delete(table)
257
+ end
258
+
259
+ end # generate
260
+
261
+ end
262
+ end
263
+ end
264
+ end
265
+
266
+ ActiveFacts::Registry.generator('transform/datavault', ActiveFacts::Generate::Transform::DataVault)
@@ -256,6 +256,10 @@ module ActiveFacts
256
256
  @columns =
257
257
  all_columns({})
258
258
  end
259
+
260
+ def wipe_columns
261
+ @columns = nil
262
+ end
259
263
  end
260
264
 
261
265
  # The ValueType class is defined in the metamodel; full documentation is not generated.
@@ -150,6 +150,7 @@ module ActiveFacts
150
150
  (rr = c.role_sequence.all_role_ref.single) and
151
151
  rr.role == self
152
152
  end
153
+ # REVISIT: check mapping pragmas, e.g. by to_1.concept.all_concept_annotation.detect{|ca| ca.mapping_annotation == 'separate'}
153
154
 
154
155
  if fact_type.entity_type
155
156
  # This is a role in an objectified fact type
@@ -190,6 +191,7 @@ module ActiveFacts
190
191
  def wipe_existing_mapping
191
192
  all_object_type.each do |object_type|
192
193
  object_type.clear_references
194
+ object_type.wipe_columns
193
195
  object_type.is_table = nil # Undecided; force an attempt to decide
194
196
  object_type.tentative = true # Uncertain
195
197
  end
@@ -7,8 +7,8 @@
7
7
  module ActiveFacts
8
8
  module Version
9
9
  MAJOR = 1
10
- MINOR = 5
11
- PATCH = 3
10
+ MINOR = 6
11
+ PATCH = 0
12
12
 
13
13
  STRING = [MAJOR, MINOR, PATCH].compact.join('.')
14
14
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: activefacts
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.5.3
4
+ version: 1.6.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Clifford Heath
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2015-08-05 00:00:00.000000000 Z
11
+ date: 2015-09-02 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activefacts-api
@@ -310,9 +310,11 @@ files:
310
310
  - lib/activefacts/generate/stats.rb
311
311
  - lib/activefacts/generate/text.rb
312
312
  - lib/activefacts/generate/topics.rb
313
+ - lib/activefacts/generate/traits/datavault.rb
313
314
  - lib/activefacts/generate/traits/oo.rb
314
315
  - lib/activefacts/generate/traits/ordered.rb
315
316
  - lib/activefacts/generate/traits/ruby.rb
317
+ - lib/activefacts/generate/transform/datavault.rb
316
318
  - lib/activefacts/generate/transform/surrogate.rb
317
319
  - lib/activefacts/generate/version.rb
318
320
  - lib/activefacts/input/cql.rb