activefacts 0.8.16 → 0.8.18

data/TODO

@@ -1,308 +0,0 @@
-	ActiveFacts ToDo list
-
-	Metamodel
-ALWAYS		Integrate Metamodel changes through lib/activefacts/vocabulary/metamodel.rb
-SOON		Add Terms for Object Types
-			using Words? (which include adjectives/rolenames)
-		Notes (Extend context notes)
-			comments in CQL?
-			Derivation notes?
-			HelpText?
-		FBM Standards Metamodel
-			Migrate towards standard terminology
-			Make ORM model emit good CQL
-		Use UUIDs
-		Renamed from ("Person (was called User) ...")
-			Multiple revisions?
-		Extensions (new vocabularies?) for Metamodel
-			security
-			processes
-			discussion/conversation
-		Implement abstract supertypes?
-		Allow Object naming "Paris means City 'Paris'"
-		Use subtype extension in Metamodel (based on abstract SBVR supertypes as shown?)
-			Meaning
-				Concept
-					NounConcept
-						ObjectType
-						RolePlaying
-					FactType
-				Fact
-					Value
-				Query
-			Lexeme (language/locale sensitive text)
-				Term
-				Reading (template text, adjectives)
-				Value (literal)
-					Or just have per-locale syntax parser/generator to/from canonical form?
-		Role names
-			need to attach to RoleRefs instead of Roles
-		Refactor RoleRef and RoleSequence so RoleSequence isn't re-used out of context
-		Index/access-path/ordering pragmas
-		Temporal literals (expressions using current time)
-		Temporal: e.g. Date/Time {{/not} {after/before}} Date/Time
-SOON		Refactor Derived Fact Types (Queries) using new terminology
-
-	CQL
-		Object Types
-			ValueTypes
-				Change so Data Types are not inherited
-				Implement named parameters
-				Autogenerated
-					On assert (UUID)
-					On save (ID, Timestamp, etc)
-					Partial auto-counters (Role, *Ordinal*)
-				Implement base VTs
-SOON					"vt is written;" syntax
-			Entity Types
-SOON				"Et is identified;" forward-reference syntax
-				Allow other possessive pronouns "Ship is identified by her Name"
-				Allow shorthand for multi-part identification
-					"Person is identified by his given-Name and family-Name;"
-			Subtypes
-				Support Role subtypes as distinct (subtype migration allowed)
-				Subtyping fact type role names?
-		Object Names
-			"Paris means City 'Paris'"
-
-		Joins
-SOON			Anti-joins (no, none, negation, "it is not the case that", etc)
-SOON			Double-contraction "Person who came to Party and was not invited to"
-SOON			Outer joins ("maybe")
-SOON			Compile Value join nodes
-STARTED, DIFFICULT	Implement comparison operators as derived fact types (<, etc)
-			Automatically detect type conversion joins
-
-		Queries
-			Expression clauses
-				Redo grammar as normal clause (A < B -> A rel B)
-				User-defined expression operators/clauses?
-			Fact Type Negation (A rel no B, no A rel B)
-			"returning" clause
-			Invoke queries using defined role names:
-				Company has young Person (as Director) where ...
-				(invoke as "Company has young Director")
-			Temporal terms and phrases
-				"now" "4 year ago", "3 minutes from now"
-			Temporal ValueType to act as a Universal fact type over the domain
-				e.g. Time.hour -> all possible hours
-			Figure out what base constraints propagate to the result set
-			Allow "which" to project roles from a query "which Person gatecrashed Party?"
-			Value range restrictions (in {1,2,4}) instead of (restricted to {...})
-
-		Linguistic aspects
-			UTF-8 Support
-			Other languages
-				Define syntax for and handle Gender
-				French
-				Spanish
-				Use filename to select (Bar.fr.cql or Bar.cql.fr?)
-			Use metamodel predicates instead of pragmas
-				e.g. for "Object type is independent", say "Object Type 'Foo' is independent";
-				How to handle Fact Type names in the above syntax? Objectify? But need ring constraint roles.
-				Vocabulary name is implicit here...
-SOON			Allow more than one Business Context Note per item
-			Fact type binding
-				Use non-local role names in fact type matching?
-				Check that subscripts are being tested properly
-			Implement some/that binding
-				subscripts not necessary where "that" is unambiguous.
-				which/that: "which Person likes that Person?"
-			Double-hyphen for hyphenated adjectives
-
-		Constraints
-SOON			Implement "maybe" to make min-frequency non-mandatory
-			Value Literal syntax constraints
-				Regexp: "vt is written to match /xyz/"
-				Regexp for role values
-				subtype of Value Constraint
-				PEG Grammars for literals? (extend CQL grammar)
-			Ring constraints
-				how to handle ambiguous rings, e.g. in ternaries?
-				how to handle multiple rings in one FT?
-				Think about syntax to allow this
-				Populate Ring Types in metamodel (five unaries?)
-			Constraint names (esp Presence Constraints, they become index names)
-			Temporal Modeling ([at most] one at a time)
-				single table mapping
-				two tables mappings (current/history)
-			Add Exclusion constraint syntax using negative clause:
-				Person is ceo if and only if no Manager manages (that?) Person
-				Similar case exists for subset constraint?
-					A rel B only if no B rel C - think through
-			Delay creation of default identifying constraints for objectified fact types
-				Perhaps before relational/object mapping?
-				Or don't create them at all - handle missing PI for OFTs instead
-			Asserted instances
-				"including {'foo', 'bar', ...}" akin to "restricted to"...
-
-		Importing vocabularies
-			Distinguish "import" from "use"?
-			Bind to imported ValueTypes (and other object types?)
-			Base vocabularies for SQL
-				Multiple SQL base vocabs, using db pref to choose between.
-
-		CQL Verbaliser
-SOON			Detect new fact types that would be interpreted as an existing (only adjectives differ), and use single reading (before double-reading) e.g. other-Role of RingConstraint
-			Implement some/that binding
-				use some/that where "that" is unambiguous
-				Don't emit some/that until binding will use it properly
-			Instance data verbalisation
-				Emit objectification joins
-				Contractions (use Verbaliser!)
-				Emit example fact from ORM2009 presentation
-			Output clustering
-				Emit Value Types just before first use
-				Emit Constraints as soon as possible
-				Choose "focus" items with most 1st and 2nd generation dependencies, and do all precursors/followers before choosing another
-			Left-contraction of joins "Xyz is ... and is ...;"
-
-		Units
-SOON			Store sample data with specified unit
-				Handle storing numeric Instance values (noted to be in the lexical form; but strings aren't!)
-SOON			Save Units on ValueTypes
-			Generate conversion formula to fundamental types
-
-		CQL Shell
-			Save definitions, including instances
-			/sql; generate the SQL for this vocabulary
-			/valid; check all metamodel constraints over the vocabulary, and/or vocabulary constraints over the population
-			Queries
-			/population {open, check, create} database_name
-			/ruby
-			/cql
-
-	NORMA reader
-		NORMA Joins
-			Detect and store implicit joins in presence constraints (if needed???)
-			Convert all NORMA Join paths
-		NORMA input: Warn when preferred reading role order doesn't match the preferred identifier for objectified FTs
-		Import instance data for objectified fact types
-		Double-hyphen for hyphenated adjectives
-
-	Generators
-		Ruby generator
-			Emit a module method for each example population
-
-	Relational Composition
-		Subtype Mapping
-			extension (subtype in separate table)
-			partition (separate and include all supertype roles)
-			Make sure all test scenarios are covered
-		Transforms
-			Rails ID injection
-			Subtype discriminator injection where no mandatory role on subtype
-				Don't introduce additional discriminator for sub-sub-type
-				Detect non-exclusive subtypes and discriminate combinations
-		Column name generation
-			Provide alternative generation pattern(s) (snake-case)
-			Duplicate column names (detect, handle)
-		SQL Generation
-			Auto-generated column types
-				IDentity fields
-				GUIDs
-				partial auto-counters (Ordinal)
-				Timestamps
-				other auto-counters?
-			Foreign Key fields
-				Index both ends
-				Determine cascading based on mandatory-ness
-				Emit triggers where PK is in a subtype (and view was generated)
-			Existing database/vocabulary
-				Minimal impact vs refactor and migrate
-
-	Runtime API
-		Autogenerated Data Types
-			GUID, Ordinal
-		Entity
-			Save identifying roles as Role objects not symbols
-SOON		Propagate retract() properly (replace delete?)
-		Subtype migration
-			Replace instance by instance of different class, de-/re-assign all roles?
-			Re-use same role proxy/ies?
-		Role value array proxy class
-			Handle sorting by residual fields in counterpart's key
-			Detect & preserve adds/deletes
-			Correctly handle values across multiple constellations or in no constellation
-		Verbalisation
-SOON			Add readings to API
-			Redo all verbalisation to use readings
-				Use existing verbalise for to_s
-				Ensure that all objects and classes verbalise and to_s
-SOON			verbalise Role Value Array
-			verbalise takes optional "verbalisation context" object
-				Which objects have already been verbalised
-				What role names and/or subscripts have been assigned
-				Possible to implement as a single-pass process?
-		Queries
-			Query DSL/API in Ruby?
-			CQL Query conversion to SQL (via Ruby?)
-			Constellation boundaries
-				Record unpopulated roles and object types during query execution (nil != mu)
-				Hard vs soft boundary (Auto-fetch vs exception)
-		RDBMS Platform Support
-			DataObjects (SQL Server, MySQL, PostgreSQL, Oracle, DB2)
-			Enumerate tables instead of needing "table" keyword
-			Log changes to allow update
-
-	Database reverse engineering
-		Read Linda Bird's PhD thesis
-		Extract raw schema info
-			Tables
-			Columns & types
-			Null constraints
-			Unique & primary keys
-			Foreign keys
-			Check constraints
-		Naming Heuristics
-			Rails
-				Intuition (FK naming)
-				Load Rails models
-			Detect pluralisation
-			intuit common reference modes (e.g. thing_ID)
-		Examine data for likely FKs
-			count, min, max, median, distinct, count(NULL)
-			Check non-violation of possible FKs:
-				select count(*) where not exists ...
-		Decompose to elementary form
-		Schema transform by manual selection (GUI?)
-			Deabsorb (incl to use existing type)
-			Alias name
-			Project subclass using discriminator
-			... other transforms
-
-	Test stubs/mocks
-		Quality/usage metrics
-		Mock API
-		Mock data
-			Use sample population as test data
-			incorporate Faker, Machinist, etc
-
-	Packaging and documentation
-		Split into multiple gems?
-
-	Example models
-		Convert microformat RDF: http://microformats.org/about, http://www.data-vocabulary.org/
-
-	APRIMO
-		Overall design
-			Subscription, list/search models, friends
-			Diagram view
-			CQL view
-			Object type browser
-				Object Types
-				Fact Types
-				Constraints
-				Instance data
-			Query builder
-		AJAX
-			Design JSON API
-		Server
-			Stand-alone???
-		Metamodel Diagrams
-			Convert from metamodel to JSON
-		Database
-			Derived Fact Types
-			Constellation declarations
-			Process models
-		Units

data/lib/activefacts/cql/compiler/join.rb

@@ -1,162 +0,0 @@
-module ActiveFacts
-  module CQL
-    class Compiler < ActiveFacts::CQL::Parser
-      class Definition
-        # Make a JoinNode for every variable present in these clauses
-        def build_join_nodes(clauses_list)
-          debug :join, "Building join nodes" do
-            join = @constellation.Join(:new)
-            all_variables_in_clauses(clauses_list).
-              each do |variable|
-                debug :join, "Creating join node #{join.all_join_node.size} for #{variable.inspect}"
-                variable.join_node = @constellation.JoinNode(join, join.all_join_node.size, :object_type => variable.player)
-                if literal = variable.refs.detect{|r| r.literal}
-                  if literal.kind_of?(ActiveFacts::CQL::Compiler::VarRef)
-                    literal = literal.literal
-                  end
-                  unit = @constellation.Unit.detect{|k, v| [v.name, v.plural_name].include? literal.unit} if literal.unit
-                  variable.join_node.value = [literal.literal.to_s, literal.is_a?(String), unit]
-                end
-              end
-            join
-          end
-        end
-
-        def build_all_join_steps(clauses_list)
-          roles_by_variable = {}
-          debug :join, "Building join steps" do
-            clauses_list.each do |clause|
-              next if clause.is_naked_object_type
-              build_join_steps(clause, roles_by_variable)
-            end
-          end
-          roles_by_variable
-        end
-
-        def build_join_steps clause, roles_by_variable = {}, objectification_node = nil
-          join_roles = []
-          incidental_roles = []
-          debug :join, "Creating join Role Sequence for #{clause.inspect} with #{clause.var_refs.size} role refs" do
-            objectification_step = nil
-            clause.var_refs.each do |var_ref|
-              # These var_refs are the Compiler::VarRefs, which have associated Metamodel::RoleRefs,
-              # but we need to create JoinRoles for those roles.
-              # REVISIT: JoinRoles may need to save residual_adjectives
-              variable = var_ref.variable
-              role = (var_ref && var_ref.role) || (var_ref.role_ref && var_ref.role_ref.role)
-              join_role = nil
-
-              debugger unless clause.fact_type
-              if (clause.fact_type.entity_type)
-                # This clause is of an objectified fact type.
-                # We need a join step from this role to the phantom role, but not
-                # for a role that has only one var_ref (this one) in their variable.
-                # Create the JoinNode and JoinRole in any case though.
-                refs_count = variable.refs.size
-                objectification_ref_count = 0
-                if var_ref.nested_clauses
-                  var_ref.nested_clauses.each do |ojc|
-                    objectification_ref_count += ojc.var_refs.select{|var_ref| var_ref.variable.refs.size > 1}.size
-                  end
-                end
-                refs_count += objectification_ref_count
-
-                debug :join, "Creating Join Node #{var_ref.inspect} (counts #{refs_count}/#{objectification_ref_count}) and objectification Join Step for #{var_ref.inspect}" do
-
-                  raise "Internal error: Trying to add role of #{role.object_type.name} to join node for #{variable.join_node.object_type.name}" unless variable.join_node.object_type == role.object_type
-                  join_role = @constellation.JoinRole(variable.join_node, role)
-
-                  if (refs_count <= 1)   # Our work here is done if there are no other refs
-                    if objectification_step
-                      join_role.join_step = objectification_step
-                    else
-                      incidental_roles << join_role
-                    end
-                    next
-                  end
-
-                  join_roles << join_role
-                  unless objectification_node
-                    # This is an implicit objectification, just the FT clause, not ET(where ...clause...)
-                    # We need to create a JoinNode for this object, even though it has no VarRefs
-                    join = variable.join_node.join
-                    debug :join, "Creating JN#{join.all_join_node.size} for #{clause.fact_type.entity_type.name} in objectification"
-                    objectification_node = @constellation.JoinNode(join, join.all_join_node.size, :object_type => clause.fact_type.entity_type)
-                  end
-                  raise "Internal error: Trying to add role of #{role.implicit_fact_type.all_role.single.object_type.name} to join node for #{objectification_node.object_type.name}" unless objectification_node.object_type == role.implicit_fact_type.all_role.single.object_type
-
-                  irole = role.implicit_fact_type.all_role.single
-                  raise "Internal error: Trying to add role of #{irole.object_type.name} to join node for #{objectification_node.object_type.name}" unless objectification_node.object_type == irole.object_type
-                  objectification_role = @constellation.JoinRole(objectification_node, role.implicit_fact_type.all_role.single)
-                  objectification_step = @constellation.JoinStep(objectification_role, join_role, :fact_type => role.implicit_fact_type)
-                  debug :join, "New #{objectification_step.describe}"
-                  debug :join, "Associating #{incidental_roles.map(&:describe)*', '} incidental roles with #{objectification_step.describe}" if incidental_roles.size > 0
-                  incidental_roles.each { |jr| jr.join_step = objectification_step }
-                  incidental_roles = []
-                  join_roles = []
-                end
-              else
-                debug :join, "Creating VarRef for #{var_ref.inspect}" do
-                    # REVISIT: If there's an implicit subtyping join here, create it; then always raise the error here.
-                    # I don't want to do this for now because the verbaliser will always verbalise all join steps.
-                  if variable.join_node.object_type != role.object_type and
-                    0 == (variable.join_node.object_type.supertypes_transitive & role.object_type.supertypes_transitive).size
-                    raise "Internal error: Trying to add role of #{role.object_type.name} to join node #{variable.join_node.ordinal} for #{variable.join_node.object_type.name} in '#{clause.fact_type.default_reading}'"
-                  end
-                  raise "Internal error: Trying to add role of #{role.object_type.name} to join node #{variable.join_node.ordinal} for #{variable.join_node.object_type.name}" unless variable.join_node.object_type == role.object_type
-                  begin
-                    join_role = @constellation.JoinRole(variable.join_node, role)
-                  rescue ArgumentError => e
-                    join_role = @constellation.JoinRole(variable.join_node, role)
-                  end
-                  join_roles << join_role
-                end
-              end
-
-              if var_ref.nested_clauses
-                # We are looking at a role whose player is an objectification of a fact type,
-                # which will have ImplicitFactTypes for each role.
-                # Each of these ImplicitFactTypes has a single phantom role played by the objectifying entity type
-                # One of these phantom roles is likely to be the subject of an objectification join step.
-                var_ref.nested_clauses.each do |r|
-                  debug :join, "Building objectification join for #{var_ref.nested_clauses.inspect}" do
-                    build_join_steps r, roles_by_variable, variable.join_node
-                  end
-                end
-              end
-              roles_by_variable[variable] = [role, join_role]
-            end
-          end
-
-          if join_roles.size > 0
-            end_node = join_roles[-1].join_node
-            if !clause.fact_type.entity_type and role = clause.fact_type.all_role.single
-              # Don't give the ImplicitBoolean a join_node. We can live without one, for now.
-              # The Join Step will have a duplicate node, and the fact type will tell us what's happening
-              join_roles << join_roles[0]
-            end
-            # We aren't talking about objectification here, so there must be exactly two roles.
-            raise "REVISIT: Internal error constructing join for #{clause.inspect}" if join_roles.size != 2
-            js = @constellation.JoinStep(join_roles[0], join_roles[1], :fact_type => clause.fact_type)
-            debug :join, "New Join Step #{js.describe}"
-            debug :join, "Associating #{incidental_roles.map(&:describe)*', '} incidental roles with #{js.describe}" if incidental_roles.size > 0
-            incidental_roles.each { |jr| jr.join_step = js }
-          end
-          roles_by_variable
-        end
-
-        # Return the unique array of all variables in these clauses, including in objectification joins
-        def all_variables_in_clauses clauses
-          clauses.map do |clause|
-            clause.var_refs.map do |var_ref|
-              raise "Variable reference #{var_ref.inspect} is not bound to a variable" unless var_ref.variable
-              [var_ref.variable] + (var_ref.nested_clauses ? all_variables_in_clauses(var_ref.nested_clauses) : [])
-            end
-          end.
-            flatten.
-            uniq
-        end
-      end
-    end
-  end
-end