conceptql 0.0.3 → 0.0.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 09e9c3738772ff72625b848a26ac0e6d98c8275a
-  data.tar.gz: eb7753304d9a505e29c805019c222fe3d7a3119c
+  metadata.gz: 9e181385920abe60a6094cfddfac2e8991688084
+  data.tar.gz: 0394dd07282ac56252927290a597749a68798b60
 SHA512:
-  metadata.gz: e761811c0bcbcb2745cc5f3c9a0131d83a1b51ad26de632c3c02ca4d0f3b84ce6ba468f454a66a940027289d027747304501f52ecee648f41aa8f57aa91d98d5
-  data.tar.gz: 65b907da6c8bddb46d61a772dfc4c965fa1c8c87e3d1729b5ec0175c047b9860043b5760eba9583ce3c2f12d18cfc9dc901114d79a7cae410282e9c6e9dbefd2
+  metadata.gz: c65a2c6339d95b2ac59f75eebd81c6f38df7a219c6f51c17849d12799263a5a488f748f144b992f7c6069bc7fa0f49f266a20f106d54f58b36e2c160981b8bdc
+  data.tar.gz: 4cad8f6903fc03e7780f3428520ecd4a74befd30cc90e853febc6808ca6ab9b5a472471f005317f587832ddbd6a655607222ba3a405515492815c49eaa2a5be5

data/CHANGELOG.md

@@ -1,6 +1,24 @@
 # Changelog
 All notable changes to this project will be documented in this file.
 
+## 0.0.4 - 2014-08-19
+
+### Added
+- Support for 5 instead of 13 column internal representation of results.
+- `define` node, used to create "variables" in ConceptQL.
+- `recall` node, used to pull results from "variables" in ConceptQL.
+
+### Deprecated
+- Nothing.
+
+### Removed
+- Support for 13 column results.
+- Dependency on a set of views to run SQL queries.
+
+### Fixed
+- Bug where `place_of_service_code` wasn't limited to vocabulary_id 14
+
+
 ## 0.0.3 - 2014-08-12
 
 ### Added

data/README.md

@@ -45,6 +45,11 @@ As stated above, one of the goals of ConcegtQL is to make it easy to assemble fa
 If you're interested in reading more about ConceptQL, a rought draft of the specifications document is [available as a PDF](https://github.com/outcomesinsights/conceptql/blob/master/doc/ConceptQL%20Specification%20(alpha).pdf?raw=true) in this repository.
 
 
+## Try Before You Buy
+
+If you'd like to interact with ConceptQL a bit before deciding to dive in, head over to the [ConceptQL Sandbox](http://sandbox.cohortjigsaw.com) for an online demonstration of the language and its features.
+
+
 ## Requirements
 
 ConceptQL is in an early-alpha state.  For now it is limited to working with [OMOP CDM](http://omop.org/CDM)-structured data stored in the [PostgreSQL](http://www.postgresql.org/) database.  It has been tested under [Ubuntu Linux](http://www.ubuntu.com/) and Mac OS X 10.8+.  The interpreter is written in Ruby and theoretically should be platform independent, but your mileage may vary.
@@ -75,7 +80,7 @@ Or install it yourself as:
 
 ## Usage
 
-The easiest way to try out ConceptQL is to use the [conceptql-dev-box](http://github.com/outcomesinsights/conceptql-dev-box)
+The easiest way to try out ConceptQL locally is to use the [conceptql-dev-box](http://github.com/outcomesinsights/conceptql-dev-box)
 
 ConceptQL comes with a [Thor](http://whatisthor.com/)-based command-line program: `conceptql`
 

data/doc/spec.md

@@ -1049,6 +1049,47 @@ I don't have a good feel for:
     - Do we throw an exception if not?
     - Do we require calling programs to invoke a check on the concept before generating the query?
 
+
+##### Update 2014-08-13
+I've hacked some variable support into an experimental branch of ConceptQL.  So far, here's how it works:
+- Define node
+    - Give it two params, a name, and a stream
+    - Stream is used to populate a temporary table that is named after the name
+    - name is plain english sentence that is then turned into a hexdigest for a name
+        - Avoids collisions with other names
+        - Avoids truncation issues if name is WAY long
+- Recall node
+    - Takes a single argument: the name used in a define node
+    - Re-written to fetch results from the temp table that has the name provided
+
+Current issues:
+
+- Sequel's create table statement runs out-of-band with rest of ConceptQL statemnt
+    - Gets executed immediately
+- Type information in "define" needs to be made available to "from"
+    - Currently attempting to pass this information from define to from using an attribute tacked onto the shared db connection
+    - Recall may not have access to this information until #query is called
+    - This is bad and needs to be fixed/rethought
+
+
+Considerations for the future:
+- Probably want to rename these nodes to something better
+- It would still be nice to drop a concept into a concept that has "slots" waiting
+    - Perhaps slot is a different node from "define"
+- I had to retool Tree and Query and Graph to expect an array of concepts in a ConceptQL statement
+    - I'm not sure I like this
+    - A ConceptQL statement perhaps should be only a single statement at the end
+    - If an array of sub-concepts is fed into Query, maybe we only execute the last one after parsing the others
+        - This is consistent with how Sequel wants to live and would yield a single set of results
+        - I think I like this
+- Defines need to occur before they are used
+    - Most languages have a "forward definition" ability
+        - I have no use cases for when we might need those?
+        - Perhaps a definition that uses a definition that doesn't exist?
+        - Is that recursive?
+    - Is that something we want/need in ConceptQL?
+
+
 ### Value Nodes
 So far, we can’t recreate the Charlson comorbidity index using ConceptQL.  If we added a “value” node, we could.
 

data/lib/conceptql/behaviors/dottable.rb

@@ -44,6 +44,7 @@ module ConceptQL
           me = g.add_nodes(node_name)
           me[:label] = display_name
           me[:color] = type_color(types)
+          me[:shape] = shape if respond_to?(:shape)
           me
         end
       end

data/lib/conceptql/cli.rb

@@ -49,7 +49,7 @@ module ConceptQL
 
     desc 'show_and_tell_file statement_file', 'Reads the ConceptQL statement from the file and shows the contents as a ConceptQL graph, then executes the statement against the DB'
     option :full
-    def show_and_tell(file)
+    def show_and_tell_file(file)
       show_and_tell(criteria_from_file(file), options)
     end
 

data/lib/conceptql/graph.rb

@@ -34,14 +34,16 @@ module ConceptQL
     attr :yaml, :tree, :db
 
     def build_graph(g)
-      last_node = tree.root(self)
-      last_node.graph_it(g, db)
-      if dangler
-        blank_node = g.add_nodes('')
-        blank_node[:shape] = 'none'
-        blank_node[:height] = 0
-        blank_node[:fixedsize] = true
-        last_node.link_to(g, blank_node)
+      tree.root(self).each.with_index do |last_node, index|
+        last_node.graph_it(g, db)
+        if dangler
+          blank_node = g.add_nodes("_#{index}")
+          blank_node[:shape] = 'none'
+          blank_node[:height] = 0
+          blank_node[:label] = ''
+          blank_node[:fixedsize] = true
+          last_node.link_to(g, blank_node)
+        end
       end
     end
   end

data/lib/conceptql/graph_nodifier.rb

@@ -11,6 +11,7 @@ module ConceptQL
         condition: :condition_occurrence,
         primary_diagnosis: :condition_occurrence,
         icd9: :condition_occurrence,
+        icd10: :condition_occurrence,
         condition_type: :condition_occurrence,
 
         # Procedures
@@ -24,6 +25,7 @@ module ConceptQL
         # Visits
         visit_occurrence: :visit_occurrence,
         place_of_service: :visit_occurrence,
+        place_of_service_code: :visit_occurrence,
 
         # Person
         person: :person,
@@ -41,7 +43,10 @@ module ConceptQL
 
         # Drug
         drug_exposure: :drug_exposure,
+        rxnorm: :drug_exposure,
         drug_cost: :drug_cost,
+        drug_type_concept_id: :drug_exposure,
+        drug_type_concept: :drug_exposure,
 
         # Date Nodes
         date_range: :date,
@@ -120,11 +125,62 @@ module ConceptQL
       end
     end
 
+    class DefineNode < DotNode
+      def initialize(*args)
+        @gn = args.pop
+        super(*args)
+      end
+
+      def types
+        @gn.types[namify(arguments.first)] = super
+      end
+
+      def shape
+        :cds
+      end
+    end
+
+    class FromNode < DotNode
+      def initialize(*args)
+        @gn = args.pop
+        super(*args)
+      end
+
+      def types
+        @gn.types[namify(arguments.first)]
+      end
+
+      def shape
+        :cds
+      end
+    end
+
+    class VsacNode < DotNode
+      def initialize(name, values, types)
+        @types = types
+        super(name, values)
+      end
+
+      def types
+        [ @types ].flatten.compact.map(&:to_sym)
+      end
+    end
+
     BINARY_OPERATOR_TYPES = %w(before after meets met_by started_by starts contains during overlaps overlapped_by finished_by finishes coincides except person_filter less_than less_than_or_equal equal not_equal greater_than greater_than_or_equal filter).map { |temp| [temp, "not_#{temp}"] }.flatten.map(&:to_sym)
 
+    def types
+      @types ||= {}
+    end
     def create(type, values)
       if BINARY_OPERATOR_TYPES.include?(type)
         return BinaryOperatorNode.new(type, values)
+      elsif type == :define
+        return DefineNode.new(type, values, self)
+      elsif type == :from
+        return FromNode.new(type, values, self)
+      elsif type == :vsac
+        types = values.pop
+        return VsacNode.new(type, values, types)
       end
       DotNode.new(type, values)
     end

data/lib/conceptql/nodes/casting_node.rb

@@ -24,7 +24,11 @@ module ConceptQL
     # rows in its table as results.
     class CastingNode < Node
       def types
-        [my_type]
+        [type]
+      end
+
+      def type
+        my_type
       end
 
       def castables
@@ -39,36 +43,52 @@ module ConceptQL
       private
 
       def base_query(db, stream_query)
-        if (stream.types & castables).length < stream.types.length
-          # We have a situation where one of the incoming streams
+        uncastable_types = stream.types - castables
+        to_me_types = stream.types & these_point_at_me
+        from_me_types = stream.types & i_point_at
+
+        destination_table = make_table_name(my_type)
+        casting_query = db.from(destination_table)
+        wheres = []
+
+        unless uncastable_types.empty?
+          # We have a situation where one or more of the incoming streams
           # isn't castable so we'll just return all rows for
-          # all people
-          db.from(make_table_name(my_type))
-            .where(person_id: db.from(stream_query).select_group(:person_id))
-        else
-          # Every type in the stream is castable, so let's setup a query that
+          # all people in each uncastable stream
+          uncastable_person_ids = db.from(stream_query)
+            .where(criterion_type: uncastable_types.map(&:to_s))
+            .select_group(:person_id)
+          wheres << Sequel.expr(person_id: uncastable_person_ids)
+        end
+
+        destination_type_id = type_id(my_type)
+
+        unless to_me_types.empty?
+          # For each castable type in the stream, setup a query that
           # casts each type to a set of IDs, union those IDs and fetch
           # them from the source table
-          my_ids = stream.types.map do |type|
-            cast_type(db, type, stream_query)
-          end.inject do |union_query, query|
-            union_query.union(query)
-          end
+          castable_type_queries = to_me_types.map do |source_type|
+            source_ids = db.from(stream_query)
+              .where(criterion_type: source_type.to_s)
+              .select_group(:criterion_id)
+            source_table = make_table_name(source_type)
+            source_type_id = type_id(source_type)
 
-          db.from(make_table_name(my_type))
-            .where(type_id(my_type) => my_ids)
+            db.from(source_table)
+              .where(source_type_id => source_ids)
+              .select(destination_type_id)
+          end
+          wheres << Sequel.expr(destination_type_id => castable_type_queries)
         end
-      end
 
-      def cast_type(db, type, stream_query)
-        query = if i_point_at.include?(type)
-          db.from(make_table_name(my_type))
-            .where(type_id(type) => db.from(stream_query.select_group(type_id(type))))
-        else
-          db.from(make_table_name(type))
-            .where(type_id(type) => db.from(stream_query.select_group(type_id(type))))
+        unless from_me_types.empty?
+          from_me_types.each do |from_me_type|
+            fk_type_id = type_id(from_me_type)
+            wheres << Sequel.expr(fk_type_id => db.from(stream_query).where(criterion_type: from_me_type.to_s).select_group(:criterion_id))
+          end
         end
-        query.select(type_id(my_type))
+
+        casting_query.where(wheres.inject(&:|))
       end
     end
   end

data/lib/conceptql/nodes/complement.rb

@@ -6,11 +6,30 @@ module ConceptQL
       def query(db)
         child = children.first
         child.types.map do |type|
-          select_it(db.from(make_table_name(type)).exclude(type_id(type) => child.evaluate(db).select(type_id(type)).from_self.exclude(type_id(type) => nil)), [type])
+          positive_query = db.from(child.evaluate(db))
+            .select(:criterion_id)
+            .exclude(:criterion_id => nil)
+            .where(:criterion_type => type.to_s)
+          query = db.from(make_table_name(type))
+            .exclude(type_id(type) => positive_query)
+          db.from(select_it(query, type))
         end.inject do |union_query, q|
           union_query.union(q, all: true)
         end
       end
+
+=begin
+This is an alternate, but equally accurate way to do complement.
+We'll need to benchmark which is faster.
+      def query2(db)
+        child = children.first
+        froms = child.types.map do |type|
+          select_it(db.from(make_table_name(type)), type)
+        end
+        big_from = froms.inject { |union_query, q| union_query.union(q, all:true) }
+        db.from(big_from).except(child.evaluate(db))
+      end
+=end
     end
   end
 end

data/lib/conceptql/nodes/condition_type.rb

@@ -10,12 +10,12 @@ module ConceptQL
     #
     # Multiple types can be specified at once
     class ConditionType < Node
-      def types
-        [:condition_occurrence]
+      def type
+        :condition_occurrence
       end
 
       def query(db)
-        db.from(:condition_occurrence_with_dates)
+        db.from(:condition_occurrence)
           .where(condition_type_concept_id: condition_occurrence_type_concept_ids)
       end
 

data/lib/conceptql/nodes/date_range.rb

@@ -10,6 +10,8 @@ module ConceptQL
     class DateRange < Node
       def query(db)
         db.from(:person)
+          .select_append(Sequel.cast('person', :text).as(:criterion_type))
+          .select_append(Sequel.expr(:person_id).as(:criterion_id))
           .select_append(Sequel.expr(start_date(db)).cast(:date).as(:start_date),Sequel.expr(end_date(db)).cast(:date).as(:end_date)).from_self
       end
 

data/lib/conceptql/nodes/define.rb

@@ -0,0 +1,75 @@
+require_relative 'node'
+
+module ConceptQL
+  module Nodes
+    # Mimics creating a variable name that stores an itermediate result as
+    # part of a larger concept
+    #
+    # The idea is that a concept might be very complex and it helps to break
+    # that complex concept into a set of sub-concepts to better understand it.
+    #
+    # Also, sometimes a particular piece of a concept is used in many places,
+    # so it makes sense to write that piece out once, store it as a "variable"
+    # and then insert that variable into the concept as needed.
+    # run the query once and subsequent calls
+    class Define < Node
+      # Create a temporary table and store the stream of  results in that table.
+      # This "caches" the results so we only have to execute stream's query
+      # once.
+      #
+      # The logic here is that if something is assigned to a variable, chances
+      # are that it will be used more than once, so why run the query more than
+      # once?
+      #
+      # ConceptQL's SQL generator normally translates the entire statement into
+      # one, large query that can be executed later.
+      #
+      # Unfortunately, Sequel's "create_table" function actually executes the
+      # 'CREATE TABLE' SQL right away, meaning that the "define" node will
+      # execute immediately _during_ the processing of the ConceptQL statement.
+      # We'll see what kinds of problems this causes
+      #
+      # Lastly, this node does NOT pass its results to the next node.  The
+      # reason for this exception is to allow us to return the SQL that
+      # generates the temp table.  This is done so that the ConceptQL sandbox
+      # can return the entire set of SQL statements needed to run a query.
+      #
+      # Perhaps in the future we can find a way around this.
+      #
+      # Also, things will blow up if you try to use a variable that hasn't been
+      # defined yet.
+      def query(db)
+        table_name = namify(arguments.first)
+        stash_types(db, table_name, types)
+        db.create_table!(table_name, temp: true, as: stream.evaluate(db))
+        db[db.send(:create_table_as_sql, table_name, stream.evaluate(db).sql, temp: true)]
+      end
+
+      def types
+        stream.types
+      end
+
+      private
+
+      # TODO: Fix this.  This is shameful.
+      # In order to remember which types a table is storing, we need to preserve
+      # the type information outside of the "define" statement because the "recall"
+      # statement most likely doesn't have a reference to this node (that's the
+      # whole point).
+      #
+      # There is only one object shared between all the nodes: the database
+      # connection.  We'll do something terrible and piggyback the type
+      # information about this variable on the database connection so that
+      # the "recall" node can pull that information back out.
+      #
+      # Ugh.
+      def stash_types(db, name, types)
+        def db.types
+          @types ||= {}
+        end
+        db.types[name] = types
+      end
+    end
+  end
+end
+

data/lib/conceptql/nodes/drug_type_concept.rb

@@ -0,0 +1,18 @@
+require_relative 'node'
+
+module ConceptQL
+  module Nodes
+    class DrugTypeConcept < Node
+      def type
+        :drug_exposure
+      end
+
+      def query(db)
+        db.from(:drug_exposure)
+          .where(drug_type_concept_id: arguments)
+      end
+    end
+  end
+end
+
+

data/lib/conceptql/nodes/gender.rb

@@ -3,8 +3,8 @@ require_relative 'node'
 module ConceptQL
   module Nodes
     class Gender < Node
-      def types
-        [:person]
+      def type
+        :person
       end
 
       def query(db)
@@ -19,7 +19,7 @@ module ConceptQL
           end
         end
 
-        db.from(:person_with_dates)
+        db.from(:person)
           .where(gender_concept_id: gender_concept_ids)
       end
     end

data/lib/conceptql/nodes/intersect.rb

@@ -10,8 +10,9 @@ module ConceptQL
       def query(db)
         exprs = {}
         values.each do |expression|
+          evaled = expression.evaluate(db)
           expression.types.each do |type|
-            (exprs[type] ||= []) << expression.evaluate(db)
+            (exprs[type] ||= []) << evaled
           end
         end
         typed_queries = exprs.map do |type, queries|

data/lib/conceptql/nodes/node.rb

@@ -2,18 +2,6 @@ require 'active_support/core_ext/hash'
 module ConceptQL
   module Nodes
     class Node
-      KNOWN_TYPES = %i(
-        condition_occurrence
-        death
-        drug_cost
-        drug_exposure
-        observation
-        payer_plan_period
-        procedure_cost
-        procedure_occurrence
-        visit_occurrence
-      )
-
       attr :values, :options
       def initialize(*args)
         args.flatten!
@@ -28,12 +16,13 @@ module ConceptQL
         select_it(query(db))
       end
 
-      def select_it(query, select_types=types)
-        query.select(*columns(select_types))
+      def select_it(query, specific_type = nil)
+        specific_type = type if specific_type.nil? && respond_to?(:type)
+        query.select(*columns(specific_type))
       end
 
       def types
-        @types ||= children.map(&:types).flatten.uniq
+        @types ||= determine_types
       end
 
       def children
@@ -48,23 +37,98 @@ module ConceptQL
         @arguments ||= values.reject { |v| v.is_a?(Node) }
       end
 
-      def columns(select_types = types)
-        [:person_id___person_id] + KNOWN_TYPES.map do |known_type|
-          select_types.include?(known_type) ? "#{known_type}_id___#{known_type}_id".to_sym : Sequel.expr(nil).cast(:bigint).as("#{known_type}_id".to_sym)
-        end + date_columns
+      def columns(local_type = nil)
+        criterion_type = Sequel.expr(:criterion_type)
+        if local_type
+          criterion_type = Sequel.expr(local_type.to_s).cast(:text)
+        end
+        [:person_id___person_id,
+         Sequel.expr(type_id(local_type)).as(:criterion_id),
+         criterion_type.as(:criterion_type)] + date_columns(local_type)
       end
 
-      def date_columns
-        [:start_date, :end_date]
+      private
+      def criterion_id
+        :criterion_id
       end
 
-      private
-      def type_id(type)
+      def type_id(type = nil)
+        return :criterion_id if type.nil?
+        type = :person if type == :death
         (type.to_s + '_id').to_sym
       end
 
       def make_table_name(table)
-        "#{table}_with_dates___tab".to_sym
+        "#{table}___tab".to_sym
+      end
+
+      def date_columns(type = nil)
+        return [:start_date, :end_date] unless type
+        sd = start_date_column(type)
+        sd = Sequel.expr(sd).cast(:date).as(:start_date) unless sd == :start_date
+        ed = end_date_column(type)
+        ed = Sequel.expr(ed).cast(:date).as(:end_date) unless ed == :end_date
+        [sd, ed]
+      end
+
+      def start_date_column(type)
+        {
+          condition_occurrence: :condition_start_date,
+          death: :death_date,
+          drug_exposure: :drug_exposure_start_date,
+          drug_cost: nil,
+          payer_plan_period: :payer_plan_period_start_date,
+          person: person_date_of_birth,
+          procedure_occurrence: :procedure_date,
+          procedure_cost: nil,
+          observation: :observation_date,
+          visit_occurrence: :visit_start_date
+        }[type]
+      end
+
+      def end_date_column(type)
+        {
+          condition_occurrence: :condition_end_date,
+          death: :death_date,
+          drug_exposure: :drug_exposure_end_date,
+          drug_cost: nil,
+          payer_plan_period: :payer_plan_period_end_date,
+          person: person_date_of_birth,
+          procedure_occurrence: :procedure_date,
+          procedure_cost: nil,
+          observation: :observation_date,
+          visit_occurrence: :visit_end_date
+        }[type]
+      end
+
+      def person_date_of_birth
+        assemble_date(:year_of_birth, :month_of_birth, :day_of_birth)
+      end
+
+      def assemble_date(*symbols)
+        strings = symbols.map do |symbol|
+          Sequel.function(:coalesce, Sequel.expr(symbol).cast(:text), Sequel.expr('01').cast(:text)).cast(:text)
+        end
+        strings = strings.zip(['-'] * (symbols.length - 1)).flatten.compact
+        Sequel.function(:date, Sequel.join(strings))
+      end
+
+      def determine_types
+        if children.empty?
+          if respond_to?(:type)
+            [type]
+          else
+            raise "Node doesn't seem to specify any type"
+          end
+        else
+          children.map(&:types).flatten.uniq
+        end
+      end
+
+      def namify(name)
+        require 'digest'
+        digest = Digest::SHA256.hexdigest name
+        ('_' + digest).to_sym
       end
     end
   end

data/lib/conceptql/nodes/occurrence.rb

@@ -23,7 +23,7 @@ module ConceptQL
     # occurrence, this node returns nothing for that person
     class Occurrence < Node
       def query(db)
-        db.from(stream.evaluate(db)
+        db.from(db.from(stream.evaluate(db))
           .select_append { |o| o.row_number(:over, partition: :person_id, order: ordered_columns){}.as(:rn) })
           .where(rn: occurrence.abs)
       end
@@ -39,7 +39,7 @@ module ConceptQL
 
       def ordered_columns
         ordered_columns = [Sequel.send(asc_or_desc, :start_date)]
-        ordered_columns += types.map { |t| Sequel.asc(type_id(t)) }
+        ordered_columns += [:criterion_type, :criterion_id]
       end
     end
   end

data/lib/conceptql/nodes/place_of_service_code.rb

@@ -9,14 +9,15 @@ module ConceptQL
     # concept_name column of the concept table.  If you misspell the place_of_service_code name
     # you won't get any matches
     class PlaceOfServiceCode < Node
-      def types
-        [:visit_occurrence]
+      def type
+        :visit_occurrence
       end
 
       def query(db)
-        db.from(:visit_occurrence_with_dates___v)
+        db.from(:visit_occurrence___v)
           .join(:vocabulary__concept___vc, { vc__concept_id: :v__place_of_service_concept_id })
           .where(vc__concept_code: arguments.map(&:to_s))
+          .where(vc__vocabulary_id: 14)
       end
     end
   end

data/lib/conceptql/nodes/race.rb

@@ -9,12 +9,12 @@ module ConceptQL
     # concept_name column of the concept table.  If you misspell the race name
     # you won't get any matches
     class Race < Node
-      def types
-        [:person]
+      def type
+        :person
       end
 
       def query(db)
-        db.from(:person_with_dates___p)
+        db.from(:person___p)
           .join(:vocabulary__concept___vc, { vc__concept_id: :p__race_concept_id })
           .where(Sequel.function(:lower, :vc__concept_name) => arguments.map(&:downcase))
       end

data/lib/conceptql/nodes/recall.rb

@@ -0,0 +1,34 @@
+require_relative 'pass_thru'
+
+module ConceptQL
+  module Nodes
+    # Mimics using a variable that has been set via "define" node
+    #
+    # The idea is that a concept might be very complex and it helps to break
+    # that complex concept into a set of sub-concepts to better understand it.
+    #
+    # This node will look for a sub-concept that has been created through the
+    # "define" node and will fetch the results cached in the corresponding table
+    class Recall < Node
+      # Behind the scenes we simply fetch all rows from the temp table that
+      # corresponds to the name fed to "recall"
+      #
+      # We also set the @types variable by pulling the type information out
+      # of the hash piggybacking on the database connection.
+      #
+      # TODO: This might be an issue since we might need the type information
+      # before we call #query.  Probably time to reevaluate how we're caching
+      # the type information.
+      def query(db)
+        table_name = namify(arguments.first)
+        @types = db.types[table_name]
+        db.from(table_name)
+      end
+
+      def types
+        @types
+      end
+    end
+  end
+end
+

data/lib/conceptql/nodes/source_vocabulary_node.rb

@@ -32,11 +32,12 @@ module ConceptQL
           .where(Sequel.expr(scm__source_code: values, scm__source_vocabulary_id: vocabulary_id).&(Sequel.expr(scm__source_code: table_source_column)))
       end
 
-      def types
-        [table]
+      def type
+        table
       end
 
       private
+
       def table_name
         @table_name ||= make_table_name(table)
       end

data/lib/conceptql/nodes/standard_vocabulary_node.rb

@@ -25,11 +25,11 @@ module ConceptQL
           .where(c__concept_code: values, c__vocabulary_id: vocabulary_id)
       end
 
-      def types
-        [table]
+      def type
+        table
       end
-
       private
+
       def table_name
         @table_name ||= make_table_name(table)
       end

data/lib/conceptql/nodes/temporal_node.rb

@@ -9,8 +9,8 @@ module ConceptQL
     # Proc.new { l.end_date < r.start_date }
     class TemporalNode < BinaryOperatorNode
       def query(db)
-        db.from(db.from(Sequel.expr(left.evaluate(db)).as(:l))
-                  .join(Sequel.expr(right.evaluate(db)).as(:r), [:person_id])
+        db.from(db.from(left_stream(db))
+                  .join(right_stream(db), [:person_id])
                   .where(where_clause)
                   .select_all(:l))
       end
@@ -18,6 +18,14 @@ module ConceptQL
       def inclusive?
         options[:inclusive]
       end
+
+      def left_stream(db)
+        Sequel.expr(left.evaluate(db)).as(:l)
+      end
+
+      def right_stream(db)
+        Sequel.expr(right.evaluate(db)).as(:r)
+      end
     end
   end
 end

data/lib/conceptql/nodes/time_window.rb

@@ -25,7 +25,7 @@ module ConceptQL
       end
 
       private
-      def date_columns
+      def date_columns(type = nil)
         [adjusted_start_date, adjusted_end_date]
       end
 

data/lib/conceptql/query.rb

@@ -1,6 +1,5 @@
 require 'psych'
 require_relative 'tree'
-require_relative 'view_maker'
 
 module ConceptQL
   class Query
@@ -11,29 +10,34 @@ module ConceptQL
       @tree = tree
     end
 
-    def query
+    def queries
       build_query(db)
     end
 
+    def query
+      queries.last
+    end
+
+    def sql
+      queries.map(&:sql).join(";\n") + ';'
+    end
+
+    # To avoid a performance penalty, only execute the last
+    # SQL statement in an array of ConceptQL statements so that define's
+    # "create_table" SQL isn't executed twice
     def execute
-      ensure_views
-      build_query(db).all
+      query.all
     end
 
     def types
-      tree.root(self).types
+      tree.root(self).last.types
     end
 
     private
     attr :yaml, :tree, :db
 
     def build_query(db)
-      tree.root(self).evaluate(db)
-    end
-
-    def ensure_views
-      return if db.views.include?(:person_with_dates)
-      ViewMaker.make_views(db)
+      tree.root(self).map { |n| n.evaluate(db) }
     end
   end
 end

data/lib/conceptql/tree.rb

@@ -9,8 +9,8 @@ module ConceptQL
       @behavior = opts.fetch(:behavior, nil)
     end
 
-    def root(query)
-      @root ||= traverse(query.statement.deep_symbolize_keys)
+    def root(*queries)
+      @root ||= traverse(queries.flatten.map(&:statement).flatten.map(&:deep_symbolize_keys))
     end
 
     private

data/lib/conceptql/version.rb

@@ -1,3 +1,3 @@
 module ConceptQL
-  VERSION = "0.0.3"
+  VERSION = "0.0.4"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: conceptql
 version: !ruby/object:Gem::Version
-  version: 0.0.3
+  version: 0.0.4
 platform: ruby
 authors:
 - Ryan Duryea
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-08-12 00:00:00.000000000 Z
+date: 2014-08-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -175,6 +175,8 @@ files:
 - lib/conceptql/nodes/cpt.rb
 - lib/conceptql/nodes/date_range.rb
 - lib/conceptql/nodes/death.rb
+- lib/conceptql/nodes/define.rb
+- lib/conceptql/nodes/drug_type_concept.rb
 - lib/conceptql/nodes/during.rb
 - lib/conceptql/nodes/except.rb
 - lib/conceptql/nodes/first.rb
@@ -195,6 +197,7 @@ files:
 - lib/conceptql/nodes/place_of_service_code.rb
 - lib/conceptql/nodes/procedure_occurrence.rb
 - lib/conceptql/nodes/race.rb
+- lib/conceptql/nodes/recall.rb
 - lib/conceptql/nodes/rxnorm.rb
 - lib/conceptql/nodes/snomed.rb
 - lib/conceptql/nodes/source_vocabulary_node.rb
@@ -209,7 +212,6 @@ files:
 - lib/conceptql/query.rb
 - lib/conceptql/tree.rb
 - lib/conceptql/version.rb
-- lib/conceptql/view_maker.rb
 - spec/conceptql/behaviors/dottable_spec.rb
 - spec/conceptql/date_adjuster_spec.rb
 - spec/conceptql/nodes/after_spec.rb

data/lib/conceptql/view_maker.rb

@@ -1,56 +0,0 @@
-require 'sequel'
-
-module ConceptQL
-  module ViewMaker
-    def self.make_views(db, schema = nil)
-      views.each do |view, columns|
-        make_view(db, schema, view, columns)
-      end
-    end
-
-    def self.make_view(db, schema, view, columns)
-      view_name = (view.to_s + '_with_dates')
-      view_name = schema + '__' + view_name if schema
-      view_name = view_name.to_sym
-
-      table_name = view.to_s
-      table_name = schema + '__' + table_name if schema
-      table_name = table_name.to_sym
-
-      additional_columns = [Sequel.expr(columns.shift).cast(:date).as(:start_date), Sequel.expr(columns.shift).cast(:date).as(:end_date)]
-      unless columns.empty?
-        additional_columns += columns.last.map do |column_name, column_value|
-          Sequel.expr(column_value).as(column_name)
-        end
-      end
-      query = db.from(table_name).select_all.select_append(*additional_columns)
-      puts query.sql
-      db.drop_view(view_name, if_exists: true)
-      db.create_view(view_name, query)
-    end
-
-    def self.views
-      person_date_of_birth = assemble_date(:year_of_birth, :month_of_birth, :day_of_birth)
-      {
-        condition_occurrence: [:condition_start_date, :condition_end_date],
-        death: [:death_date, :death_date, { death_id: :person_id }],
-        drug_exposure: [:drug_exposure_start_date, :drug_exposure_end_date],
-        drug_cost: [nil, nil],
-        payer_plan_period: [:payer_plan_period_start_date, :payer_plan_period_end_date],
-        person: [person_date_of_birth, person_date_of_birth],
-        procedure_occurrence: [:procedure_date, :procedure_date],
-        procedure_cost: [nil, nil],
-        observation: [:observation_date, :observation_date],
-        visit_occurrence: [:visit_start_date, :visit_end_date]
-      }
-    end
-
-    def self.assemble_date(*symbols)
-      strings = symbols.map do |symbol|
-        Sequel.function(:coalesce, symbol, '01').cast(:text)
-      end
-      strings = strings.zip(['-'] * (symbols.length - 1)).flatten.compact
-      Sequel.function(:date, Sequel.join(strings))
-    end
-  end
-end