conceptql 0.0.3 → 0.0.4
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +18 -0
- data/README.md +6 -1
- data/doc/spec.md +41 -0
- data/lib/conceptql/behaviors/dottable.rb +1 -0
- data/lib/conceptql/cli.rb +1 -1
- data/lib/conceptql/graph.rb +10 -8
- data/lib/conceptql/graph_nodifier.rb +56 -0
- data/lib/conceptql/nodes/casting_node.rb +44 -24
- data/lib/conceptql/nodes/complement.rb +20 -1
- data/lib/conceptql/nodes/condition_type.rb +3 -3
- data/lib/conceptql/nodes/date_range.rb +2 -0
- data/lib/conceptql/nodes/define.rb +75 -0
- data/lib/conceptql/nodes/drug_type_concept.rb +18 -0
- data/lib/conceptql/nodes/gender.rb +3 -3
- data/lib/conceptql/nodes/intersect.rb +2 -1
- data/lib/conceptql/nodes/node.rb +88 -24
- data/lib/conceptql/nodes/occurrence.rb +2 -2
- data/lib/conceptql/nodes/place_of_service_code.rb +4 -3
- data/lib/conceptql/nodes/race.rb +3 -3
- data/lib/conceptql/nodes/recall.rb +34 -0
- data/lib/conceptql/nodes/source_vocabulary_node.rb +3 -2
- data/lib/conceptql/nodes/standard_vocabulary_node.rb +3 -3
- data/lib/conceptql/nodes/temporal_node.rb +10 -2
- data/lib/conceptql/nodes/time_window.rb +1 -1
- data/lib/conceptql/query.rb +15 -11
- data/lib/conceptql/tree.rb +2 -2
- data/lib/conceptql/version.rb +1 -1
- metadata +5 -3
- data/lib/conceptql/view_maker.rb +0 -56
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 9e181385920abe60a6094cfddfac2e8991688084
|
4
|
+
data.tar.gz: 0394dd07282ac56252927290a597749a68798b60
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: c65a2c6339d95b2ac59f75eebd81c6f38df7a219c6f51c17849d12799263a5a488f748f144b992f7c6069bc7fa0f49f266a20f106d54f58b36e2c160981b8bdc
|
7
|
+
data.tar.gz: 4cad8f6903fc03e7780f3428520ecd4a74befd30cc90e853febc6808ca6ab9b5a472471f005317f587832ddbd6a655607222ba3a405515492815c49eaa2a5be5
|
data/CHANGELOG.md
CHANGED
@@ -1,6 +1,24 @@
|
|
1
1
|
# Changelog
|
2
2
|
All notable changes to this project will be documented in this file.
|
3
3
|
|
4
|
+
## 0.0.4 - 2014-08-19
|
5
|
+
|
6
|
+
### Added
|
7
|
+
- Support for 5 instead of 13 column internal representation of results.
|
8
|
+
- `define` node, used to create "variables" in ConceptQL.
|
9
|
+
- `recall` node, used to pull results from "variables" in ConceptQL.
|
10
|
+
|
11
|
+
### Deprecated
|
12
|
+
- Nothing.
|
13
|
+
|
14
|
+
### Removed
|
15
|
+
- Support for 13 column results.
|
16
|
+
- Dependency on a set of views to run SQL queries.
|
17
|
+
|
18
|
+
### Fixed
|
19
|
+
- Bug where `place_of_service_code` wasn't limited to vocabulary_id 14
|
20
|
+
|
21
|
+
|
4
22
|
## 0.0.3 - 2014-08-12
|
5
23
|
|
6
24
|
### Added
|
data/README.md
CHANGED
@@ -45,6 +45,11 @@ As stated above, one of the goals of ConcegtQL is to make it easy to assemble fa
|
|
45
45
|
If you're interested in reading more about ConceptQL, a rought draft of the specifications document is [available as a PDF](https://github.com/outcomesinsights/conceptql/blob/master/doc/ConceptQL%20Specification%20(alpha).pdf?raw=true) in this repository.
|
46
46
|
|
47
47
|
|
48
|
+
## Try Before You Buy
|
49
|
+
|
50
|
+
If you'd like to interact with ConceptQL a bit before deciding to dive in, head over to the [ConceptQL Sandbox](http://sandbox.cohortjigsaw.com) for an online demonstration of the language and its features.
|
51
|
+
|
52
|
+
|
48
53
|
## Requirements
|
49
54
|
|
50
55
|
ConceptQL is in an early-alpha state. For now it is limited to working with [OMOP CDM](http://omop.org/CDM)-structured data stored in the [PostgreSQL](http://www.postgresql.org/) database. It has been tested under [Ubuntu Linux](http://www.ubuntu.com/) and Mac OS X 10.8+. The interpreter is written in Ruby and theoretically should be platform independent, but your mileage may vary.
|
@@ -75,7 +80,7 @@ Or install it yourself as:
|
|
75
80
|
|
76
81
|
## Usage
|
77
82
|
|
78
|
-
The easiest way to try out ConceptQL is to use the [conceptql-dev-box](http://github.com/outcomesinsights/conceptql-dev-box)
|
83
|
+
The easiest way to try out ConceptQL locally is to use the [conceptql-dev-box](http://github.com/outcomesinsights/conceptql-dev-box)
|
79
84
|
|
80
85
|
ConceptQL comes with a [Thor](http://whatisthor.com/)-based command-line program: `conceptql`
|
81
86
|
|
data/doc/spec.md
CHANGED
@@ -1049,6 +1049,47 @@ I don't have a good feel for:
|
|
1049
1049
|
- Do we throw an exception if not?
|
1050
1050
|
- Do we require calling programs to invoke a check on the concept before generating the query?
|
1051
1051
|
|
1052
|
+
|
1053
|
+
##### Update 2014-08-13
|
1054
|
+
I've hacked some variable support into an experimental branch of ConceptQL. So far, here's how it works:
|
1055
|
+
- Define node
|
1056
|
+
- Give it two params, a name, and a stream
|
1057
|
+
- Stream is used to populate a temporary table that is named after the name
|
1058
|
+
- name is plain english sentence that is then turned into a hexdigest for a name
|
1059
|
+
- Avoids collisions with other names
|
1060
|
+
- Avoids truncation issues if name is WAY long
|
1061
|
+
- Recall node
|
1062
|
+
- Takes a single argument: the name used in a define node
|
1063
|
+
- Re-written to fetch results from the temp table that has the name provided
|
1064
|
+
|
1065
|
+
Current issues:
|
1066
|
+
|
1067
|
+
- Sequel's create table statement runs out-of-band with rest of ConceptQL statemnt
|
1068
|
+
- Gets executed immediately
|
1069
|
+
- Type information in "define" needs to be made available to "from"
|
1070
|
+
- Currently attempting to pass this information from define to from using an attribute tacked onto the shared db connection
|
1071
|
+
- Recall may not have access to this information until #query is called
|
1072
|
+
- This is bad and needs to be fixed/rethought
|
1073
|
+
|
1074
|
+
|
1075
|
+
Considerations for the future:
|
1076
|
+
- Probably want to rename these nodes to something better
|
1077
|
+
- It would still be nice to drop a concept into a concept that has "slots" waiting
|
1078
|
+
- Perhaps slot is a different node from "define"
|
1079
|
+
- I had to retool Tree and Query and Graph to expect an array of concepts in a ConceptQL statement
|
1080
|
+
- I'm not sure I like this
|
1081
|
+
- A ConceptQL statement perhaps should be only a single statement at the end
|
1082
|
+
- If an array of sub-concepts is fed into Query, maybe we only execute the last one after parsing the others
|
1083
|
+
- This is consistent with how Sequel wants to live and would yield a single set of results
|
1084
|
+
- I think I like this
|
1085
|
+
- Defines need to occur before they are used
|
1086
|
+
- Most languages have a "forward definition" ability
|
1087
|
+
- I have no use cases for when we might need those?
|
1088
|
+
- Perhaps a definition that uses a definition that doesn't exist?
|
1089
|
+
- Is that recursive?
|
1090
|
+
- Is that something we want/need in ConceptQL?
|
1091
|
+
|
1092
|
+
|
1052
1093
|
### Value Nodes
|
1053
1094
|
So far, we can’t recreate the Charlson comorbidity index using ConceptQL. If we added a “value” node, we could.
|
1054
1095
|
|
data/lib/conceptql/cli.rb
CHANGED
@@ -49,7 +49,7 @@ module ConceptQL
|
|
49
49
|
|
50
50
|
desc 'show_and_tell_file statement_file', 'Reads the ConceptQL statement from the file and shows the contents as a ConceptQL graph, then executes the statement against the DB'
|
51
51
|
option :full
|
52
|
-
def
|
52
|
+
def show_and_tell_file(file)
|
53
53
|
show_and_tell(criteria_from_file(file), options)
|
54
54
|
end
|
55
55
|
|
data/lib/conceptql/graph.rb
CHANGED
@@ -34,14 +34,16 @@ module ConceptQL
|
|
34
34
|
attr :yaml, :tree, :db
|
35
35
|
|
36
36
|
def build_graph(g)
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
|
41
|
-
|
42
|
-
|
43
|
-
|
44
|
-
|
37
|
+
tree.root(self).each.with_index do |last_node, index|
|
38
|
+
last_node.graph_it(g, db)
|
39
|
+
if dangler
|
40
|
+
blank_node = g.add_nodes("_#{index}")
|
41
|
+
blank_node[:shape] = 'none'
|
42
|
+
blank_node[:height] = 0
|
43
|
+
blank_node[:label] = ''
|
44
|
+
blank_node[:fixedsize] = true
|
45
|
+
last_node.link_to(g, blank_node)
|
46
|
+
end
|
45
47
|
end
|
46
48
|
end
|
47
49
|
end
|
@@ -11,6 +11,7 @@ module ConceptQL
|
|
11
11
|
condition: :condition_occurrence,
|
12
12
|
primary_diagnosis: :condition_occurrence,
|
13
13
|
icd9: :condition_occurrence,
|
14
|
+
icd10: :condition_occurrence,
|
14
15
|
condition_type: :condition_occurrence,
|
15
16
|
|
16
17
|
# Procedures
|
@@ -24,6 +25,7 @@ module ConceptQL
|
|
24
25
|
# Visits
|
25
26
|
visit_occurrence: :visit_occurrence,
|
26
27
|
place_of_service: :visit_occurrence,
|
28
|
+
place_of_service_code: :visit_occurrence,
|
27
29
|
|
28
30
|
# Person
|
29
31
|
person: :person,
|
@@ -41,7 +43,10 @@ module ConceptQL
|
|
41
43
|
|
42
44
|
# Drug
|
43
45
|
drug_exposure: :drug_exposure,
|
46
|
+
rxnorm: :drug_exposure,
|
44
47
|
drug_cost: :drug_cost,
|
48
|
+
drug_type_concept_id: :drug_exposure,
|
49
|
+
drug_type_concept: :drug_exposure,
|
45
50
|
|
46
51
|
# Date Nodes
|
47
52
|
date_range: :date,
|
@@ -120,11 +125,62 @@ module ConceptQL
|
|
120
125
|
end
|
121
126
|
end
|
122
127
|
|
128
|
+
class DefineNode < DotNode
|
129
|
+
def initialize(*args)
|
130
|
+
@gn = args.pop
|
131
|
+
super(*args)
|
132
|
+
end
|
133
|
+
|
134
|
+
def types
|
135
|
+
@gn.types[namify(arguments.first)] = super
|
136
|
+
end
|
137
|
+
|
138
|
+
def shape
|
139
|
+
:cds
|
140
|
+
end
|
141
|
+
end
|
142
|
+
|
143
|
+
class FromNode < DotNode
|
144
|
+
def initialize(*args)
|
145
|
+
@gn = args.pop
|
146
|
+
super(*args)
|
147
|
+
end
|
148
|
+
|
149
|
+
def types
|
150
|
+
@gn.types[namify(arguments.first)]
|
151
|
+
end
|
152
|
+
|
153
|
+
def shape
|
154
|
+
:cds
|
155
|
+
end
|
156
|
+
end
|
157
|
+
|
158
|
+
class VsacNode < DotNode
|
159
|
+
def initialize(name, values, types)
|
160
|
+
@types = types
|
161
|
+
super(name, values)
|
162
|
+
end
|
163
|
+
|
164
|
+
def types
|
165
|
+
[ @types ].flatten.compact.map(&:to_sym)
|
166
|
+
end
|
167
|
+
end
|
168
|
+
|
123
169
|
BINARY_OPERATOR_TYPES = %w(before after meets met_by started_by starts contains during overlaps overlapped_by finished_by finishes coincides except person_filter less_than less_than_or_equal equal not_equal greater_than greater_than_or_equal filter).map { |temp| [temp, "not_#{temp}"] }.flatten.map(&:to_sym)
|
124
170
|
|
171
|
+
def types
|
172
|
+
@types ||= {}
|
173
|
+
end
|
125
174
|
def create(type, values)
|
126
175
|
if BINARY_OPERATOR_TYPES.include?(type)
|
127
176
|
return BinaryOperatorNode.new(type, values)
|
177
|
+
elsif type == :define
|
178
|
+
return DefineNode.new(type, values, self)
|
179
|
+
elsif type == :from
|
180
|
+
return FromNode.new(type, values, self)
|
181
|
+
elsif type == :vsac
|
182
|
+
types = values.pop
|
183
|
+
return VsacNode.new(type, values, types)
|
128
184
|
end
|
129
185
|
DotNode.new(type, values)
|
130
186
|
end
|
@@ -24,7 +24,11 @@ module ConceptQL
|
|
24
24
|
# rows in its table as results.
|
25
25
|
class CastingNode < Node
|
26
26
|
def types
|
27
|
-
[
|
27
|
+
[type]
|
28
|
+
end
|
29
|
+
|
30
|
+
def type
|
31
|
+
my_type
|
28
32
|
end
|
29
33
|
|
30
34
|
def castables
|
@@ -39,36 +43,52 @@ module ConceptQL
|
|
39
43
|
private
|
40
44
|
|
41
45
|
def base_query(db, stream_query)
|
42
|
-
|
43
|
-
|
46
|
+
uncastable_types = stream.types - castables
|
47
|
+
to_me_types = stream.types & these_point_at_me
|
48
|
+
from_me_types = stream.types & i_point_at
|
49
|
+
|
50
|
+
destination_table = make_table_name(my_type)
|
51
|
+
casting_query = db.from(destination_table)
|
52
|
+
wheres = []
|
53
|
+
|
54
|
+
unless uncastable_types.empty?
|
55
|
+
# We have a situation where one or more of the incoming streams
|
44
56
|
# isn't castable so we'll just return all rows for
|
45
|
-
# all people
|
46
|
-
db.from(
|
47
|
-
.where(
|
48
|
-
|
49
|
-
|
57
|
+
# all people in each uncastable stream
|
58
|
+
uncastable_person_ids = db.from(stream_query)
|
59
|
+
.where(criterion_type: uncastable_types.map(&:to_s))
|
60
|
+
.select_group(:person_id)
|
61
|
+
wheres << Sequel.expr(person_id: uncastable_person_ids)
|
62
|
+
end
|
63
|
+
|
64
|
+
destination_type_id = type_id(my_type)
|
65
|
+
|
66
|
+
unless to_me_types.empty?
|
67
|
+
# For each castable type in the stream, setup a query that
|
50
68
|
# casts each type to a set of IDs, union those IDs and fetch
|
51
69
|
# them from the source table
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
70
|
+
castable_type_queries = to_me_types.map do |source_type|
|
71
|
+
source_ids = db.from(stream_query)
|
72
|
+
.where(criterion_type: source_type.to_s)
|
73
|
+
.select_group(:criterion_id)
|
74
|
+
source_table = make_table_name(source_type)
|
75
|
+
source_type_id = type_id(source_type)
|
57
76
|
|
58
|
-
|
59
|
-
|
77
|
+
db.from(source_table)
|
78
|
+
.where(source_type_id => source_ids)
|
79
|
+
.select(destination_type_id)
|
80
|
+
end
|
81
|
+
wheres << Sequel.expr(destination_type_id => castable_type_queries)
|
60
82
|
end
|
61
|
-
end
|
62
83
|
|
63
|
-
|
64
|
-
|
65
|
-
|
66
|
-
.
|
67
|
-
|
68
|
-
db.from(make_table_name(type))
|
69
|
-
.where(type_id(type) => db.from(stream_query.select_group(type_id(type))))
|
84
|
+
unless from_me_types.empty?
|
85
|
+
from_me_types.each do |from_me_type|
|
86
|
+
fk_type_id = type_id(from_me_type)
|
87
|
+
wheres << Sequel.expr(fk_type_id => db.from(stream_query).where(criterion_type: from_me_type.to_s).select_group(:criterion_id))
|
88
|
+
end
|
70
89
|
end
|
71
|
-
|
90
|
+
|
91
|
+
casting_query.where(wheres.inject(&:|))
|
72
92
|
end
|
73
93
|
end
|
74
94
|
end
|
@@ -6,11 +6,30 @@ module ConceptQL
|
|
6
6
|
def query(db)
|
7
7
|
child = children.first
|
8
8
|
child.types.map do |type|
|
9
|
-
|
9
|
+
positive_query = db.from(child.evaluate(db))
|
10
|
+
.select(:criterion_id)
|
11
|
+
.exclude(:criterion_id => nil)
|
12
|
+
.where(:criterion_type => type.to_s)
|
13
|
+
query = db.from(make_table_name(type))
|
14
|
+
.exclude(type_id(type) => positive_query)
|
15
|
+
db.from(select_it(query, type))
|
10
16
|
end.inject do |union_query, q|
|
11
17
|
union_query.union(q, all: true)
|
12
18
|
end
|
13
19
|
end
|
20
|
+
|
21
|
+
=begin
|
22
|
+
This is an alternate, but equally accurate way to do complement.
|
23
|
+
We'll need to benchmark which is faster.
|
24
|
+
def query2(db)
|
25
|
+
child = children.first
|
26
|
+
froms = child.types.map do |type|
|
27
|
+
select_it(db.from(make_table_name(type)), type)
|
28
|
+
end
|
29
|
+
big_from = froms.inject { |union_query, q| union_query.union(q, all:true) }
|
30
|
+
db.from(big_from).except(child.evaluate(db))
|
31
|
+
end
|
32
|
+
=end
|
14
33
|
end
|
15
34
|
end
|
16
35
|
end
|
@@ -10,12 +10,12 @@ module ConceptQL
|
|
10
10
|
#
|
11
11
|
# Multiple types can be specified at once
|
12
12
|
class ConditionType < Node
|
13
|
-
def
|
14
|
-
|
13
|
+
def type
|
14
|
+
:condition_occurrence
|
15
15
|
end
|
16
16
|
|
17
17
|
def query(db)
|
18
|
-
db.from(:
|
18
|
+
db.from(:condition_occurrence)
|
19
19
|
.where(condition_type_concept_id: condition_occurrence_type_concept_ids)
|
20
20
|
end
|
21
21
|
|
@@ -10,6 +10,8 @@ module ConceptQL
|
|
10
10
|
class DateRange < Node
|
11
11
|
def query(db)
|
12
12
|
db.from(:person)
|
13
|
+
.select_append(Sequel.cast('person', :text).as(:criterion_type))
|
14
|
+
.select_append(Sequel.expr(:person_id).as(:criterion_id))
|
13
15
|
.select_append(Sequel.expr(start_date(db)).cast(:date).as(:start_date),Sequel.expr(end_date(db)).cast(:date).as(:end_date)).from_self
|
14
16
|
end
|
15
17
|
|
@@ -0,0 +1,75 @@
|
|
1
|
+
require_relative 'node'
|
2
|
+
|
3
|
+
module ConceptQL
|
4
|
+
module Nodes
|
5
|
+
# Mimics creating a variable name that stores an itermediate result as
|
6
|
+
# part of a larger concept
|
7
|
+
#
|
8
|
+
# The idea is that a concept might be very complex and it helps to break
|
9
|
+
# that complex concept into a set of sub-concepts to better understand it.
|
10
|
+
#
|
11
|
+
# Also, sometimes a particular piece of a concept is used in many places,
|
12
|
+
# so it makes sense to write that piece out once, store it as a "variable"
|
13
|
+
# and then insert that variable into the concept as needed.
|
14
|
+
# run the query once and subsequent calls
|
15
|
+
class Define < Node
|
16
|
+
# Create a temporary table and store the stream of results in that table.
|
17
|
+
# This "caches" the results so we only have to execute stream's query
|
18
|
+
# once.
|
19
|
+
#
|
20
|
+
# The logic here is that if something is assigned to a variable, chances
|
21
|
+
# are that it will be used more than once, so why run the query more than
|
22
|
+
# once?
|
23
|
+
#
|
24
|
+
# ConceptQL's SQL generator normally translates the entire statement into
|
25
|
+
# one, large query that can be executed later.
|
26
|
+
#
|
27
|
+
# Unfortunately, Sequel's "create_table" function actually executes the
|
28
|
+
# 'CREATE TABLE' SQL right away, meaning that the "define" node will
|
29
|
+
# execute immediately _during_ the processing of the ConceptQL statement.
|
30
|
+
# We'll see what kinds of problems this causes
|
31
|
+
#
|
32
|
+
# Lastly, this node does NOT pass its results to the next node. The
|
33
|
+
# reason for this exception is to allow us to return the SQL that
|
34
|
+
# generates the temp table. This is done so that the ConceptQL sandbox
|
35
|
+
# can return the entire set of SQL statements needed to run a query.
|
36
|
+
#
|
37
|
+
# Perhaps in the future we can find a way around this.
|
38
|
+
#
|
39
|
+
# Also, things will blow up if you try to use a variable that hasn't been
|
40
|
+
# defined yet.
|
41
|
+
def query(db)
|
42
|
+
table_name = namify(arguments.first)
|
43
|
+
stash_types(db, table_name, types)
|
44
|
+
db.create_table!(table_name, temp: true, as: stream.evaluate(db))
|
45
|
+
db[db.send(:create_table_as_sql, table_name, stream.evaluate(db).sql, temp: true)]
|
46
|
+
end
|
47
|
+
|
48
|
+
def types
|
49
|
+
stream.types
|
50
|
+
end
|
51
|
+
|
52
|
+
private
|
53
|
+
|
54
|
+
# TODO: Fix this. This is shameful.
|
55
|
+
# In order to remember which types a table is storing, we need to preserve
|
56
|
+
# the type information outside of the "define" statement because the "recall"
|
57
|
+
# statement most likely doesn't have a reference to this node (that's the
|
58
|
+
# whole point).
|
59
|
+
#
|
60
|
+
# There is only one object shared between all the nodes: the database
|
61
|
+
# connection. We'll do something terrible and piggyback the type
|
62
|
+
# information about this variable on the database connection so that
|
63
|
+
# the "recall" node can pull that information back out.
|
64
|
+
#
|
65
|
+
# Ugh.
|
66
|
+
def stash_types(db, name, types)
|
67
|
+
def db.types
|
68
|
+
@types ||= {}
|
69
|
+
end
|
70
|
+
db.types[name] = types
|
71
|
+
end
|
72
|
+
end
|
73
|
+
end
|
74
|
+
end
|
75
|
+
|
@@ -0,0 +1,18 @@
|
|
1
|
+
require_relative 'node'
|
2
|
+
|
3
|
+
module ConceptQL
|
4
|
+
module Nodes
|
5
|
+
class DrugTypeConcept < Node
|
6
|
+
def type
|
7
|
+
:drug_exposure
|
8
|
+
end
|
9
|
+
|
10
|
+
def query(db)
|
11
|
+
db.from(:drug_exposure)
|
12
|
+
.where(drug_type_concept_id: arguments)
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
16
|
+
end
|
17
|
+
|
18
|
+
|
@@ -3,8 +3,8 @@ require_relative 'node'
|
|
3
3
|
module ConceptQL
|
4
4
|
module Nodes
|
5
5
|
class Gender < Node
|
6
|
-
def
|
7
|
-
|
6
|
+
def type
|
7
|
+
:person
|
8
8
|
end
|
9
9
|
|
10
10
|
def query(db)
|
@@ -19,7 +19,7 @@ module ConceptQL
|
|
19
19
|
end
|
20
20
|
end
|
21
21
|
|
22
|
-
db.from(:
|
22
|
+
db.from(:person)
|
23
23
|
.where(gender_concept_id: gender_concept_ids)
|
24
24
|
end
|
25
25
|
end
|
@@ -10,8 +10,9 @@ module ConceptQL
|
|
10
10
|
def query(db)
|
11
11
|
exprs = {}
|
12
12
|
values.each do |expression|
|
13
|
+
evaled = expression.evaluate(db)
|
13
14
|
expression.types.each do |type|
|
14
|
-
(exprs[type] ||= []) <<
|
15
|
+
(exprs[type] ||= []) << evaled
|
15
16
|
end
|
16
17
|
end
|
17
18
|
typed_queries = exprs.map do |type, queries|
|
data/lib/conceptql/nodes/node.rb
CHANGED
@@ -2,18 +2,6 @@ require 'active_support/core_ext/hash'
|
|
2
2
|
module ConceptQL
|
3
3
|
module Nodes
|
4
4
|
class Node
|
5
|
-
KNOWN_TYPES = %i(
|
6
|
-
condition_occurrence
|
7
|
-
death
|
8
|
-
drug_cost
|
9
|
-
drug_exposure
|
10
|
-
observation
|
11
|
-
payer_plan_period
|
12
|
-
procedure_cost
|
13
|
-
procedure_occurrence
|
14
|
-
visit_occurrence
|
15
|
-
)
|
16
|
-
|
17
5
|
attr :values, :options
|
18
6
|
def initialize(*args)
|
19
7
|
args.flatten!
|
@@ -28,12 +16,13 @@ module ConceptQL
|
|
28
16
|
select_it(query(db))
|
29
17
|
end
|
30
18
|
|
31
|
-
def select_it(query,
|
32
|
-
|
19
|
+
def select_it(query, specific_type = nil)
|
20
|
+
specific_type = type if specific_type.nil? && respond_to?(:type)
|
21
|
+
query.select(*columns(specific_type))
|
33
22
|
end
|
34
23
|
|
35
24
|
def types
|
36
|
-
@types ||=
|
25
|
+
@types ||= determine_types
|
37
26
|
end
|
38
27
|
|
39
28
|
def children
|
@@ -48,23 +37,98 @@ module ConceptQL
|
|
48
37
|
@arguments ||= values.reject { |v| v.is_a?(Node) }
|
49
38
|
end
|
50
39
|
|
51
|
-
def columns(
|
52
|
-
|
53
|
-
|
54
|
-
|
40
|
+
def columns(local_type = nil)
|
41
|
+
criterion_type = Sequel.expr(:criterion_type)
|
42
|
+
if local_type
|
43
|
+
criterion_type = Sequel.expr(local_type.to_s).cast(:text)
|
44
|
+
end
|
45
|
+
[:person_id___person_id,
|
46
|
+
Sequel.expr(type_id(local_type)).as(:criterion_id),
|
47
|
+
criterion_type.as(:criterion_type)] + date_columns(local_type)
|
55
48
|
end
|
56
49
|
|
57
|
-
|
58
|
-
|
50
|
+
private
|
51
|
+
def criterion_id
|
52
|
+
:criterion_id
|
59
53
|
end
|
60
54
|
|
61
|
-
|
62
|
-
|
55
|
+
def type_id(type = nil)
|
56
|
+
return :criterion_id if type.nil?
|
57
|
+
type = :person if type == :death
|
63
58
|
(type.to_s + '_id').to_sym
|
64
59
|
end
|
65
60
|
|
66
61
|
def make_table_name(table)
|
67
|
-
"#{table}
|
62
|
+
"#{table}___tab".to_sym
|
63
|
+
end
|
64
|
+
|
65
|
+
def date_columns(type = nil)
|
66
|
+
return [:start_date, :end_date] unless type
|
67
|
+
sd = start_date_column(type)
|
68
|
+
sd = Sequel.expr(sd).cast(:date).as(:start_date) unless sd == :start_date
|
69
|
+
ed = end_date_column(type)
|
70
|
+
ed = Sequel.expr(ed).cast(:date).as(:end_date) unless ed == :end_date
|
71
|
+
[sd, ed]
|
72
|
+
end
|
73
|
+
|
74
|
+
def start_date_column(type)
|
75
|
+
{
|
76
|
+
condition_occurrence: :condition_start_date,
|
77
|
+
death: :death_date,
|
78
|
+
drug_exposure: :drug_exposure_start_date,
|
79
|
+
drug_cost: nil,
|
80
|
+
payer_plan_period: :payer_plan_period_start_date,
|
81
|
+
person: person_date_of_birth,
|
82
|
+
procedure_occurrence: :procedure_date,
|
83
|
+
procedure_cost: nil,
|
84
|
+
observation: :observation_date,
|
85
|
+
visit_occurrence: :visit_start_date
|
86
|
+
}[type]
|
87
|
+
end
|
88
|
+
|
89
|
+
def end_date_column(type)
|
90
|
+
{
|
91
|
+
condition_occurrence: :condition_end_date,
|
92
|
+
death: :death_date,
|
93
|
+
drug_exposure: :drug_exposure_end_date,
|
94
|
+
drug_cost: nil,
|
95
|
+
payer_plan_period: :payer_plan_period_end_date,
|
96
|
+
person: person_date_of_birth,
|
97
|
+
procedure_occurrence: :procedure_date,
|
98
|
+
procedure_cost: nil,
|
99
|
+
observation: :observation_date,
|
100
|
+
visit_occurrence: :visit_end_date
|
101
|
+
}[type]
|
102
|
+
end
|
103
|
+
|
104
|
+
def person_date_of_birth
|
105
|
+
assemble_date(:year_of_birth, :month_of_birth, :day_of_birth)
|
106
|
+
end
|
107
|
+
|
108
|
+
def assemble_date(*symbols)
|
109
|
+
strings = symbols.map do |symbol|
|
110
|
+
Sequel.function(:coalesce, Sequel.expr(symbol).cast(:text), Sequel.expr('01').cast(:text)).cast(:text)
|
111
|
+
end
|
112
|
+
strings = strings.zip(['-'] * (symbols.length - 1)).flatten.compact
|
113
|
+
Sequel.function(:date, Sequel.join(strings))
|
114
|
+
end
|
115
|
+
|
116
|
+
def determine_types
|
117
|
+
if children.empty?
|
118
|
+
if respond_to?(:type)
|
119
|
+
[type]
|
120
|
+
else
|
121
|
+
raise "Node doesn't seem to specify any type"
|
122
|
+
end
|
123
|
+
else
|
124
|
+
children.map(&:types).flatten.uniq
|
125
|
+
end
|
126
|
+
end
|
127
|
+
|
128
|
+
def namify(name)
|
129
|
+
require 'digest'
|
130
|
+
digest = Digest::SHA256.hexdigest name
|
131
|
+
('_' + digest).to_sym
|
68
132
|
end
|
69
133
|
end
|
70
134
|
end
|
@@ -23,7 +23,7 @@ module ConceptQL
|
|
23
23
|
# occurrence, this node returns nothing for that person
|
24
24
|
class Occurrence < Node
|
25
25
|
def query(db)
|
26
|
-
db.from(stream.evaluate(db)
|
26
|
+
db.from(db.from(stream.evaluate(db))
|
27
27
|
.select_append { |o| o.row_number(:over, partition: :person_id, order: ordered_columns){}.as(:rn) })
|
28
28
|
.where(rn: occurrence.abs)
|
29
29
|
end
|
@@ -39,7 +39,7 @@ module ConceptQL
|
|
39
39
|
|
40
40
|
def ordered_columns
|
41
41
|
ordered_columns = [Sequel.send(asc_or_desc, :start_date)]
|
42
|
-
ordered_columns +=
|
42
|
+
ordered_columns += [:criterion_type, :criterion_id]
|
43
43
|
end
|
44
44
|
end
|
45
45
|
end
|
@@ -9,14 +9,15 @@ module ConceptQL
|
|
9
9
|
# concept_name column of the concept table. If you misspell the place_of_service_code name
|
10
10
|
# you won't get any matches
|
11
11
|
class PlaceOfServiceCode < Node
|
12
|
-
def
|
13
|
-
|
12
|
+
def type
|
13
|
+
:visit_occurrence
|
14
14
|
end
|
15
15
|
|
16
16
|
def query(db)
|
17
|
-
db.from(:
|
17
|
+
db.from(:visit_occurrence___v)
|
18
18
|
.join(:vocabulary__concept___vc, { vc__concept_id: :v__place_of_service_concept_id })
|
19
19
|
.where(vc__concept_code: arguments.map(&:to_s))
|
20
|
+
.where(vc__vocabulary_id: 14)
|
20
21
|
end
|
21
22
|
end
|
22
23
|
end
|
data/lib/conceptql/nodes/race.rb
CHANGED
@@ -9,12 +9,12 @@ module ConceptQL
|
|
9
9
|
# concept_name column of the concept table. If you misspell the race name
|
10
10
|
# you won't get any matches
|
11
11
|
class Race < Node
|
12
|
-
def
|
13
|
-
|
12
|
+
def type
|
13
|
+
:person
|
14
14
|
end
|
15
15
|
|
16
16
|
def query(db)
|
17
|
-
db.from(:
|
17
|
+
db.from(:person___p)
|
18
18
|
.join(:vocabulary__concept___vc, { vc__concept_id: :p__race_concept_id })
|
19
19
|
.where(Sequel.function(:lower, :vc__concept_name) => arguments.map(&:downcase))
|
20
20
|
end
|
@@ -0,0 +1,34 @@
|
|
1
|
+
require_relative 'pass_thru'
|
2
|
+
|
3
|
+
module ConceptQL
|
4
|
+
module Nodes
|
5
|
+
# Mimics using a variable that has been set via "define" node
|
6
|
+
#
|
7
|
+
# The idea is that a concept might be very complex and it helps to break
|
8
|
+
# that complex concept into a set of sub-concepts to better understand it.
|
9
|
+
#
|
10
|
+
# This node will look for a sub-concept that has been created through the
|
11
|
+
# "define" node and will fetch the results cached in the corresponding table
|
12
|
+
class Recall < Node
|
13
|
+
# Behind the scenes we simply fetch all rows from the temp table that
|
14
|
+
# corresponds to the name fed to "recall"
|
15
|
+
#
|
16
|
+
# We also set the @types variable by pulling the type information out
|
17
|
+
# of the hash piggybacking on the database connection.
|
18
|
+
#
|
19
|
+
# TODO: This might be an issue since we might need the type information
|
20
|
+
# before we call #query. Probably time to reevaluate how we're caching
|
21
|
+
# the type information.
|
22
|
+
def query(db)
|
23
|
+
table_name = namify(arguments.first)
|
24
|
+
@types = db.types[table_name]
|
25
|
+
db.from(table_name)
|
26
|
+
end
|
27
|
+
|
28
|
+
def types
|
29
|
+
@types
|
30
|
+
end
|
31
|
+
end
|
32
|
+
end
|
33
|
+
end
|
34
|
+
|
@@ -32,11 +32,12 @@ module ConceptQL
|
|
32
32
|
.where(Sequel.expr(scm__source_code: values, scm__source_vocabulary_id: vocabulary_id).&(Sequel.expr(scm__source_code: table_source_column)))
|
33
33
|
end
|
34
34
|
|
35
|
-
def
|
36
|
-
|
35
|
+
def type
|
36
|
+
table
|
37
37
|
end
|
38
38
|
|
39
39
|
private
|
40
|
+
|
40
41
|
def table_name
|
41
42
|
@table_name ||= make_table_name(table)
|
42
43
|
end
|
@@ -9,8 +9,8 @@ module ConceptQL
|
|
9
9
|
# Proc.new { l.end_date < r.start_date }
|
10
10
|
class TemporalNode < BinaryOperatorNode
|
11
11
|
def query(db)
|
12
|
-
db.from(db.from(
|
13
|
-
.join(
|
12
|
+
db.from(db.from(left_stream(db))
|
13
|
+
.join(right_stream(db), [:person_id])
|
14
14
|
.where(where_clause)
|
15
15
|
.select_all(:l))
|
16
16
|
end
|
@@ -18,6 +18,14 @@ module ConceptQL
|
|
18
18
|
def inclusive?
|
19
19
|
options[:inclusive]
|
20
20
|
end
|
21
|
+
|
22
|
+
def left_stream(db)
|
23
|
+
Sequel.expr(left.evaluate(db)).as(:l)
|
24
|
+
end
|
25
|
+
|
26
|
+
def right_stream(db)
|
27
|
+
Sequel.expr(right.evaluate(db)).as(:r)
|
28
|
+
end
|
21
29
|
end
|
22
30
|
end
|
23
31
|
end
|
data/lib/conceptql/query.rb
CHANGED
@@ -1,6 +1,5 @@
|
|
1
1
|
require 'psych'
|
2
2
|
require_relative 'tree'
|
3
|
-
require_relative 'view_maker'
|
4
3
|
|
5
4
|
module ConceptQL
|
6
5
|
class Query
|
@@ -11,29 +10,34 @@ module ConceptQL
|
|
11
10
|
@tree = tree
|
12
11
|
end
|
13
12
|
|
14
|
-
def
|
13
|
+
def queries
|
15
14
|
build_query(db)
|
16
15
|
end
|
17
16
|
|
17
|
+
def query
|
18
|
+
queries.last
|
19
|
+
end
|
20
|
+
|
21
|
+
def sql
|
22
|
+
queries.map(&:sql).join(";\n") + ';'
|
23
|
+
end
|
24
|
+
|
25
|
+
# To avoid a performance penalty, only execute the last
|
26
|
+
# SQL statement in an array of ConceptQL statements so that define's
|
27
|
+
# "create_table" SQL isn't executed twice
|
18
28
|
def execute
|
19
|
-
|
20
|
-
build_query(db).all
|
29
|
+
query.all
|
21
30
|
end
|
22
31
|
|
23
32
|
def types
|
24
|
-
tree.root(self).types
|
33
|
+
tree.root(self).last.types
|
25
34
|
end
|
26
35
|
|
27
36
|
private
|
28
37
|
attr :yaml, :tree, :db
|
29
38
|
|
30
39
|
def build_query(db)
|
31
|
-
tree.root(self).evaluate(db)
|
32
|
-
end
|
33
|
-
|
34
|
-
def ensure_views
|
35
|
-
return if db.views.include?(:person_with_dates)
|
36
|
-
ViewMaker.make_views(db)
|
40
|
+
tree.root(self).map { |n| n.evaluate(db) }
|
37
41
|
end
|
38
42
|
end
|
39
43
|
end
|
data/lib/conceptql/tree.rb
CHANGED
@@ -9,8 +9,8 @@ module ConceptQL
|
|
9
9
|
@behavior = opts.fetch(:behavior, nil)
|
10
10
|
end
|
11
11
|
|
12
|
-
def root(
|
13
|
-
@root ||= traverse(
|
12
|
+
def root(*queries)
|
13
|
+
@root ||= traverse(queries.flatten.map(&:statement).flatten.map(&:deep_symbolize_keys))
|
14
14
|
end
|
15
15
|
|
16
16
|
private
|
data/lib/conceptql/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: conceptql
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.4
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ryan Duryea
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-08-
|
11
|
+
date: 2014-08-20 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activesupport
|
@@ -175,6 +175,8 @@ files:
|
|
175
175
|
- lib/conceptql/nodes/cpt.rb
|
176
176
|
- lib/conceptql/nodes/date_range.rb
|
177
177
|
- lib/conceptql/nodes/death.rb
|
178
|
+
- lib/conceptql/nodes/define.rb
|
179
|
+
- lib/conceptql/nodes/drug_type_concept.rb
|
178
180
|
- lib/conceptql/nodes/during.rb
|
179
181
|
- lib/conceptql/nodes/except.rb
|
180
182
|
- lib/conceptql/nodes/first.rb
|
@@ -195,6 +197,7 @@ files:
|
|
195
197
|
- lib/conceptql/nodes/place_of_service_code.rb
|
196
198
|
- lib/conceptql/nodes/procedure_occurrence.rb
|
197
199
|
- lib/conceptql/nodes/race.rb
|
200
|
+
- lib/conceptql/nodes/recall.rb
|
198
201
|
- lib/conceptql/nodes/rxnorm.rb
|
199
202
|
- lib/conceptql/nodes/snomed.rb
|
200
203
|
- lib/conceptql/nodes/source_vocabulary_node.rb
|
@@ -209,7 +212,6 @@ files:
|
|
209
212
|
- lib/conceptql/query.rb
|
210
213
|
- lib/conceptql/tree.rb
|
211
214
|
- lib/conceptql/version.rb
|
212
|
-
- lib/conceptql/view_maker.rb
|
213
215
|
- spec/conceptql/behaviors/dottable_spec.rb
|
214
216
|
- spec/conceptql/date_adjuster_spec.rb
|
215
217
|
- spec/conceptql/nodes/after_spec.rb
|
data/lib/conceptql/view_maker.rb
DELETED
@@ -1,56 +0,0 @@
|
|
1
|
-
require 'sequel'
|
2
|
-
|
3
|
-
module ConceptQL
|
4
|
-
module ViewMaker
|
5
|
-
def self.make_views(db, schema = nil)
|
6
|
-
views.each do |view, columns|
|
7
|
-
make_view(db, schema, view, columns)
|
8
|
-
end
|
9
|
-
end
|
10
|
-
|
11
|
-
def self.make_view(db, schema, view, columns)
|
12
|
-
view_name = (view.to_s + '_with_dates')
|
13
|
-
view_name = schema + '__' + view_name if schema
|
14
|
-
view_name = view_name.to_sym
|
15
|
-
|
16
|
-
table_name = view.to_s
|
17
|
-
table_name = schema + '__' + table_name if schema
|
18
|
-
table_name = table_name.to_sym
|
19
|
-
|
20
|
-
additional_columns = [Sequel.expr(columns.shift).cast(:date).as(:start_date), Sequel.expr(columns.shift).cast(:date).as(:end_date)]
|
21
|
-
unless columns.empty?
|
22
|
-
additional_columns += columns.last.map do |column_name, column_value|
|
23
|
-
Sequel.expr(column_value).as(column_name)
|
24
|
-
end
|
25
|
-
end
|
26
|
-
query = db.from(table_name).select_all.select_append(*additional_columns)
|
27
|
-
puts query.sql
|
28
|
-
db.drop_view(view_name, if_exists: true)
|
29
|
-
db.create_view(view_name, query)
|
30
|
-
end
|
31
|
-
|
32
|
-
def self.views
|
33
|
-
person_date_of_birth = assemble_date(:year_of_birth, :month_of_birth, :day_of_birth)
|
34
|
-
{
|
35
|
-
condition_occurrence: [:condition_start_date, :condition_end_date],
|
36
|
-
death: [:death_date, :death_date, { death_id: :person_id }],
|
37
|
-
drug_exposure: [:drug_exposure_start_date, :drug_exposure_end_date],
|
38
|
-
drug_cost: [nil, nil],
|
39
|
-
payer_plan_period: [:payer_plan_period_start_date, :payer_plan_period_end_date],
|
40
|
-
person: [person_date_of_birth, person_date_of_birth],
|
41
|
-
procedure_occurrence: [:procedure_date, :procedure_date],
|
42
|
-
procedure_cost: [nil, nil],
|
43
|
-
observation: [:observation_date, :observation_date],
|
44
|
-
visit_occurrence: [:visit_start_date, :visit_end_date]
|
45
|
-
}
|
46
|
-
end
|
47
|
-
|
48
|
-
def self.assemble_date(*symbols)
|
49
|
-
strings = symbols.map do |symbol|
|
50
|
-
Sequel.function(:coalesce, symbol, '01').cast(:text)
|
51
|
-
end
|
52
|
-
strings = strings.zip(['-'] * (symbols.length - 1)).flatten.compact
|
53
|
-
Sequel.function(:date, Sequel.join(strings))
|
54
|
-
end
|
55
|
-
end
|
56
|
-
end
|