db_fuel 1.0.0.pre.alpha → 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +5 -2
- data/README.md +175 -3
- data/db_fuel.gemspec +2 -2
- data/lib/db_fuel/library/dbee/base.rb +5 -0
- data/lib/db_fuel/library/dbee/query.rb +2 -2
- data/lib/db_fuel/library/dbee/range.rb +20 -7
- data/lib/db_fuel/version.rb +1 -1
- metadata +7 -6
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 181a57c237b43052c5aa968663e4d3d6d8e6141a3affaf5b0de1ad4147e0725c
|
4
|
+
data.tar.gz: 817df3889c7c39f228cc2a860e19cd1cefb97f9ba23697972e04aa163413b359
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: d8bdbd205ee2c03f2954057f199c58e7d94d2ee64ebcc3735bb4a380dea175be7eb0ed9ac05e1dff12f985f8831e2bb15d76227d61bea0c4e3091b4185cb47d8
|
7
|
+
data.tar.gz: f03df133b8563d7e035654ccaab7c4789a818683dd68a4bf5471d4d7261ea53fcddf989ab5c5e91cbfd3bbc57cb1ed75ab86ec355496571aeb5b816ff4a19f35
|
data/CHANGELOG.md
CHANGED
data/README.md
CHANGED
@@ -1,5 +1,177 @@
|
|
1
|
-
#
|
1
|
+
# DB Fuel
|
2
2
|
|
3
|
-
|
3
|
+
[![Gem Version](https://badge.fury.io/rb/db_fuel.svg)](https://badge.fury.io/rb/db_fuel) [![Build Status](https://travis-ci.org/bluemarblepayroll/db_fuel.svg?branch=master)](https://travis-ci.org/bluemarblepayroll/db_fuel) [![Maintainability](https://api.codeclimate.com/v1/badges/21945483950d9c35fabb/maintainability)](https://codeclimate.com/github/bluemarblepayroll/db_fuel/maintainability) [![Test Coverage](https://api.codeclimate.com/v1/badges/21945483950d9c35fabb/test_coverage)](https://codeclimate.com/github/bluemarblepayroll/db_fuel/test_coverage) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
|
4
4
|
|
5
|
-
|
5
|
+
This library is a plugin for [Burner](https://github.com/bluemarblepayroll/burner). Burner, by itself, cannot use a database. So, if you wish to use a database as a data source or as a target for mutation then you need to add a library similar to this.
|
6
|
+
|
7
|
+
## Installation
|
8
|
+
|
9
|
+
To install through Rubygems:
|
10
|
+
|
11
|
+
````bash
|
12
|
+
gem install db_fuel
|
13
|
+
````
|
14
|
+
|
15
|
+
You can also add this to your Gemfile:
|
16
|
+
|
17
|
+
````bash
|
18
|
+
bundle add db_fuel
|
19
|
+
````
|
20
|
+
|
21
|
+
## Jobs
|
22
|
+
|
23
|
+
Refer to the [Burner](https://github.com/bluemarblepayroll/burner) library for more specific information on how Burner works. This section will just focus on what this library directly adds.
|
24
|
+
|
25
|
+
* **db_fuel/dbee/query** [model, query, register]: Pass in a [Dbee](https://github.com/bluemarblepayroll/dbee) model and query and store the results in the specified register. Refer to the [Dbee](https://github.com/bluemarblepayroll/dbee) library directly on how to craft a model or query.
|
26
|
+
* **db_fuel/dbee/range** [key, key_path, model, query, register, separator]: Similar to `db_fuel/dbee/query` with the addition of being able to grab a list of values from the register to use as a Dbee EQUALS/IN filter. This helps to dynamically limit the resulting record set. The key is used to specify where to grab the list of values, while the key_path will be used to craft the [Dbee equal's filter](https://github.com/bluemarblepayroll/dbee/blob/master/lib/dbee/query/filters/equals.rb). Separator is exposed in case nested object support is necessary.
|
27
|
+
|
28
|
+
## Examples
|
29
|
+
|
30
|
+
### Querying the Database
|
31
|
+
|
32
|
+
The `db_fuel/dbee/query` job can be utilized to process a SQL query and store the results in a Burner::Payload register.
|
33
|
+
|
34
|
+
Let's say for example we have a list of patients we would like to retrieve:
|
35
|
+
|
36
|
+
````ruby
|
37
|
+
pipeline = {
|
38
|
+
jobs: [
|
39
|
+
{
|
40
|
+
name: 'retrieve_patients',
|
41
|
+
type: 'db_fuel/dbee/query',
|
42
|
+
model: {
|
43
|
+
name: :patients
|
44
|
+
},
|
45
|
+
query: {
|
46
|
+
fields: [
|
47
|
+
{ key_path: :id },
|
48
|
+
{ key_path: :first_name }
|
49
|
+
],
|
50
|
+
sorters: [
|
51
|
+
{ key_path: :first_name }
|
52
|
+
]
|
53
|
+
},
|
54
|
+
register: :patients
|
55
|
+
}
|
56
|
+
],
|
57
|
+
steps: %w[retrieve_patients]
|
58
|
+
}
|
59
|
+
|
60
|
+
payload = Burner::Payload.new
|
61
|
+
|
62
|
+
Burner::Pipeline.make(pipeline).execute(payload: payload)
|
63
|
+
````
|
64
|
+
|
65
|
+
If we were to inspect the contents of `payload` we should see the patient's result set loaded:
|
66
|
+
|
67
|
+
````ruby
|
68
|
+
payload['patients'] # array in form of: [ { "id" => 1, "first_name" => "Something" }, ... ]
|
69
|
+
````
|
70
|
+
|
71
|
+
### Limiting Result Sets
|
72
|
+
|
73
|
+
The `db_fuel/dbee/query` does not provide a way to dynamically connect the query to existing data. You are free to put any Dbee query filters in the query declaration, but what if you would like to further limit this based on the knowledge of a range of values? The `db_fuel/dbee/range` job is meant to do exactly this. On the surface it is mainly an extension of the `db_fuel/dbee/query` job.
|
74
|
+
|
75
|
+
Let's say we would like to query patients but we want to limit it to an inputted list of first names:
|
76
|
+
|
77
|
+
````ruby
|
78
|
+
pipeline = {
|
79
|
+
jobs: [
|
80
|
+
{
|
81
|
+
name: :load_first_names,
|
82
|
+
type: 'b/value/static',
|
83
|
+
register: :patients,
|
84
|
+
value: [
|
85
|
+
{ fname: 'Bozo' },
|
86
|
+
{ fname: 'Bugs' },
|
87
|
+
]
|
88
|
+
},
|
89
|
+
{
|
90
|
+
name: 'retrieve_patients',
|
91
|
+
type: 'db_fuel/dbee/range',
|
92
|
+
model: {
|
93
|
+
name: :patients
|
94
|
+
},
|
95
|
+
query: {
|
96
|
+
fields: [
|
97
|
+
{ key_path: :id },
|
98
|
+
{ key_path: :first_name }
|
99
|
+
],
|
100
|
+
sorters: [
|
101
|
+
{ key_path: :first_name }
|
102
|
+
]
|
103
|
+
},
|
104
|
+
register: :patients,
|
105
|
+
key: :fname,
|
106
|
+
key_path: :first_name
|
107
|
+
}
|
108
|
+
],
|
109
|
+
steps: %w[load_first_names retrieve_patients]
|
110
|
+
}
|
111
|
+
|
112
|
+
payload = Burner::Payload.new
|
113
|
+
|
114
|
+
Burner::Pipeline.make(pipeline).execute(payload: payload)
|
115
|
+
````
|
116
|
+
|
117
|
+
If we were to inspect the contents of `payload` we should see the patient's result set loaded:
|
118
|
+
|
119
|
+
````ruby
|
120
|
+
payload['patients'] # array in form of: [ { "id" => 1, "first_name" => "Something" }, ... ]
|
121
|
+
````
|
122
|
+
|
123
|
+
The only difference between the query and range jobs should be the latter is limited based on the incoming first names.
|
124
|
+
|
125
|
+
|
126
|
+
## Contributing
|
127
|
+
|
128
|
+
### Development Environment Configuration
|
129
|
+
|
130
|
+
Basic steps to take to get this repository compiling:
|
131
|
+
|
132
|
+
1. Install [Ruby](https://www.ruby-lang.org/en/documentation/installation/) (check db_fuel.gemspec for versions supported)
|
133
|
+
2. Install bundler (gem install bundler)
|
134
|
+
3. Clone the repository (git clone git@github.com:bluemarblepayroll/db_fuel.git)
|
135
|
+
4. Navigate to the root folder (cd db_fuel)
|
136
|
+
5. Install dependencies (bundle)
|
137
|
+
|
138
|
+
### Running Tests
|
139
|
+
|
140
|
+
To execute the test suite run:
|
141
|
+
|
142
|
+
````bash
|
143
|
+
bundle exec rspec spec --format documentation
|
144
|
+
````
|
145
|
+
|
146
|
+
Alternatively, you can have Guard watch for changes:
|
147
|
+
|
148
|
+
````bash
|
149
|
+
bundle exec guard
|
150
|
+
````
|
151
|
+
|
152
|
+
Also, do not forget to run Rubocop:
|
153
|
+
|
154
|
+
````bash
|
155
|
+
bundle exec rubocop
|
156
|
+
````
|
157
|
+
|
158
|
+
### Publishing
|
159
|
+
|
160
|
+
Note: ensure you have proper authorization before trying to publish new versions.
|
161
|
+
|
162
|
+
After code changes have successfully gone through the Pull Request review process then the following steps should be followed for publishing new versions:
|
163
|
+
|
164
|
+
1. Merge Pull Request into master
|
165
|
+
2. Update `lib/db_fuel/version.rb` using [semantic versioning](https://semver.org/)
|
166
|
+
3. Install dependencies: `bundle`
|
167
|
+
4. Update `CHANGELOG.md` with release notes
|
168
|
+
5. Commit & push master to remote and ensure CI builds master successfully
|
169
|
+
6. Run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
170
|
+
|
171
|
+
## Code of Conduct
|
172
|
+
|
173
|
+
Everyone interacting in this codebase, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/bluemarblepayroll/db_fuel/blob/master/CODE_OF_CONDUCT.md).
|
174
|
+
|
175
|
+
## License
|
176
|
+
|
177
|
+
This project is MIT Licensed.
|
data/db_fuel.gemspec
CHANGED
@@ -5,10 +5,10 @@ require './lib/db_fuel/version'
|
|
5
5
|
Gem::Specification.new do |s|
|
6
6
|
s.name = 'db_fuel'
|
7
7
|
s.version = DbFuel::VERSION
|
8
|
-
s.summary = '
|
8
|
+
s.summary = 'Dbee and ActiveRecord jobs for Burner'
|
9
9
|
|
10
10
|
s.description = <<-DESCRIPTION
|
11
|
-
|
11
|
+
This library adds database-centric jobs to the Burner library. Burner does not ship with database jobs out of the box.
|
12
12
|
DESCRIPTION
|
13
13
|
|
14
14
|
s.authors = ['Matthew Ruggio']
|
@@ -16,6 +16,11 @@ module DbFuel
|
|
16
16
|
:provider,
|
17
17
|
:query
|
18
18
|
|
19
|
+
# Arguments:
|
20
|
+
# - model: Dbee Model configuration
|
21
|
+
# - query: Dbee Query configuration
|
22
|
+
# - register: Name of the register to use for gathering the IN clause values and where
|
23
|
+
# to store the resulting recordset.
|
19
24
|
def initialize(
|
20
25
|
name:,
|
21
26
|
model: {},
|
@@ -12,8 +12,8 @@ require_relative 'base'
|
|
12
12
|
module DbFuel
|
13
13
|
module Library
|
14
14
|
module Dbee
|
15
|
-
#
|
16
|
-
# payload register.
|
15
|
+
# Executes a Dbee Query against a Dbee Model and stores the resulting records
|
16
|
+
# in the designated payload register.
|
17
17
|
#
|
18
18
|
# Expected Payload[register] input: nothing
|
19
19
|
# Payload[register] output: array of objects.
|
@@ -23,6 +23,15 @@ module DbFuel
|
|
23
23
|
:key_path,
|
24
24
|
:resolver
|
25
25
|
|
26
|
+
# Arguments:
|
27
|
+
# - key: Specifies which key to use to aggregate a list of values for within
|
28
|
+
# the specified register's dataset.
|
29
|
+
# - key_path: Specifies the Dbee identifier (column) to use for the IN filter.
|
30
|
+
# - model: Dbee Model configuration
|
31
|
+
# - query: Dbee Query configuration
|
32
|
+
# - register: Name of the register to use for gathering the IN clause values and where
|
33
|
+
# to store the resulting recordset.
|
34
|
+
# - separator: Character to use to split the key-path for nested object support.
|
26
35
|
def initialize(
|
27
36
|
name:,
|
28
37
|
key:,
|
@@ -58,20 +67,24 @@ module DbFuel
|
|
58
67
|
array(payload[register]).map { |o| resolver.get(o, key) }.compact
|
59
68
|
end
|
60
69
|
|
61
|
-
def
|
70
|
+
def dynamic_filters(payload)
|
62
71
|
values = map_values(payload)
|
63
72
|
|
64
|
-
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
73
|
+
return [] if values.empty?
|
74
|
+
|
75
|
+
[
|
76
|
+
{
|
77
|
+
type: :equals,
|
78
|
+
key_path: key_path,
|
79
|
+
value: values,
|
80
|
+
}
|
81
|
+
]
|
69
82
|
end
|
70
83
|
|
71
84
|
def compile_dbee_query(payload)
|
72
85
|
::Dbee::Query.make(
|
73
86
|
fields: query.fields,
|
74
|
-
filters: query.filters +
|
87
|
+
filters: query.filters + dynamic_filters(payload),
|
75
88
|
limit: query.limit,
|
76
89
|
sorters: query.sorters
|
77
90
|
)
|
data/lib/db_fuel/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: db_fuel
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0.0
|
4
|
+
version: 1.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Matthew Ruggio
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-11-
|
11
|
+
date: 2020-11-18 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activerecord
|
@@ -212,7 +212,8 @@ dependencies:
|
|
212
212
|
- - "~>"
|
213
213
|
- !ruby/object:Gem::Version
|
214
214
|
version: '1'
|
215
|
-
description: "
|
215
|
+
description: " This library adds database-centric jobs to the Burner library. Burner
|
216
|
+
does not ship with database jobs out of the box.\n"
|
216
217
|
email:
|
217
218
|
- mruggio@bluemarblepayroll.com
|
218
219
|
executables: []
|
@@ -260,12 +261,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
260
261
|
version: '2.5'
|
261
262
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
262
263
|
requirements:
|
263
|
-
- - "
|
264
|
+
- - ">="
|
264
265
|
- !ruby/object:Gem::Version
|
265
|
-
version:
|
266
|
+
version: '0'
|
266
267
|
requirements: []
|
267
268
|
rubygems_version: 3.0.3
|
268
269
|
signing_key:
|
269
270
|
specification_version: 4
|
270
|
-
summary:
|
271
|
+
summary: Dbee and ActiveRecord jobs for Burner
|
271
272
|
test_files: []
|