DUI 0.9.0 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.travis.yml +11 -0
- data/DUI.gemspec +1 -0
- data/README.md +88 -0
- data/Rakefile +9 -0
- data/lib/DUI/matcher.rb +4 -4
- data/lib/DUI/version.rb +1 -1
- data/spec/DUI/matcher_spec.rb +1 -0
- metadata +28 -12
data/.travis.yml
ADDED
data/DUI.gemspec
CHANGED
data/README.md
ADDED
@@ -0,0 +1,88 @@
|
|
1
|
+
#DUI [](http://travis-ci.org/darrencauthon/DUI)
|
2
|
+
## Delete, Update, and Insert
|
3
|
+
|
4
|
+
The purpose of this gem is to make it easy to compare two datasets to see what operations would be necessary to merge the new data into the old.
|
5
|
+
|
6
|
+
Way back in my stored procedure days, we'd write this type of update by:
|
7
|
+
|
8
|
+
1.) Deleting any current records that aren't in the new set,
|
9
|
+
2.) Updating the current records with any matches in the new set, and
|
10
|
+
3.) Inserting any new records into the current set.
|
11
|
+
|
12
|
+
This gem helps to run this process by doing the comparison for you, giving you three sets of data that you can use to make the appropriate data changes.
|
13
|
+
|
14
|
+
**Warning:** All of the work is done in memory, so I would not run it against millions of records. But for small-to-medium sets of data, it might be of some help.
|
15
|
+
|
16
|
+
### My inspiration? ###
|
17
|
+
|
18
|
+
Two things:
|
19
|
+
|
20
|
+
1.) I love that Ruby's syntax makes it easy to write code that executes a *process*, regardless of the objects being used (and without going crazy with things like generics). We'd used to write these DUI sprocs by hand, different for each table. I guess object-oriented languages have some benefits...
|
21
|
+
|
22
|
+
2.) I've seen a number of imports that start with an call to **destroy_all**, so an import is essentially a DELETE EVERYTHING, THEN IMPORT EVERYTHING AS NEW process. Yikes. After having to fix a few issues where the system would fail *after* the data was deleted but *before* the import was finished, I had to take action.
|
23
|
+
|
24
|
+
Simple Example:
|
25
|
+
|
26
|
+
```ruby
|
27
|
+
require 'DUI'
|
28
|
+
require 'awesome_print'
|
29
|
+
|
30
|
+
class Person
|
31
|
+
attr_accessor :name, :email
|
32
|
+
def initialize(hash = {})
|
33
|
+
hash.each_pair { |k,v| self.send((k.to_s+"=").to_sym, v)}
|
34
|
+
end
|
35
|
+
end
|
36
|
+
|
37
|
+
current_data = [Person.new(:email => 'john@galt.com', :name => "JG"),
|
38
|
+
Person.new(:email => 'howard@roark.com', :name => "H Roark"),
|
39
|
+
Person.new(:email => 'peter@keating.com', :name => "Peter Keating")]
|
40
|
+
|
41
|
+
new_data = [{:email => "howard@roark.com", :name => "Howard Roark"},
|
42
|
+
{:email => "john@galt.com", :name => "John Galt"},
|
43
|
+
{:email => "hank@rearden.com", :name => "Hank Rearden"}]
|
44
|
+
|
45
|
+
a_way_to_match_the_records = lambda {|current, new| current.email == new[:email]}
|
46
|
+
|
47
|
+
# create a matcher, provide it with a way to compare the two records
|
48
|
+
matcher = DUI::Matcher.new(&a_way_to_match_the_records)
|
49
|
+
|
50
|
+
# get the results of the matches between the two sets
|
51
|
+
results = matcher.get_results current_data, new_data
|
52
|
+
|
53
|
+
# now we know which records to delete, update, or insert, so let's do it
|
54
|
+
|
55
|
+
results.records_to_delete.each do |delete_me|
|
56
|
+
current_data.delete delete_me # bye peter!
|
57
|
+
end
|
58
|
+
|
59
|
+
results.records_to_update.each do |match|
|
60
|
+
match.current.name = match.new[:name] #fix those names!
|
61
|
+
end
|
62
|
+
|
63
|
+
results.records_to_insert.each do |add_me|
|
64
|
+
new_person = Person.new :email => add_me[:email], :name => add_me[:name]
|
65
|
+
current_data << new_person # hello hank!
|
66
|
+
end
|
67
|
+
|
68
|
+
ap current_data
|
69
|
+
|
70
|
+
```
|
71
|
+
Here's the final result:
|
72
|
+
|
73
|
+
```ruby
|
74
|
+
[
|
75
|
+
[0] #<Person:0x7ffae4937920
|
76
|
+
attr_accessor :email = "john@galt.com",
|
77
|
+
attr_accessor :name = "John Galt"
|
78
|
+
>,
|
79
|
+
[1] #<Person:0x7ffae49376c8
|
80
|
+
attr_accessor :email = "howard@roark.com",
|
81
|
+
attr_accessor :name = "Howard Roark"
|
82
|
+
>,
|
83
|
+
[2] #<Person:0x7ffae4966bf8
|
84
|
+
attr_accessor :email = "hank@rearden.com",
|
85
|
+
attr_accessor :name = "Hank Rearden"
|
86
|
+
>
|
87
|
+
]
|
88
|
+
```
|
data/Rakefile
CHANGED
data/lib/DUI/matcher.rb
CHANGED
@@ -3,7 +3,7 @@ module DUI
|
|
3
3
|
class Matcher
|
4
4
|
|
5
5
|
def initialize(&compare_method)
|
6
|
-
@compare_method = compare_method || Proc.new {|c, n| c.id == n.id }
|
6
|
+
@compare_method = compare_method || Proc.new { |c, n| c.id == n.id }
|
7
7
|
end
|
8
8
|
|
9
9
|
def get_results(current_data, new_data)
|
@@ -28,8 +28,8 @@ module DUI
|
|
28
28
|
|
29
29
|
def all_current_data_with_possible_matches_in_new_data(current_data, new_data)
|
30
30
|
current_data.map do |c|
|
31
|
-
an_object_with({:current => c}) do |result|
|
32
|
-
result.new = new_data.select {|n| @compare_method.call(c, n) }.first
|
31
|
+
an_object_with( {:current => c} ) do |result|
|
32
|
+
result.new = new_data.select { |n| @compare_method.call(c, n) }.first
|
33
33
|
result.current_not_found_in_new = result.new.nil?
|
34
34
|
end
|
35
35
|
end
|
@@ -37,7 +37,7 @@ module DUI
|
|
37
37
|
|
38
38
|
def get_records_to_insert(results, new_data)
|
39
39
|
current_records = results.records_to_update.map{|u| u.current}
|
40
|
-
new_data.select {|n| current_records.select {|c| @compare_method.call(c, n) }.count == 0 }
|
40
|
+
new_data.select { |n| current_records.select { |c| @compare_method.call(c, n) }.count == 0 }
|
41
41
|
end
|
42
42
|
|
43
43
|
def an_object_with(hash)
|
data/lib/DUI/version.rb
CHANGED
data/spec/DUI/matcher_spec.rb
CHANGED
metadata
CHANGED
@@ -1,13 +1,13 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: DUI
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
hash:
|
4
|
+
hash: 23
|
5
5
|
prerelease:
|
6
6
|
segments:
|
7
|
+
- 1
|
7
8
|
- 0
|
8
|
-
- 9
|
9
9
|
- 0
|
10
|
-
version: 0.
|
10
|
+
version: 1.0.0
|
11
11
|
platform: ruby
|
12
12
|
authors:
|
13
13
|
- Darren Cauthon
|
@@ -15,12 +15,10 @@ autorequire:
|
|
15
15
|
bindir: bin
|
16
16
|
cert_chain: []
|
17
17
|
|
18
|
-
date: 2012-
|
18
|
+
date: 2012-08-17 00:00:00 Z
|
19
19
|
dependencies:
|
20
20
|
- !ruby/object:Gem::Dependency
|
21
|
-
|
22
|
-
prerelease: false
|
23
|
-
requirement: &id001 !ruby/object:Gem::Requirement
|
21
|
+
version_requirements: &id001 !ruby/object:Gem::Requirement
|
24
22
|
none: false
|
25
23
|
requirements:
|
26
24
|
- - ">="
|
@@ -29,12 +27,12 @@ dependencies:
|
|
29
27
|
segments:
|
30
28
|
- 0
|
31
29
|
version: "0"
|
30
|
+
requirement: *id001
|
32
31
|
type: :development
|
33
|
-
version_requirements: *id001
|
34
|
-
- !ruby/object:Gem::Dependency
|
35
|
-
name: hashie
|
36
32
|
prerelease: false
|
37
|
-
|
33
|
+
name: minitest
|
34
|
+
- !ruby/object:Gem::Dependency
|
35
|
+
version_requirements: &id002 !ruby/object:Gem::Requirement
|
38
36
|
none: false
|
39
37
|
requirements:
|
40
38
|
- - ">="
|
@@ -43,8 +41,24 @@ dependencies:
|
|
43
41
|
segments:
|
44
42
|
- 0
|
45
43
|
version: "0"
|
44
|
+
requirement: *id002
|
46
45
|
type: :runtime
|
47
|
-
|
46
|
+
prerelease: false
|
47
|
+
name: hashie
|
48
|
+
- !ruby/object:Gem::Dependency
|
49
|
+
version_requirements: &id003 !ruby/object:Gem::Requirement
|
50
|
+
none: false
|
51
|
+
requirements:
|
52
|
+
- - ">="
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
hash: 3
|
55
|
+
segments:
|
56
|
+
- 0
|
57
|
+
version: "0"
|
58
|
+
requirement: *id003
|
59
|
+
type: :development
|
60
|
+
prerelease: false
|
61
|
+
name: rake
|
48
62
|
description: |-
|
49
63
|
This gem provides a small API for comparing two datasets,
|
50
64
|
for determining what records should be deleted, updated, or inserted.
|
@@ -59,9 +73,11 @@ extra_rdoc_files: []
|
|
59
73
|
files:
|
60
74
|
- .gitignore
|
61
75
|
- .rvmrc
|
76
|
+
- .travis.yml
|
62
77
|
- DUI.gemspec
|
63
78
|
- Gemfile
|
64
79
|
- Guardfile
|
80
|
+
- README.md
|
65
81
|
- Rakefile
|
66
82
|
- lib/DUI.rb
|
67
83
|
- lib/DUI/matcher.rb
|