DUI 0.9.0 → 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- data/.travis.yml +11 -0
- data/DUI.gemspec +1 -0
- data/README.md +88 -0
- data/Rakefile +9 -0
- data/lib/DUI/matcher.rb +4 -4
- data/lib/DUI/version.rb +1 -1
- data/spec/DUI/matcher_spec.rb +1 -0
- metadata +28 -12
data/.travis.yml
ADDED
data/DUI.gemspec
CHANGED
data/README.md
ADDED
@@ -0,0 +1,88 @@
|
|
1
|
+
#DUI [![Build Status](https://secure.travis-ci.org/darrencauthon/DUI.png?branch=master)](http://travis-ci.org/darrencauthon/DUI)
|
2
|
+
## Delete, Update, and Insert
|
3
|
+
|
4
|
+
The purpose of this gem is to make it easy to compare two datasets to see what operations would be necessary to merge the new data into the old.
|
5
|
+
|
6
|
+
Way back in my stored procedure days, we'd write this type of update by:
|
7
|
+
|
8
|
+
1.) Deleting any current records that aren't in the new set,
|
9
|
+
2.) Updating the current records with any matches in the new set, and
|
10
|
+
3.) Inserting any new records into the current set.
|
11
|
+
|
12
|
+
This gem helps to run this process by doing the comparison for you, giving you three sets of data that you can use to make the appropriate data changes.
|
13
|
+
|
14
|
+
**Warning:** All of the work is done in memory, so I would not run it against millions of records. But for small-to-medium sets of data, it might be of some help.
|
15
|
+
|
16
|
+
### My inspiration? ###
|
17
|
+
|
18
|
+
Two things:
|
19
|
+
|
20
|
+
1.) I love that Ruby's syntax makes it easy to write code that executes a *process*, regardless of the objects being used (and without going crazy with things like generics). We'd used to write these DUI sprocs by hand, different for each table. I guess object-oriented languages have some benefits...
|
21
|
+
|
22
|
+
2.) I've seen a number of imports that start with an call to **destroy_all**, so an import is essentially a DELETE EVERYTHING, THEN IMPORT EVERYTHING AS NEW process. Yikes. After having to fix a few issues where the system would fail *after* the data was deleted but *before* the import was finished, I had to take action.
|
23
|
+
|
24
|
+
Simple Example:
|
25
|
+
|
26
|
+
```ruby
|
27
|
+
require 'DUI'
|
28
|
+
require 'awesome_print'
|
29
|
+
|
30
|
+
class Person
|
31
|
+
attr_accessor :name, :email
|
32
|
+
def initialize(hash = {})
|
33
|
+
hash.each_pair { |k,v| self.send((k.to_s+"=").to_sym, v)}
|
34
|
+
end
|
35
|
+
end
|
36
|
+
|
37
|
+
current_data = [Person.new(:email => 'john@galt.com', :name => "JG"),
|
38
|
+
Person.new(:email => 'howard@roark.com', :name => "H Roark"),
|
39
|
+
Person.new(:email => 'peter@keating.com', :name => "Peter Keating")]
|
40
|
+
|
41
|
+
new_data = [{:email => "howard@roark.com", :name => "Howard Roark"},
|
42
|
+
{:email => "john@galt.com", :name => "John Galt"},
|
43
|
+
{:email => "hank@rearden.com", :name => "Hank Rearden"}]
|
44
|
+
|
45
|
+
a_way_to_match_the_records = lambda {|current, new| current.email == new[:email]}
|
46
|
+
|
47
|
+
# create a matcher, provide it with a way to compare the two records
|
48
|
+
matcher = DUI::Matcher.new(&a_way_to_match_the_records)
|
49
|
+
|
50
|
+
# get the results of the matches between the two sets
|
51
|
+
results = matcher.get_results current_data, new_data
|
52
|
+
|
53
|
+
# now we know which records to delete, update, or insert, so let's do it
|
54
|
+
|
55
|
+
results.records_to_delete.each do |delete_me|
|
56
|
+
current_data.delete delete_me # bye peter!
|
57
|
+
end
|
58
|
+
|
59
|
+
results.records_to_update.each do |match|
|
60
|
+
match.current.name = match.new[:name] #fix those names!
|
61
|
+
end
|
62
|
+
|
63
|
+
results.records_to_insert.each do |add_me|
|
64
|
+
new_person = Person.new :email => add_me[:email], :name => add_me[:name]
|
65
|
+
current_data << new_person # hello hank!
|
66
|
+
end
|
67
|
+
|
68
|
+
ap current_data
|
69
|
+
|
70
|
+
```
|
71
|
+
Here's the final result:
|
72
|
+
|
73
|
+
```ruby
|
74
|
+
[
|
75
|
+
[0] #<Person:0x7ffae4937920
|
76
|
+
attr_accessor :email = "john@galt.com",
|
77
|
+
attr_accessor :name = "John Galt"
|
78
|
+
>,
|
79
|
+
[1] #<Person:0x7ffae49376c8
|
80
|
+
attr_accessor :email = "howard@roark.com",
|
81
|
+
attr_accessor :name = "Howard Roark"
|
82
|
+
>,
|
83
|
+
[2] #<Person:0x7ffae4966bf8
|
84
|
+
attr_accessor :email = "hank@rearden.com",
|
85
|
+
attr_accessor :name = "Hank Rearden"
|
86
|
+
>
|
87
|
+
]
|
88
|
+
```
|
data/Rakefile
CHANGED
data/lib/DUI/matcher.rb
CHANGED
@@ -3,7 +3,7 @@ module DUI
|
|
3
3
|
class Matcher
|
4
4
|
|
5
5
|
def initialize(&compare_method)
|
6
|
-
@compare_method = compare_method || Proc.new {|c, n| c.id == n.id }
|
6
|
+
@compare_method = compare_method || Proc.new { |c, n| c.id == n.id }
|
7
7
|
end
|
8
8
|
|
9
9
|
def get_results(current_data, new_data)
|
@@ -28,8 +28,8 @@ module DUI
|
|
28
28
|
|
29
29
|
def all_current_data_with_possible_matches_in_new_data(current_data, new_data)
|
30
30
|
current_data.map do |c|
|
31
|
-
an_object_with({:current => c}) do |result|
|
32
|
-
result.new = new_data.select {|n| @compare_method.call(c, n) }.first
|
31
|
+
an_object_with( {:current => c} ) do |result|
|
32
|
+
result.new = new_data.select { |n| @compare_method.call(c, n) }.first
|
33
33
|
result.current_not_found_in_new = result.new.nil?
|
34
34
|
end
|
35
35
|
end
|
@@ -37,7 +37,7 @@ module DUI
|
|
37
37
|
|
38
38
|
def get_records_to_insert(results, new_data)
|
39
39
|
current_records = results.records_to_update.map{|u| u.current}
|
40
|
-
new_data.select {|n| current_records.select {|c| @compare_method.call(c, n) }.count == 0 }
|
40
|
+
new_data.select { |n| current_records.select { |c| @compare_method.call(c, n) }.count == 0 }
|
41
41
|
end
|
42
42
|
|
43
43
|
def an_object_with(hash)
|
data/lib/DUI/version.rb
CHANGED
data/spec/DUI/matcher_spec.rb
CHANGED
metadata
CHANGED
@@ -1,13 +1,13 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: DUI
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
hash:
|
4
|
+
hash: 23
|
5
5
|
prerelease:
|
6
6
|
segments:
|
7
|
+
- 1
|
7
8
|
- 0
|
8
|
-
- 9
|
9
9
|
- 0
|
10
|
-
version: 0.
|
10
|
+
version: 1.0.0
|
11
11
|
platform: ruby
|
12
12
|
authors:
|
13
13
|
- Darren Cauthon
|
@@ -15,12 +15,10 @@ autorequire:
|
|
15
15
|
bindir: bin
|
16
16
|
cert_chain: []
|
17
17
|
|
18
|
-
date: 2012-
|
18
|
+
date: 2012-08-17 00:00:00 Z
|
19
19
|
dependencies:
|
20
20
|
- !ruby/object:Gem::Dependency
|
21
|
-
|
22
|
-
prerelease: false
|
23
|
-
requirement: &id001 !ruby/object:Gem::Requirement
|
21
|
+
version_requirements: &id001 !ruby/object:Gem::Requirement
|
24
22
|
none: false
|
25
23
|
requirements:
|
26
24
|
- - ">="
|
@@ -29,12 +27,12 @@ dependencies:
|
|
29
27
|
segments:
|
30
28
|
- 0
|
31
29
|
version: "0"
|
30
|
+
requirement: *id001
|
32
31
|
type: :development
|
33
|
-
version_requirements: *id001
|
34
|
-
- !ruby/object:Gem::Dependency
|
35
|
-
name: hashie
|
36
32
|
prerelease: false
|
37
|
-
|
33
|
+
name: minitest
|
34
|
+
- !ruby/object:Gem::Dependency
|
35
|
+
version_requirements: &id002 !ruby/object:Gem::Requirement
|
38
36
|
none: false
|
39
37
|
requirements:
|
40
38
|
- - ">="
|
@@ -43,8 +41,24 @@ dependencies:
|
|
43
41
|
segments:
|
44
42
|
- 0
|
45
43
|
version: "0"
|
44
|
+
requirement: *id002
|
46
45
|
type: :runtime
|
47
|
-
|
46
|
+
prerelease: false
|
47
|
+
name: hashie
|
48
|
+
- !ruby/object:Gem::Dependency
|
49
|
+
version_requirements: &id003 !ruby/object:Gem::Requirement
|
50
|
+
none: false
|
51
|
+
requirements:
|
52
|
+
- - ">="
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
hash: 3
|
55
|
+
segments:
|
56
|
+
- 0
|
57
|
+
version: "0"
|
58
|
+
requirement: *id003
|
59
|
+
type: :development
|
60
|
+
prerelease: false
|
61
|
+
name: rake
|
48
62
|
description: |-
|
49
63
|
This gem provides a small API for comparing two datasets,
|
50
64
|
for determining what records should be deleted, updated, or inserted.
|
@@ -59,9 +73,11 @@ extra_rdoc_files: []
|
|
59
73
|
files:
|
60
74
|
- .gitignore
|
61
75
|
- .rvmrc
|
76
|
+
- .travis.yml
|
62
77
|
- DUI.gemspec
|
63
78
|
- Gemfile
|
64
79
|
- Guardfile
|
80
|
+
- README.md
|
65
81
|
- Rakefile
|
66
82
|
- lib/DUI.rb
|
67
83
|
- lib/DUI/matcher.rb
|