match_reduce 1.0.0.pre.alpha → 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +4 -0
- data/README.md +13 -13
- data/lib/match_reduce.rb +6 -10
- data/lib/match_reduce/{aggregate.rb → aggregator.rb} +4 -4
- data/lib/match_reduce/index.rb +13 -13
- data/lib/match_reduce/processor.rb +16 -16
- data/lib/match_reduce/processor/result.rb +1 -1
- data/lib/match_reduce/processor/result_builder.rb +10 -10
- data/lib/match_reduce/processor/results_builder.rb +6 -6
- data/lib/match_reduce/version.rb +1 -1
- data/match_reduce.gemspec +1 -1
- data/spec/fixtures/snapshots/processor/abstract.yaml +1 -1
- data/spec/fixtures/snapshots/processor/teams_and_players.yaml +1 -1
- data/spec/match_reduce/{aggregate_spec.rb → aggregator_spec.rb} +1 -1
- data/spec/match_reduce/index_spec.rb +16 -16
- data/spec/match_reduce/processor_spec.rb +6 -8
- data/spec/match_reduce_spec.rb +12 -9
- metadata +11 -14
- data/lib/match_reduce/any.rb +0 -30
- data/spec/match_reduce/any_spec.rb +0 -32
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c0ca62d59ead19578324d9c7e81ad9f671d6d92b467b5ffe8b110e1b60d0e127
|
4
|
+
data.tar.gz: 63d0e69d3e187763f8fc8c4f2413c2bc00f6a5e61eddfd30e744473f6f04f2a4
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 9eb86c78525d6de4bd4abc06c47c8d27fab79e88488cdde1c3e36c8fa4b06182aee657433cb003a2d7f9986d1422d69e1d61c8777a1be453c2d51efc2ba3d7b9
|
7
|
+
data.tar.gz: 0ba05d835f6bd6403a9a1c4d9d360a4904ea6eca5a92e732cfd207cf8a35c93ab64e4869189c18fd638c586b6f2c8f4d5c7bf357b818669ca8db490c5f9a2c18
|
data/CHANGELOG.md
CHANGED
data/README.md
CHANGED
@@ -4,11 +4,11 @@
|
|
4
4
|
|
5
5
|
MatchReduce is a high-speed, in-memory data aggregation and reduction algorithm. The lifecycle is:
|
6
6
|
|
7
|
-
1. define
|
7
|
+
1. define aggregators
|
8
8
|
2. pump records into algorithm
|
9
9
|
3. grab results
|
10
10
|
|
11
|
-
The dataset will only be processed once no matter how many
|
11
|
+
The dataset will only be processed once no matter how many aggregators you define. An aggregator is expressed as:
|
12
12
|
|
13
13
|
* patterns: an array of hashes, which are used to pattern match records
|
14
14
|
* reducer: a function, which is used when a record matches
|
@@ -37,7 +37,7 @@ A very basic example of calling this library would be:
|
|
37
37
|
````ruby
|
38
38
|
require 'match_reduce'
|
39
39
|
|
40
|
-
|
40
|
+
aggregators = [
|
41
41
|
{
|
42
42
|
name: :total_game_points
|
43
43
|
reducer: ->(memo, record, resolver) { memo.to_i + resolver.get(record, :game_points).to_i },
|
@@ -52,7 +52,7 @@ records = [
|
|
52
52
|
{ game: 2, game_points: 240, team: 'Bulls', team_points: 110 }
|
53
53
|
]
|
54
54
|
|
55
|
-
results = MatchReduce.process(
|
55
|
+
results = MatchReduce.process(aggregators, records)
|
56
56
|
````
|
57
57
|
|
58
58
|
`results` would be equal to:
|
@@ -72,10 +72,10 @@ results = MatchReduce.process(aggregates, records)
|
|
72
72
|
|
73
73
|
Notes:
|
74
74
|
|
75
|
-
* Not specifying patterns means: "match on everything"
|
75
|
+
* Not specifying patterns, as in the example above, means: "match on everything"
|
76
76
|
* group_keys will limit the records matched on, per aggregator, to the first record only. This is why only the first and third records matched.
|
77
|
-
* keys are type-indifferent and will extracted using (the Objectable library)[https://github.com/bluemarblepayroll/objectable]. This means you can leverage dot-notation, non-hash record types, and indifference. You can also customize the resolver used and pass it as a third argument to MatchReduce#process(
|
78
|
-
* Names of
|
77
|
+
* keys are type-indifferent and will extracted using (the Objectable library)[https://github.com/bluemarblepayroll/objectable]. This means you can leverage dot-notation, non-hash record types, and indifference. You can also customize the resolver used and pass it as a third argument to MatchReduce#process(aggregators, records, resolver).
|
78
|
+
* Names of aggregators are type-sensitive, so: `:total_game_points` and `'total_game_points'` are two different aggregators and will produce two different results.
|
79
79
|
|
80
80
|
### Adding Patterns
|
81
81
|
|
@@ -84,7 +84,7 @@ Let's say we want to discretely know how many points the Bulls, Celtics, and Roc
|
|
84
84
|
````ruby
|
85
85
|
require 'match_reduce'
|
86
86
|
|
87
|
-
|
87
|
+
aggregators = [
|
88
88
|
{
|
89
89
|
name: :bulls_points
|
90
90
|
patterns: { team: 'Bulls' },
|
@@ -112,7 +112,7 @@ records = [
|
|
112
112
|
{ game: 2, game_points: 240, team: 'Bulls', team_points: 110 }
|
113
113
|
]
|
114
114
|
|
115
|
-
results = MatchReduce.process(
|
115
|
+
results = MatchReduce.process(aggregators, records)
|
116
116
|
````
|
117
117
|
|
118
118
|
`results` would now be equal to:
|
@@ -144,12 +144,12 @@ results = MatchReduce.process(aggregates, records)
|
|
144
144
|
]
|
145
145
|
````
|
146
146
|
|
147
|
-
We could also choose to
|
147
|
+
We could also choose to aggregator multiple teams together by providing multiple patterns:
|
148
148
|
|
149
149
|
````ruby
|
150
150
|
require 'match_reduce'
|
151
151
|
|
152
|
-
|
152
|
+
aggregators = [
|
153
153
|
{
|
154
154
|
name: :bulls_and_celtics_points
|
155
155
|
patterns: [
|
@@ -168,7 +168,7 @@ records = [
|
|
168
168
|
{ game: 2, game_points: 240, team: 'Bulls', team_points: 110 }
|
169
169
|
]
|
170
170
|
|
171
|
-
results = MatchReduce.process(
|
171
|
+
results = MatchReduce.process(aggregators, records)
|
172
172
|
````
|
173
173
|
|
174
174
|
`results` would now be equal to:
|
@@ -176,7 +176,7 @@ results = MatchReduce.process(aggregates, records)
|
|
176
176
|
````ruby
|
177
177
|
[
|
178
178
|
MatchReduce::Processor::Result.new(
|
179
|
-
name: :
|
179
|
+
name: :bulls_and_celtics_points,
|
180
180
|
records: [
|
181
181
|
{ game: 1, game_points: 199, team: 'Bulls', team_points: 100 },
|
182
182
|
{ game: 1, game_points: 199, team: 'Celtics', team_points: 99 },
|
data/lib/match_reduce.rb
CHANGED
@@ -11,23 +11,19 @@ require 'acts_as_hashable'
|
|
11
11
|
require 'hash_math'
|
12
12
|
require 'objectable'
|
13
13
|
|
14
|
-
require_relative 'match_reduce/
|
15
|
-
require_relative 'match_reduce/aggregate'
|
14
|
+
require_relative 'match_reduce/aggregator'
|
16
15
|
require_relative 'match_reduce/index'
|
17
16
|
require_relative 'match_reduce/processor'
|
18
17
|
|
19
18
|
# Top-level namespace
|
20
19
|
module MatchReduce
|
21
|
-
#
|
22
|
-
#
|
23
|
-
|
24
|
-
# special flag indicating: "match on any value". So even if we were to instantiate
|
25
|
-
# multiple Any objects, the point is moot.
|
26
|
-
ANY = Any.new
|
20
|
+
# Something unique which will represent "match on all values". This is used as the base
|
21
|
+
# value for all pattern keys.
|
22
|
+
ANY = :__ANY__
|
27
23
|
|
28
24
|
class << self
|
29
|
-
def process(
|
30
|
-
Processor.new(
|
25
|
+
def process(aggregators, records, resolver: Objectable.resolver, any: ANY)
|
26
|
+
Processor.new(aggregators, resolver: resolver, any: any)
|
31
27
|
.add_each(records)
|
32
28
|
.results
|
33
29
|
end
|
@@ -8,8 +8,8 @@
|
|
8
8
|
#
|
9
9
|
|
10
10
|
module MatchReduce
|
11
|
-
# An
|
12
|
-
class
|
11
|
+
# An aggregator is a group of patterns with a reducer that you wish to report on.
|
12
|
+
class Aggregator
|
13
13
|
acts_as_hashable
|
14
14
|
|
15
15
|
attr_reader :group_keys,
|
@@ -22,7 +22,7 @@ module MatchReduce
|
|
22
22
|
|
23
23
|
@name = name
|
24
24
|
@group_keys = Array(group_keys)
|
25
|
-
@patterns = stringed_keys(
|
25
|
+
@patterns = stringed_keys(ensure_not_empty(array(patterns)))
|
26
26
|
@reducer = reducer
|
27
27
|
|
28
28
|
freeze
|
@@ -54,7 +54,7 @@ module MatchReduce
|
|
54
54
|
val.is_a?(Hash) ? [val] : Array(val)
|
55
55
|
end
|
56
56
|
|
57
|
-
def
|
57
|
+
def ensure_not_empty(val)
|
58
58
|
val.empty? ? [{}] : val
|
59
59
|
end
|
60
60
|
end
|
data/lib/match_reduce/index.rb
CHANGED
@@ -8,30 +8,30 @@
|
|
8
8
|
#
|
9
9
|
|
10
10
|
module MatchReduce
|
11
|
-
# The Index holds all the
|
12
|
-
# to retrieve
|
11
|
+
# The Index holds all the aggregators, the reverse lookup data structure, and the ability
|
12
|
+
# to retrieve aggregators based on a pattern
|
13
13
|
class Index
|
14
14
|
extend Forwardable
|
15
15
|
|
16
|
-
attr_reader :
|
17
|
-
:
|
16
|
+
attr_reader :aggregators,
|
17
|
+
:any,
|
18
18
|
:lookup
|
19
19
|
|
20
20
|
def_delegators :record, :keys
|
21
21
|
|
22
|
-
def initialize(
|
23
|
-
@
|
24
|
-
@
|
25
|
-
@lookup
|
22
|
+
def initialize(aggregators = [], any: ANY)
|
23
|
+
@any = any
|
24
|
+
@aggregators = Aggregator.array(aggregators).uniq(&:name)
|
25
|
+
@lookup = {}
|
26
26
|
|
27
|
-
all_keys = @
|
28
|
-
@record = HashMath::Record.new(all_keys,
|
27
|
+
all_keys = @aggregators.flat_map(&:keys)
|
28
|
+
@record = HashMath::Record.new(all_keys, any)
|
29
29
|
|
30
|
-
@
|
31
|
-
|
30
|
+
@aggregators.map do |aggregator|
|
31
|
+
aggregator.patterns.each do |pattern|
|
32
32
|
normalized_pattern = record.make!(pattern)
|
33
33
|
|
34
|
-
get(normalized_pattern) <<
|
34
|
+
get(normalized_pattern) << aggregator
|
35
35
|
end
|
36
36
|
end
|
37
37
|
|
@@ -11,18 +11,18 @@ require_relative 'processor/results_builder'
|
|
11
11
|
|
12
12
|
module MatchReduce
|
13
13
|
# This is the main lifecycle of the algorithm. You initialize a new instance of this
|
14
|
-
# class using
|
14
|
+
# class using aggregators, then you pump in records into it. Once done, call #results
|
15
15
|
# to get the results.
|
16
16
|
class Processor
|
17
17
|
extend Forwardable
|
18
18
|
|
19
19
|
def_delegators :results_builder, :results, :resolver
|
20
20
|
|
21
|
-
def_delegators :index, :
|
21
|
+
def_delegators :index, :aggregators, :any
|
22
22
|
|
23
|
-
def initialize(
|
24
|
-
@index = Index.new(
|
25
|
-
@results_builder = ResultsBuilder.new(index.
|
23
|
+
def initialize(aggregators, resolver: Objectable.resolver, any: ANY)
|
24
|
+
@index = Index.new(aggregators, any: any)
|
25
|
+
@results_builder = ResultsBuilder.new(index.aggregators, resolver)
|
26
26
|
|
27
27
|
freeze
|
28
28
|
end
|
@@ -32,16 +32,16 @@ module MatchReduce
|
|
32
32
|
end
|
33
33
|
|
34
34
|
def add(record)
|
35
|
-
|
35
|
+
hit_aggregators = Set.new
|
36
36
|
|
37
37
|
record_patterns(record).each do |hash_pattern|
|
38
|
-
# Each index find hit means the
|
39
|
-
index.find(hash_pattern).each do |
|
40
|
-
next if
|
38
|
+
# Each index find hit means the aggregator matched on the record
|
39
|
+
index.find(hash_pattern).each do |aggregator|
|
40
|
+
next if hit_aggregators.include?(aggregator)
|
41
41
|
|
42
|
-
add_to_results_builder(
|
42
|
+
add_to_results_builder(aggregator, record)
|
43
43
|
|
44
|
-
|
44
|
+
hit_aggregators << aggregator
|
45
45
|
end
|
46
46
|
end
|
47
47
|
|
@@ -53,8 +53,8 @@ module MatchReduce
|
|
53
53
|
attr_reader :index,
|
54
54
|
:results_builder
|
55
55
|
|
56
|
-
def make_group_id(
|
57
|
-
|
56
|
+
def make_group_id(aggregator, record)
|
57
|
+
aggregator.group_keys.map { |group_key| resolver.get(record, group_key) }
|
58
58
|
end
|
59
59
|
|
60
60
|
def record_matrix(record)
|
@@ -69,10 +69,10 @@ module MatchReduce
|
|
69
69
|
[{}] + record_matrix(record).to_a
|
70
70
|
end
|
71
71
|
|
72
|
-
def add_to_results_builder(
|
73
|
-
group_id = make_group_id(
|
72
|
+
def add_to_results_builder(aggregator, record)
|
73
|
+
group_id = make_group_id(aggregator, record)
|
74
74
|
|
75
|
-
results_builder.add(
|
75
|
+
results_builder.add(aggregator, record, group_id)
|
76
76
|
end
|
77
77
|
end
|
78
78
|
end
|
@@ -11,14 +11,14 @@ require_relative 'result'
|
|
11
11
|
|
12
12
|
module MatchReduce
|
13
13
|
class Processor
|
14
|
-
# This class understands how to take an
|
14
|
+
# This class understands how to take an aggregator and derive a result for it.
|
15
15
|
class ResultBuilder
|
16
|
-
def initialize(
|
17
|
-
raise ArgumentError, '
|
18
|
-
raise ArgumentError, 'resolver is required'
|
16
|
+
def initialize(aggregator, resolver)
|
17
|
+
raise ArgumentError, 'aggregator is required' unless aggregator
|
18
|
+
raise ArgumentError, 'resolver is required' unless resolver
|
19
19
|
|
20
|
-
@
|
21
|
-
@resolver
|
20
|
+
@aggregator = aggregator
|
21
|
+
@resolver = resolver
|
22
22
|
|
23
23
|
@records = []
|
24
24
|
@value = nil
|
@@ -26,7 +26,7 @@ module MatchReduce
|
|
26
26
|
end
|
27
27
|
|
28
28
|
def add(record, group_id)
|
29
|
-
if
|
29
|
+
if aggregator.grouped?
|
30
30
|
return self if group_ids.include?(group_id)
|
31
31
|
|
32
32
|
group_ids << group_id
|
@@ -34,18 +34,18 @@ module MatchReduce
|
|
34
34
|
|
35
35
|
records << record
|
36
36
|
|
37
|
-
@value =
|
37
|
+
@value = aggregator.reduce(value, record, resolver)
|
38
38
|
|
39
39
|
self
|
40
40
|
end
|
41
41
|
|
42
42
|
def result
|
43
|
-
Result.new(
|
43
|
+
Result.new(aggregator.name, records, value)
|
44
44
|
end
|
45
45
|
|
46
46
|
private
|
47
47
|
|
48
|
-
attr_reader :
|
48
|
+
attr_reader :aggregator,
|
49
49
|
:group_ids,
|
50
50
|
:records,
|
51
51
|
:resolver,
|
@@ -11,21 +11,21 @@ require_relative 'result_builder'
|
|
11
11
|
|
12
12
|
module MatchReduce
|
13
13
|
class Processor
|
14
|
-
# This class knows how to group together
|
14
|
+
# This class knows how to group together aggregators in order to produce results.
|
15
15
|
class ResultsBuilder
|
16
16
|
attr_reader :resolver
|
17
17
|
|
18
|
-
def initialize(
|
19
|
-
raise ArgumentError, '
|
18
|
+
def initialize(aggregators, resolver)
|
19
|
+
raise ArgumentError, 'aggregators are required' unless aggregators
|
20
20
|
|
21
|
-
@result_by_name =
|
21
|
+
@result_by_name = aggregators.map { |a| [a.name, ResultBuilder.new(a, resolver)] }.to_h
|
22
22
|
@resolver = resolver
|
23
23
|
|
24
24
|
freeze
|
25
25
|
end
|
26
26
|
|
27
|
-
def add(
|
28
|
-
tap { result_by_name[
|
27
|
+
def add(aggregator, record, group_id)
|
28
|
+
tap { result_by_name[aggregator.name].add(record, group_id) }
|
29
29
|
end
|
30
30
|
|
31
31
|
def results
|
data/lib/match_reduce/version.rb
CHANGED
data/match_reduce.gemspec
CHANGED
@@ -22,7 +22,7 @@ Gem::Specification.new do |s|
|
|
22
22
|
s.required_ruby_version = '>= 2.3.8'
|
23
23
|
|
24
24
|
s.add_dependency('acts_as_hashable', '~>1', '>=1.1.0')
|
25
|
-
s.add_dependency('hash_math', '
|
25
|
+
s.add_dependency('hash_math', '~>1')
|
26
26
|
s.add_dependency('objectable', '~>1')
|
27
27
|
|
28
28
|
s.add_development_dependency('guard-rspec', '~>4.7')
|
@@ -11,23 +11,23 @@ require 'spec_helper'
|
|
11
11
|
|
12
12
|
describe MatchReduce::Index do
|
13
13
|
def lookup_by_name(lookup)
|
14
|
-
lookup.each_with_object({}) do |(pattern,
|
15
|
-
memo[pattern] =
|
14
|
+
lookup.each_with_object({}) do |(pattern, aggregators), memo|
|
15
|
+
memo[pattern] = aggregators.map(&:name)
|
16
16
|
end
|
17
17
|
end
|
18
18
|
|
19
19
|
let(:base_value) { MatchReduce::ANY }
|
20
20
|
|
21
21
|
describe '#initialization' do
|
22
|
-
context 'constructing
|
23
|
-
specify 'when all
|
22
|
+
context 'constructing aggregators' do
|
23
|
+
specify 'when all aggregators when empty' do
|
24
24
|
subject = described_class.new
|
25
25
|
|
26
|
-
expect(subject.
|
26
|
+
expect(subject.aggregators).to eq([])
|
27
27
|
end
|
28
28
|
|
29
|
-
specify 'only each first unique
|
30
|
-
|
29
|
+
specify 'only each first unique aggregator name is kept' do
|
30
|
+
aggregators = [
|
31
31
|
{ name: :sig3 },
|
32
32
|
{ name: :sig1 },
|
33
33
|
{ name: 'sig2' },
|
@@ -39,14 +39,14 @@ describe MatchReduce::Index do
|
|
39
39
|
{ name: :sig5 }
|
40
40
|
]
|
41
41
|
|
42
|
-
subject = described_class.new(
|
42
|
+
subject = described_class.new(aggregators)
|
43
43
|
|
44
|
-
expect(subject.
|
44
|
+
expect(subject.aggregators.length).to eq(7)
|
45
45
|
end
|
46
46
|
end
|
47
47
|
|
48
48
|
context 'constructing lookup' do
|
49
|
-
it 'creates lookup with
|
49
|
+
it 'creates lookup with aggregators' do
|
50
50
|
subject = described_class.new
|
51
51
|
|
52
52
|
expected = {}
|
@@ -54,13 +54,13 @@ describe MatchReduce::Index do
|
|
54
54
|
expect(subject.lookup).to eq(expected)
|
55
55
|
end
|
56
56
|
|
57
|
-
it 'creates lookup with
|
58
|
-
|
57
|
+
it 'creates lookup with aggregators that have no patterns' do
|
58
|
+
aggregators = [
|
59
59
|
{ name: :sig3 },
|
60
60
|
{ name: :sig1 }
|
61
61
|
]
|
62
62
|
|
63
|
-
subject = described_class.new(
|
63
|
+
subject = described_class.new(aggregators)
|
64
64
|
|
65
65
|
expected = {
|
66
66
|
{} => %i[sig3 sig1]
|
@@ -69,8 +69,8 @@ describe MatchReduce::Index do
|
|
69
69
|
expect(lookup_by_name(subject.lookup)).to eq(expected)
|
70
70
|
end
|
71
71
|
|
72
|
-
it 'creates lookup with
|
73
|
-
|
72
|
+
it 'creates lookup with aggregators that have patterns and no patterns' do
|
73
|
+
aggregators = [
|
74
74
|
{ name: :sig3 },
|
75
75
|
{ name: :sig1 },
|
76
76
|
{
|
@@ -79,7 +79,7 @@ describe MatchReduce::Index do
|
|
79
79
|
}
|
80
80
|
]
|
81
81
|
|
82
|
-
subject = described_class.new(
|
82
|
+
subject = described_class.new(aggregators)
|
83
83
|
|
84
84
|
expected = {
|
85
85
|
{ 'a' => base_value, 'b' => base_value, 'c' => base_value } => %i[sig3 sig1],
|
@@ -17,8 +17,8 @@ describe MatchReduce::Processor do
|
|
17
17
|
end
|
18
18
|
|
19
19
|
def snapshot(snapshot)
|
20
|
-
records
|
21
|
-
|
20
|
+
records = snapshot.fetch('records', [])
|
21
|
+
aggregators = snapshot.fetch('aggregators', []).map do |a|
|
22
22
|
{
|
23
23
|
name: a['name'],
|
24
24
|
patterns: a['patterns'],
|
@@ -33,25 +33,23 @@ describe MatchReduce::Processor do
|
|
33
33
|
|
34
34
|
OpenStruct.new(
|
35
35
|
records: records,
|
36
|
-
|
36
|
+
aggregators: aggregators,
|
37
37
|
results: results
|
38
38
|
)
|
39
39
|
end
|
40
40
|
|
41
|
-
let(:resolver) { Objectable.resolver }
|
42
|
-
|
43
41
|
describe 'snapshots' do
|
44
42
|
yaml_fixture_files('snapshots', 'processor').each_pair do |filename, snapshot_config|
|
45
43
|
specify File.basename(filename) do
|
46
44
|
example = snapshot(snapshot_config)
|
47
45
|
|
48
|
-
subject = described_class.new(example.
|
46
|
+
subject = described_class.new(example.aggregators)
|
49
47
|
|
50
48
|
results = subject.add_each(example.records).results
|
51
49
|
|
52
|
-
err_msg = "invalid: #{example.results.length} results != #{example.
|
50
|
+
err_msg = "invalid: #{example.results.length} results != #{example.aggregators.length} aggs"
|
53
51
|
|
54
|
-
expect(example.results.length).to eq(example.
|
52
|
+
expect(example.results.length).to eq(example.aggregators.length), err_msg
|
55
53
|
|
56
54
|
results.each_with_index do |result, i|
|
57
55
|
expect(result).to eq(example.results[i])
|
data/spec/match_reduce_spec.rb
CHANGED
@@ -10,7 +10,7 @@
|
|
10
10
|
require 'spec_helper'
|
11
11
|
|
12
12
|
class ProcessorMock
|
13
|
-
def initialize(
|
13
|
+
def initialize(_aggregators, resolver: nil, any: nil); end
|
14
14
|
|
15
15
|
def add_each(_records)
|
16
16
|
self
|
@@ -23,17 +23,20 @@ end
|
|
23
23
|
|
24
24
|
describe MatchReduce do
|
25
25
|
specify '#process should create new Processor, call add_each, then call results' do
|
26
|
-
resolver =
|
27
|
-
|
28
|
-
|
29
|
-
|
26
|
+
resolver = 1
|
27
|
+
any = 2
|
28
|
+
aggregators = 3
|
29
|
+
records = 4
|
30
30
|
|
31
|
-
processor = ProcessorMock.new(
|
31
|
+
processor = ProcessorMock.new(aggregators, resolver: resolver, any: any)
|
32
|
+
|
33
|
+
expect(MatchReduce::Processor).to receive(:new).with(aggregators, resolver: resolver, any: any)
|
34
|
+
.and_return(processor)
|
32
35
|
|
33
|
-
expect(MatchReduce::Processor).to receive(:new).with(aggregates, resolver).and_return(processor)
|
34
36
|
expect(processor).to receive(:add_each).with(records).and_return(processor)
|
35
|
-
expect(processor).to receive(:results).and_return(results)
|
36
37
|
|
37
|
-
|
38
|
+
expect(processor).to receive(:results)
|
39
|
+
|
40
|
+
described_class.process(aggregators, records, resolver: resolver, any: any)
|
38
41
|
end
|
39
42
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: match_reduce
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0.0
|
4
|
+
version: 1.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Matthew Ruggio
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2019-09-
|
11
|
+
date: 2019-09-18 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: acts_as_hashable
|
@@ -34,16 +34,16 @@ dependencies:
|
|
34
34
|
name: hash_math
|
35
35
|
requirement: !ruby/object:Gem::Requirement
|
36
36
|
requirements:
|
37
|
-
- - "
|
37
|
+
- - "~>"
|
38
38
|
- !ruby/object:Gem::Version
|
39
|
-
version: 1
|
39
|
+
version: '1'
|
40
40
|
type: :runtime
|
41
41
|
prerelease: false
|
42
42
|
version_requirements: !ruby/object:Gem::Requirement
|
43
43
|
requirements:
|
44
|
-
- - "
|
44
|
+
- - "~>"
|
45
45
|
- !ruby/object:Gem::Version
|
46
|
-
version: 1
|
46
|
+
version: '1'
|
47
47
|
- !ruby/object:Gem::Dependency
|
48
48
|
name: objectable
|
49
49
|
requirement: !ruby/object:Gem::Requirement
|
@@ -179,8 +179,7 @@ files:
|
|
179
179
|
- Rakefile
|
180
180
|
- bin/console
|
181
181
|
- lib/match_reduce.rb
|
182
|
-
- lib/match_reduce/
|
183
|
-
- lib/match_reduce/any.rb
|
182
|
+
- lib/match_reduce/aggregator.rb
|
184
183
|
- lib/match_reduce/index.rb
|
185
184
|
- lib/match_reduce/processor.rb
|
186
185
|
- lib/match_reduce/processor/result.rb
|
@@ -190,8 +189,7 @@ files:
|
|
190
189
|
- match_reduce.gemspec
|
191
190
|
- spec/fixtures/snapshots/processor/abstract.yaml
|
192
191
|
- spec/fixtures/snapshots/processor/teams_and_players.yaml
|
193
|
-
- spec/match_reduce/
|
194
|
-
- spec/match_reduce/any_spec.rb
|
192
|
+
- spec/match_reduce/aggregator_spec.rb
|
195
193
|
- spec/match_reduce/index_spec.rb
|
196
194
|
- spec/match_reduce/processor/result_spec.rb
|
197
195
|
- spec/match_reduce/processor_spec.rb
|
@@ -212,9 +210,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
212
210
|
version: 2.3.8
|
213
211
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
214
212
|
requirements:
|
215
|
-
- - "
|
213
|
+
- - ">="
|
216
214
|
- !ruby/object:Gem::Version
|
217
|
-
version:
|
215
|
+
version: '0'
|
218
216
|
requirements: []
|
219
217
|
rubygems_version: 3.0.3
|
220
218
|
signing_key:
|
@@ -223,8 +221,7 @@ summary: Dataset aggregation and reducer algorithm
|
|
223
221
|
test_files:
|
224
222
|
- spec/fixtures/snapshots/processor/abstract.yaml
|
225
223
|
- spec/fixtures/snapshots/processor/teams_and_players.yaml
|
226
|
-
- spec/match_reduce/
|
227
|
-
- spec/match_reduce/any_spec.rb
|
224
|
+
- spec/match_reduce/aggregator_spec.rb
|
228
225
|
- spec/match_reduce/index_spec.rb
|
229
226
|
- spec/match_reduce/processor/result_spec.rb
|
230
227
|
- spec/match_reduce/processor_spec.rb
|
data/lib/match_reduce/any.rb
DELETED
@@ -1,30 +0,0 @@
|
|
1
|
-
# frozen_string_literal: true
|
2
|
-
|
3
|
-
#
|
4
|
-
# Copyright (c) 2019-present, Blue Marble Payroll, LLC
|
5
|
-
#
|
6
|
-
# This source code is licensed under the MIT license found in the
|
7
|
-
# LICENSE file in the root directory of this source tree.
|
8
|
-
#
|
9
|
-
|
10
|
-
module MatchReduce
|
11
|
-
# This class represents a wildcard. Direct use of this class should be avoided, instead,
|
12
|
-
# use the MatchReduce top-level-declared helper ANY constant.
|
13
|
-
class Any
|
14
|
-
SPECIAL_VALUE = [name, :any].freeze
|
15
|
-
|
16
|
-
private_constant :SPECIAL_VALUE
|
17
|
-
|
18
|
-
# Just be something totally unique. Matching values cannot actually be an array, therefore
|
19
|
-
# there should never be a chance of a collision if the hash of this object is based on
|
20
|
-
# an array structure.
|
21
|
-
def hash
|
22
|
-
SPECIAL_VALUE.hash
|
23
|
-
end
|
24
|
-
|
25
|
-
def ==(other)
|
26
|
-
other.instance_of?(self.class) && hash == other.hash
|
27
|
-
end
|
28
|
-
alias eql? ==
|
29
|
-
end
|
30
|
-
end
|
@@ -1,32 +0,0 @@
|
|
1
|
-
# frozen_string_literal: true
|
2
|
-
|
3
|
-
#
|
4
|
-
# Copyright (c) 2019-present, Blue Marble Payroll, LLC
|
5
|
-
#
|
6
|
-
# This source code is licensed under the MIT license found in the
|
7
|
-
# LICENSE file in the root directory of this source tree.
|
8
|
-
#
|
9
|
-
|
10
|
-
require 'spec_helper'
|
11
|
-
|
12
|
-
describe MatchReduce::Any do
|
13
|
-
describe '#hash' do
|
14
|
-
it 'should be based on an array of the class name and the symbol :any' do
|
15
|
-
expect(described_class.new.hash).to eq(['MatchReduce::Any', :any].hash)
|
16
|
-
end
|
17
|
-
end
|
18
|
-
|
19
|
-
describe 'equality' do
|
20
|
-
specify '#== should always be equal if the classes and hash are the same' do
|
21
|
-
expect(described_class.new).to eq(described_class.new)
|
22
|
-
|
23
|
-
expect(described_class.new).not_to eq(:any)
|
24
|
-
end
|
25
|
-
|
26
|
-
specify '#eql? should always be equal if the classes and hash are the same' do
|
27
|
-
expect(described_class.new).to eql(described_class.new)
|
28
|
-
|
29
|
-
expect(described_class.new).not_to eq(:any)
|
30
|
-
end
|
31
|
-
end
|
32
|
-
end
|