match_reduce 1.0.0.pre.alpha → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +4 -0
- data/README.md +13 -13
- data/lib/match_reduce.rb +6 -10
- data/lib/match_reduce/{aggregate.rb → aggregator.rb} +4 -4
- data/lib/match_reduce/index.rb +13 -13
- data/lib/match_reduce/processor.rb +16 -16
- data/lib/match_reduce/processor/result.rb +1 -1
- data/lib/match_reduce/processor/result_builder.rb +10 -10
- data/lib/match_reduce/processor/results_builder.rb +6 -6
- data/lib/match_reduce/version.rb +1 -1
- data/match_reduce.gemspec +1 -1
- data/spec/fixtures/snapshots/processor/abstract.yaml +1 -1
- data/spec/fixtures/snapshots/processor/teams_and_players.yaml +1 -1
- data/spec/match_reduce/{aggregate_spec.rb → aggregator_spec.rb} +1 -1
- data/spec/match_reduce/index_spec.rb +16 -16
- data/spec/match_reduce/processor_spec.rb +6 -8
- data/spec/match_reduce_spec.rb +12 -9
- metadata +11 -14
- data/lib/match_reduce/any.rb +0 -30
- data/spec/match_reduce/any_spec.rb +0 -32
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c0ca62d59ead19578324d9c7e81ad9f671d6d92b467b5ffe8b110e1b60d0e127
|
4
|
+
data.tar.gz: 63d0e69d3e187763f8fc8c4f2413c2bc00f6a5e61eddfd30e744473f6f04f2a4
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 9eb86c78525d6de4bd4abc06c47c8d27fab79e88488cdde1c3e36c8fa4b06182aee657433cb003a2d7f9986d1422d69e1d61c8777a1be453c2d51efc2ba3d7b9
|
7
|
+
data.tar.gz: 0ba05d835f6bd6403a9a1c4d9d360a4904ea6eca5a92e732cfd207cf8a35c93ab64e4869189c18fd638c586b6f2c8f4d5c7bf357b818669ca8db490c5f9a2c18
|
data/CHANGELOG.md
CHANGED
data/README.md
CHANGED
@@ -4,11 +4,11 @@
|
|
4
4
|
|
5
5
|
MatchReduce is a high-speed, in-memory data aggregation and reduction algorithm. The lifecycle is:
|
6
6
|
|
7
|
-
1. define
|
7
|
+
1. define aggregators
|
8
8
|
2. pump records into algorithm
|
9
9
|
3. grab results
|
10
10
|
|
11
|
-
The dataset will only be processed once no matter how many
|
11
|
+
The dataset will only be processed once no matter how many aggregators you define. An aggregator is expressed as:
|
12
12
|
|
13
13
|
* patterns: an array of hashes, which are used to pattern match records
|
14
14
|
* reducer: a function, which is used when a record matches
|
@@ -37,7 +37,7 @@ A very basic example of calling this library would be:
|
|
37
37
|
````ruby
|
38
38
|
require 'match_reduce'
|
39
39
|
|
40
|
-
|
40
|
+
aggregators = [
|
41
41
|
{
|
42
42
|
name: :total_game_points
|
43
43
|
reducer: ->(memo, record, resolver) { memo.to_i + resolver.get(record, :game_points).to_i },
|
@@ -52,7 +52,7 @@ records = [
|
|
52
52
|
{ game: 2, game_points: 240, team: 'Bulls', team_points: 110 }
|
53
53
|
]
|
54
54
|
|
55
|
-
results = MatchReduce.process(
|
55
|
+
results = MatchReduce.process(aggregators, records)
|
56
56
|
````
|
57
57
|
|
58
58
|
`results` would be equal to:
|
@@ -72,10 +72,10 @@ results = MatchReduce.process(aggregates, records)
|
|
72
72
|
|
73
73
|
Notes:
|
74
74
|
|
75
|
-
* Not specifying patterns means: "match on everything"
|
75
|
+
* Not specifying patterns, as in the example above, means: "match on everything"
|
76
76
|
* group_keys will limit the records matched on, per aggregator, to the first record only. This is why only the first and third records matched.
|
77
|
-
* keys are type-indifferent and will extracted using (the Objectable library)[https://github.com/bluemarblepayroll/objectable]. This means you can leverage dot-notation, non-hash record types, and indifference. You can also customize the resolver used and pass it as a third argument to MatchReduce#process(
|
78
|
-
* Names of
|
77
|
+
* keys are type-indifferent and will extracted using (the Objectable library)[https://github.com/bluemarblepayroll/objectable]. This means you can leverage dot-notation, non-hash record types, and indifference. You can also customize the resolver used and pass it as a third argument to MatchReduce#process(aggregators, records, resolver).
|
78
|
+
* Names of aggregators are type-sensitive, so: `:total_game_points` and `'total_game_points'` are two different aggregators and will produce two different results.
|
79
79
|
|
80
80
|
### Adding Patterns
|
81
81
|
|
@@ -84,7 +84,7 @@ Let's say we want to discretely know how many points the Bulls, Celtics, and Roc
|
|
84
84
|
````ruby
|
85
85
|
require 'match_reduce'
|
86
86
|
|
87
|
-
|
87
|
+
aggregators = [
|
88
88
|
{
|
89
89
|
name: :bulls_points
|
90
90
|
patterns: { team: 'Bulls' },
|
@@ -112,7 +112,7 @@ records = [
|
|
112
112
|
{ game: 2, game_points: 240, team: 'Bulls', team_points: 110 }
|
113
113
|
]
|
114
114
|
|
115
|
-
results = MatchReduce.process(
|
115
|
+
results = MatchReduce.process(aggregators, records)
|
116
116
|
````
|
117
117
|
|
118
118
|
`results` would now be equal to:
|
@@ -144,12 +144,12 @@ results = MatchReduce.process(aggregates, records)
|
|
144
144
|
]
|
145
145
|
````
|
146
146
|
|
147
|
-
We could also choose to
|
147
|
+
We could also choose to aggregator multiple teams together by providing multiple patterns:
|
148
148
|
|
149
149
|
````ruby
|
150
150
|
require 'match_reduce'
|
151
151
|
|
152
|
-
|
152
|
+
aggregators = [
|
153
153
|
{
|
154
154
|
name: :bulls_and_celtics_points
|
155
155
|
patterns: [
|
@@ -168,7 +168,7 @@ records = [
|
|
168
168
|
{ game: 2, game_points: 240, team: 'Bulls', team_points: 110 }
|
169
169
|
]
|
170
170
|
|
171
|
-
results = MatchReduce.process(
|
171
|
+
results = MatchReduce.process(aggregators, records)
|
172
172
|
````
|
173
173
|
|
174
174
|
`results` would now be equal to:
|
@@ -176,7 +176,7 @@ results = MatchReduce.process(aggregates, records)
|
|
176
176
|
````ruby
|
177
177
|
[
|
178
178
|
MatchReduce::Processor::Result.new(
|
179
|
-
name: :
|
179
|
+
name: :bulls_and_celtics_points,
|
180
180
|
records: [
|
181
181
|
{ game: 1, game_points: 199, team: 'Bulls', team_points: 100 },
|
182
182
|
{ game: 1, game_points: 199, team: 'Celtics', team_points: 99 },
|
data/lib/match_reduce.rb
CHANGED
@@ -11,23 +11,19 @@ require 'acts_as_hashable'
|
|
11
11
|
require 'hash_math'
|
12
12
|
require 'objectable'
|
13
13
|
|
14
|
-
require_relative 'match_reduce/
|
15
|
-
require_relative 'match_reduce/aggregate'
|
14
|
+
require_relative 'match_reduce/aggregator'
|
16
15
|
require_relative 'match_reduce/index'
|
17
16
|
require_relative 'match_reduce/processor'
|
18
17
|
|
19
18
|
# Top-level namespace
|
20
19
|
module MatchReduce
|
21
|
-
#
|
22
|
-
#
|
23
|
-
|
24
|
-
# special flag indicating: "match on any value". So even if we were to instantiate
|
25
|
-
# multiple Any objects, the point is moot.
|
26
|
-
ANY = Any.new
|
20
|
+
# Something unique which will represent "match on all values". This is used as the base
|
21
|
+
# value for all pattern keys.
|
22
|
+
ANY = :__ANY__
|
27
23
|
|
28
24
|
class << self
|
29
|
-
def process(
|
30
|
-
Processor.new(
|
25
|
+
def process(aggregators, records, resolver: Objectable.resolver, any: ANY)
|
26
|
+
Processor.new(aggregators, resolver: resolver, any: any)
|
31
27
|
.add_each(records)
|
32
28
|
.results
|
33
29
|
end
|
@@ -8,8 +8,8 @@
|
|
8
8
|
#
|
9
9
|
|
10
10
|
module MatchReduce
|
11
|
-
# An
|
12
|
-
class
|
11
|
+
# An aggregator is a group of patterns with a reducer that you wish to report on.
|
12
|
+
class Aggregator
|
13
13
|
acts_as_hashable
|
14
14
|
|
15
15
|
attr_reader :group_keys,
|
@@ -22,7 +22,7 @@ module MatchReduce
|
|
22
22
|
|
23
23
|
@name = name
|
24
24
|
@group_keys = Array(group_keys)
|
25
|
-
@patterns = stringed_keys(
|
25
|
+
@patterns = stringed_keys(ensure_not_empty(array(patterns)))
|
26
26
|
@reducer = reducer
|
27
27
|
|
28
28
|
freeze
|
@@ -54,7 +54,7 @@ module MatchReduce
|
|
54
54
|
val.is_a?(Hash) ? [val] : Array(val)
|
55
55
|
end
|
56
56
|
|
57
|
-
def
|
57
|
+
def ensure_not_empty(val)
|
58
58
|
val.empty? ? [{}] : val
|
59
59
|
end
|
60
60
|
end
|
data/lib/match_reduce/index.rb
CHANGED
@@ -8,30 +8,30 @@
|
|
8
8
|
#
|
9
9
|
|
10
10
|
module MatchReduce
|
11
|
-
# The Index holds all the
|
12
|
-
# to retrieve
|
11
|
+
# The Index holds all the aggregators, the reverse lookup data structure, and the ability
|
12
|
+
# to retrieve aggregators based on a pattern
|
13
13
|
class Index
|
14
14
|
extend Forwardable
|
15
15
|
|
16
|
-
attr_reader :
|
17
|
-
:
|
16
|
+
attr_reader :aggregators,
|
17
|
+
:any,
|
18
18
|
:lookup
|
19
19
|
|
20
20
|
def_delegators :record, :keys
|
21
21
|
|
22
|
-
def initialize(
|
23
|
-
@
|
24
|
-
@
|
25
|
-
@lookup
|
22
|
+
def initialize(aggregators = [], any: ANY)
|
23
|
+
@any = any
|
24
|
+
@aggregators = Aggregator.array(aggregators).uniq(&:name)
|
25
|
+
@lookup = {}
|
26
26
|
|
27
|
-
all_keys = @
|
28
|
-
@record = HashMath::Record.new(all_keys,
|
27
|
+
all_keys = @aggregators.flat_map(&:keys)
|
28
|
+
@record = HashMath::Record.new(all_keys, any)
|
29
29
|
|
30
|
-
@
|
31
|
-
|
30
|
+
@aggregators.map do |aggregator|
|
31
|
+
aggregator.patterns.each do |pattern|
|
32
32
|
normalized_pattern = record.make!(pattern)
|
33
33
|
|
34
|
-
get(normalized_pattern) <<
|
34
|
+
get(normalized_pattern) << aggregator
|
35
35
|
end
|
36
36
|
end
|
37
37
|
|
@@ -11,18 +11,18 @@ require_relative 'processor/results_builder'
|
|
11
11
|
|
12
12
|
module MatchReduce
|
13
13
|
# This is the main lifecycle of the algorithm. You initialize a new instance of this
|
14
|
-
# class using
|
14
|
+
# class using aggregators, then you pump in records into it. Once done, call #results
|
15
15
|
# to get the results.
|
16
16
|
class Processor
|
17
17
|
extend Forwardable
|
18
18
|
|
19
19
|
def_delegators :results_builder, :results, :resolver
|
20
20
|
|
21
|
-
def_delegators :index, :
|
21
|
+
def_delegators :index, :aggregators, :any
|
22
22
|
|
23
|
-
def initialize(
|
24
|
-
@index = Index.new(
|
25
|
-
@results_builder = ResultsBuilder.new(index.
|
23
|
+
def initialize(aggregators, resolver: Objectable.resolver, any: ANY)
|
24
|
+
@index = Index.new(aggregators, any: any)
|
25
|
+
@results_builder = ResultsBuilder.new(index.aggregators, resolver)
|
26
26
|
|
27
27
|
freeze
|
28
28
|
end
|
@@ -32,16 +32,16 @@ module MatchReduce
|
|
32
32
|
end
|
33
33
|
|
34
34
|
def add(record)
|
35
|
-
|
35
|
+
hit_aggregators = Set.new
|
36
36
|
|
37
37
|
record_patterns(record).each do |hash_pattern|
|
38
|
-
# Each index find hit means the
|
39
|
-
index.find(hash_pattern).each do |
|
40
|
-
next if
|
38
|
+
# Each index find hit means the aggregator matched on the record
|
39
|
+
index.find(hash_pattern).each do |aggregator|
|
40
|
+
next if hit_aggregators.include?(aggregator)
|
41
41
|
|
42
|
-
add_to_results_builder(
|
42
|
+
add_to_results_builder(aggregator, record)
|
43
43
|
|
44
|
-
|
44
|
+
hit_aggregators << aggregator
|
45
45
|
end
|
46
46
|
end
|
47
47
|
|
@@ -53,8 +53,8 @@ module MatchReduce
|
|
53
53
|
attr_reader :index,
|
54
54
|
:results_builder
|
55
55
|
|
56
|
-
def make_group_id(
|
57
|
-
|
56
|
+
def make_group_id(aggregator, record)
|
57
|
+
aggregator.group_keys.map { |group_key| resolver.get(record, group_key) }
|
58
58
|
end
|
59
59
|
|
60
60
|
def record_matrix(record)
|
@@ -69,10 +69,10 @@ module MatchReduce
|
|
69
69
|
[{}] + record_matrix(record).to_a
|
70
70
|
end
|
71
71
|
|
72
|
-
def add_to_results_builder(
|
73
|
-
group_id = make_group_id(
|
72
|
+
def add_to_results_builder(aggregator, record)
|
73
|
+
group_id = make_group_id(aggregator, record)
|
74
74
|
|
75
|
-
results_builder.add(
|
75
|
+
results_builder.add(aggregator, record, group_id)
|
76
76
|
end
|
77
77
|
end
|
78
78
|
end
|
@@ -11,14 +11,14 @@ require_relative 'result'
|
|
11
11
|
|
12
12
|
module MatchReduce
|
13
13
|
class Processor
|
14
|
-
# This class understands how to take an
|
14
|
+
# This class understands how to take an aggregator and derive a result for it.
|
15
15
|
class ResultBuilder
|
16
|
-
def initialize(
|
17
|
-
raise ArgumentError, '
|
18
|
-
raise ArgumentError, 'resolver is required'
|
16
|
+
def initialize(aggregator, resolver)
|
17
|
+
raise ArgumentError, 'aggregator is required' unless aggregator
|
18
|
+
raise ArgumentError, 'resolver is required' unless resolver
|
19
19
|
|
20
|
-
@
|
21
|
-
@resolver
|
20
|
+
@aggregator = aggregator
|
21
|
+
@resolver = resolver
|
22
22
|
|
23
23
|
@records = []
|
24
24
|
@value = nil
|
@@ -26,7 +26,7 @@ module MatchReduce
|
|
26
26
|
end
|
27
27
|
|
28
28
|
def add(record, group_id)
|
29
|
-
if
|
29
|
+
if aggregator.grouped?
|
30
30
|
return self if group_ids.include?(group_id)
|
31
31
|
|
32
32
|
group_ids << group_id
|
@@ -34,18 +34,18 @@ module MatchReduce
|
|
34
34
|
|
35
35
|
records << record
|
36
36
|
|
37
|
-
@value =
|
37
|
+
@value = aggregator.reduce(value, record, resolver)
|
38
38
|
|
39
39
|
self
|
40
40
|
end
|
41
41
|
|
42
42
|
def result
|
43
|
-
Result.new(
|
43
|
+
Result.new(aggregator.name, records, value)
|
44
44
|
end
|
45
45
|
|
46
46
|
private
|
47
47
|
|
48
|
-
attr_reader :
|
48
|
+
attr_reader :aggregator,
|
49
49
|
:group_ids,
|
50
50
|
:records,
|
51
51
|
:resolver,
|
@@ -11,21 +11,21 @@ require_relative 'result_builder'
|
|
11
11
|
|
12
12
|
module MatchReduce
|
13
13
|
class Processor
|
14
|
-
# This class knows how to group together
|
14
|
+
# This class knows how to group together aggregators in order to produce results.
|
15
15
|
class ResultsBuilder
|
16
16
|
attr_reader :resolver
|
17
17
|
|
18
|
-
def initialize(
|
19
|
-
raise ArgumentError, '
|
18
|
+
def initialize(aggregators, resolver)
|
19
|
+
raise ArgumentError, 'aggregators are required' unless aggregators
|
20
20
|
|
21
|
-
@result_by_name =
|
21
|
+
@result_by_name = aggregators.map { |a| [a.name, ResultBuilder.new(a, resolver)] }.to_h
|
22
22
|
@resolver = resolver
|
23
23
|
|
24
24
|
freeze
|
25
25
|
end
|
26
26
|
|
27
|
-
def add(
|
28
|
-
tap { result_by_name[
|
27
|
+
def add(aggregator, record, group_id)
|
28
|
+
tap { result_by_name[aggregator.name].add(record, group_id) }
|
29
29
|
end
|
30
30
|
|
31
31
|
def results
|
data/lib/match_reduce/version.rb
CHANGED
data/match_reduce.gemspec
CHANGED
@@ -22,7 +22,7 @@ Gem::Specification.new do |s|
|
|
22
22
|
s.required_ruby_version = '>= 2.3.8'
|
23
23
|
|
24
24
|
s.add_dependency('acts_as_hashable', '~>1', '>=1.1.0')
|
25
|
-
s.add_dependency('hash_math', '
|
25
|
+
s.add_dependency('hash_math', '~>1')
|
26
26
|
s.add_dependency('objectable', '~>1')
|
27
27
|
|
28
28
|
s.add_development_dependency('guard-rspec', '~>4.7')
|
@@ -11,23 +11,23 @@ require 'spec_helper'
|
|
11
11
|
|
12
12
|
describe MatchReduce::Index do
|
13
13
|
def lookup_by_name(lookup)
|
14
|
-
lookup.each_with_object({}) do |(pattern,
|
15
|
-
memo[pattern] =
|
14
|
+
lookup.each_with_object({}) do |(pattern, aggregators), memo|
|
15
|
+
memo[pattern] = aggregators.map(&:name)
|
16
16
|
end
|
17
17
|
end
|
18
18
|
|
19
19
|
let(:base_value) { MatchReduce::ANY }
|
20
20
|
|
21
21
|
describe '#initialization' do
|
22
|
-
context 'constructing
|
23
|
-
specify 'when all
|
22
|
+
context 'constructing aggregators' do
|
23
|
+
specify 'when all aggregators when empty' do
|
24
24
|
subject = described_class.new
|
25
25
|
|
26
|
-
expect(subject.
|
26
|
+
expect(subject.aggregators).to eq([])
|
27
27
|
end
|
28
28
|
|
29
|
-
specify 'only each first unique
|
30
|
-
|
29
|
+
specify 'only each first unique aggregator name is kept' do
|
30
|
+
aggregators = [
|
31
31
|
{ name: :sig3 },
|
32
32
|
{ name: :sig1 },
|
33
33
|
{ name: 'sig2' },
|
@@ -39,14 +39,14 @@ describe MatchReduce::Index do
|
|
39
39
|
{ name: :sig5 }
|
40
40
|
]
|
41
41
|
|
42
|
-
subject = described_class.new(
|
42
|
+
subject = described_class.new(aggregators)
|
43
43
|
|
44
|
-
expect(subject.
|
44
|
+
expect(subject.aggregators.length).to eq(7)
|
45
45
|
end
|
46
46
|
end
|
47
47
|
|
48
48
|
context 'constructing lookup' do
|
49
|
-
it 'creates lookup with
|
49
|
+
it 'creates lookup with aggregators' do
|
50
50
|
subject = described_class.new
|
51
51
|
|
52
52
|
expected = {}
|
@@ -54,13 +54,13 @@ describe MatchReduce::Index do
|
|
54
54
|
expect(subject.lookup).to eq(expected)
|
55
55
|
end
|
56
56
|
|
57
|
-
it 'creates lookup with
|
58
|
-
|
57
|
+
it 'creates lookup with aggregators that have no patterns' do
|
58
|
+
aggregators = [
|
59
59
|
{ name: :sig3 },
|
60
60
|
{ name: :sig1 }
|
61
61
|
]
|
62
62
|
|
63
|
-
subject = described_class.new(
|
63
|
+
subject = described_class.new(aggregators)
|
64
64
|
|
65
65
|
expected = {
|
66
66
|
{} => %i[sig3 sig1]
|
@@ -69,8 +69,8 @@ describe MatchReduce::Index do
|
|
69
69
|
expect(lookup_by_name(subject.lookup)).to eq(expected)
|
70
70
|
end
|
71
71
|
|
72
|
-
it 'creates lookup with
|
73
|
-
|
72
|
+
it 'creates lookup with aggregators that have patterns and no patterns' do
|
73
|
+
aggregators = [
|
74
74
|
{ name: :sig3 },
|
75
75
|
{ name: :sig1 },
|
76
76
|
{
|
@@ -79,7 +79,7 @@ describe MatchReduce::Index do
|
|
79
79
|
}
|
80
80
|
]
|
81
81
|
|
82
|
-
subject = described_class.new(
|
82
|
+
subject = described_class.new(aggregators)
|
83
83
|
|
84
84
|
expected = {
|
85
85
|
{ 'a' => base_value, 'b' => base_value, 'c' => base_value } => %i[sig3 sig1],
|
@@ -17,8 +17,8 @@ describe MatchReduce::Processor do
|
|
17
17
|
end
|
18
18
|
|
19
19
|
def snapshot(snapshot)
|
20
|
-
records
|
21
|
-
|
20
|
+
records = snapshot.fetch('records', [])
|
21
|
+
aggregators = snapshot.fetch('aggregators', []).map do |a|
|
22
22
|
{
|
23
23
|
name: a['name'],
|
24
24
|
patterns: a['patterns'],
|
@@ -33,25 +33,23 @@ describe MatchReduce::Processor do
|
|
33
33
|
|
34
34
|
OpenStruct.new(
|
35
35
|
records: records,
|
36
|
-
|
36
|
+
aggregators: aggregators,
|
37
37
|
results: results
|
38
38
|
)
|
39
39
|
end
|
40
40
|
|
41
|
-
let(:resolver) { Objectable.resolver }
|
42
|
-
|
43
41
|
describe 'snapshots' do
|
44
42
|
yaml_fixture_files('snapshots', 'processor').each_pair do |filename, snapshot_config|
|
45
43
|
specify File.basename(filename) do
|
46
44
|
example = snapshot(snapshot_config)
|
47
45
|
|
48
|
-
subject = described_class.new(example.
|
46
|
+
subject = described_class.new(example.aggregators)
|
49
47
|
|
50
48
|
results = subject.add_each(example.records).results
|
51
49
|
|
52
|
-
err_msg = "invalid: #{example.results.length} results != #{example.
|
50
|
+
err_msg = "invalid: #{example.results.length} results != #{example.aggregators.length} aggs"
|
53
51
|
|
54
|
-
expect(example.results.length).to eq(example.
|
52
|
+
expect(example.results.length).to eq(example.aggregators.length), err_msg
|
55
53
|
|
56
54
|
results.each_with_index do |result, i|
|
57
55
|
expect(result).to eq(example.results[i])
|
data/spec/match_reduce_spec.rb
CHANGED
@@ -10,7 +10,7 @@
|
|
10
10
|
require 'spec_helper'
|
11
11
|
|
12
12
|
class ProcessorMock
|
13
|
-
def initialize(
|
13
|
+
def initialize(_aggregators, resolver: nil, any: nil); end
|
14
14
|
|
15
15
|
def add_each(_records)
|
16
16
|
self
|
@@ -23,17 +23,20 @@ end
|
|
23
23
|
|
24
24
|
describe MatchReduce do
|
25
25
|
specify '#process should create new Processor, call add_each, then call results' do
|
26
|
-
resolver =
|
27
|
-
|
28
|
-
|
29
|
-
|
26
|
+
resolver = 1
|
27
|
+
any = 2
|
28
|
+
aggregators = 3
|
29
|
+
records = 4
|
30
30
|
|
31
|
-
processor = ProcessorMock.new(
|
31
|
+
processor = ProcessorMock.new(aggregators, resolver: resolver, any: any)
|
32
|
+
|
33
|
+
expect(MatchReduce::Processor).to receive(:new).with(aggregators, resolver: resolver, any: any)
|
34
|
+
.and_return(processor)
|
32
35
|
|
33
|
-
expect(MatchReduce::Processor).to receive(:new).with(aggregates, resolver).and_return(processor)
|
34
36
|
expect(processor).to receive(:add_each).with(records).and_return(processor)
|
35
|
-
expect(processor).to receive(:results).and_return(results)
|
36
37
|
|
37
|
-
|
38
|
+
expect(processor).to receive(:results)
|
39
|
+
|
40
|
+
described_class.process(aggregators, records, resolver: resolver, any: any)
|
38
41
|
end
|
39
42
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: match_reduce
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0.0
|
4
|
+
version: 1.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Matthew Ruggio
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2019-09-
|
11
|
+
date: 2019-09-18 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: acts_as_hashable
|
@@ -34,16 +34,16 @@ dependencies:
|
|
34
34
|
name: hash_math
|
35
35
|
requirement: !ruby/object:Gem::Requirement
|
36
36
|
requirements:
|
37
|
-
- - "
|
37
|
+
- - "~>"
|
38
38
|
- !ruby/object:Gem::Version
|
39
|
-
version: 1
|
39
|
+
version: '1'
|
40
40
|
type: :runtime
|
41
41
|
prerelease: false
|
42
42
|
version_requirements: !ruby/object:Gem::Requirement
|
43
43
|
requirements:
|
44
|
-
- - "
|
44
|
+
- - "~>"
|
45
45
|
- !ruby/object:Gem::Version
|
46
|
-
version: 1
|
46
|
+
version: '1'
|
47
47
|
- !ruby/object:Gem::Dependency
|
48
48
|
name: objectable
|
49
49
|
requirement: !ruby/object:Gem::Requirement
|
@@ -179,8 +179,7 @@ files:
|
|
179
179
|
- Rakefile
|
180
180
|
- bin/console
|
181
181
|
- lib/match_reduce.rb
|
182
|
-
- lib/match_reduce/
|
183
|
-
- lib/match_reduce/any.rb
|
182
|
+
- lib/match_reduce/aggregator.rb
|
184
183
|
- lib/match_reduce/index.rb
|
185
184
|
- lib/match_reduce/processor.rb
|
186
185
|
- lib/match_reduce/processor/result.rb
|
@@ -190,8 +189,7 @@ files:
|
|
190
189
|
- match_reduce.gemspec
|
191
190
|
- spec/fixtures/snapshots/processor/abstract.yaml
|
192
191
|
- spec/fixtures/snapshots/processor/teams_and_players.yaml
|
193
|
-
- spec/match_reduce/
|
194
|
-
- spec/match_reduce/any_spec.rb
|
192
|
+
- spec/match_reduce/aggregator_spec.rb
|
195
193
|
- spec/match_reduce/index_spec.rb
|
196
194
|
- spec/match_reduce/processor/result_spec.rb
|
197
195
|
- spec/match_reduce/processor_spec.rb
|
@@ -212,9 +210,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
212
210
|
version: 2.3.8
|
213
211
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
214
212
|
requirements:
|
215
|
-
- - "
|
213
|
+
- - ">="
|
216
214
|
- !ruby/object:Gem::Version
|
217
|
-
version:
|
215
|
+
version: '0'
|
218
216
|
requirements: []
|
219
217
|
rubygems_version: 3.0.3
|
220
218
|
signing_key:
|
@@ -223,8 +221,7 @@ summary: Dataset aggregation and reducer algorithm
|
|
223
221
|
test_files:
|
224
222
|
- spec/fixtures/snapshots/processor/abstract.yaml
|
225
223
|
- spec/fixtures/snapshots/processor/teams_and_players.yaml
|
226
|
-
- spec/match_reduce/
|
227
|
-
- spec/match_reduce/any_spec.rb
|
224
|
+
- spec/match_reduce/aggregator_spec.rb
|
228
225
|
- spec/match_reduce/index_spec.rb
|
229
226
|
- spec/match_reduce/processor/result_spec.rb
|
230
227
|
- spec/match_reduce/processor_spec.rb
|
data/lib/match_reduce/any.rb
DELETED
@@ -1,30 +0,0 @@
|
|
1
|
-
# frozen_string_literal: true
|
2
|
-
|
3
|
-
#
|
4
|
-
# Copyright (c) 2019-present, Blue Marble Payroll, LLC
|
5
|
-
#
|
6
|
-
# This source code is licensed under the MIT license found in the
|
7
|
-
# LICENSE file in the root directory of this source tree.
|
8
|
-
#
|
9
|
-
|
10
|
-
module MatchReduce
|
11
|
-
# This class represents a wildcard. Direct use of this class should be avoided, instead,
|
12
|
-
# use the MatchReduce top-level-declared helper ANY constant.
|
13
|
-
class Any
|
14
|
-
SPECIAL_VALUE = [name, :any].freeze
|
15
|
-
|
16
|
-
private_constant :SPECIAL_VALUE
|
17
|
-
|
18
|
-
# Just be something totally unique. Matching values cannot actually be an array, therefore
|
19
|
-
# there should never be a chance of a collision if the hash of this object is based on
|
20
|
-
# an array structure.
|
21
|
-
def hash
|
22
|
-
SPECIAL_VALUE.hash
|
23
|
-
end
|
24
|
-
|
25
|
-
def ==(other)
|
26
|
-
other.instance_of?(self.class) && hash == other.hash
|
27
|
-
end
|
28
|
-
alias eql? ==
|
29
|
-
end
|
30
|
-
end
|
@@ -1,32 +0,0 @@
|
|
1
|
-
# frozen_string_literal: true
|
2
|
-
|
3
|
-
#
|
4
|
-
# Copyright (c) 2019-present, Blue Marble Payroll, LLC
|
5
|
-
#
|
6
|
-
# This source code is licensed under the MIT license found in the
|
7
|
-
# LICENSE file in the root directory of this source tree.
|
8
|
-
#
|
9
|
-
|
10
|
-
require 'spec_helper'
|
11
|
-
|
12
|
-
describe MatchReduce::Any do
|
13
|
-
describe '#hash' do
|
14
|
-
it 'should be based on an array of the class name and the symbol :any' do
|
15
|
-
expect(described_class.new.hash).to eq(['MatchReduce::Any', :any].hash)
|
16
|
-
end
|
17
|
-
end
|
18
|
-
|
19
|
-
describe 'equality' do
|
20
|
-
specify '#== should always be equal if the classes and hash are the same' do
|
21
|
-
expect(described_class.new).to eq(described_class.new)
|
22
|
-
|
23
|
-
expect(described_class.new).not_to eq(:any)
|
24
|
-
end
|
25
|
-
|
26
|
-
specify '#eql? should always be equal if the classes and hash are the same' do
|
27
|
-
expect(described_class.new).to eql(described_class.new)
|
28
|
-
|
29
|
-
expect(described_class.new).not_to eq(:any)
|
30
|
-
end
|
31
|
-
end
|
32
|
-
end
|