jgrep 1.4.1 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.markdown +5 -0
- data/README.markdown +13 -12
- data/Rakefile +6 -1
- data/bin/jgrep +115 -129
- data/lib/jgrep.rb +286 -377
- data/lib/parser/parser.rb +109 -125
- data/lib/parser/scanner.rb +148 -149
- data/spec/Rakefile +3 -3
- data/spec/spec_helper.rb +1 -2
- data/spec/unit/jgrep_spec.rb +233 -233
- data/spec/unit/parser_spec.rb +132 -127
- data/spec/unit/scanner_spec.rb +82 -86
- metadata +6 -18
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: bc2ae50ca974a9033f6bca65254c009705305587
|
4
|
+
data.tar.gz: 36c22db07d3980bd372071c537bde74c846ccfed
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 85342e07b0e672f19cebda3b426271736b02097df611b98171f54d297e5f2d394c9a2b83fe3a42ce4936f281bdd26806ff89659a2bd52fc3ee54d05b2772b3cd
|
7
|
+
data.tar.gz: 86ac2ba2a35df8aa5a7884f3b223a1e82a2d4b2980b69e86b84cdfde19227a3fd541f522d427b63891f1f482ce5d4b47a689798123bdfb5e142f28fc2a437798
|
data/CHANGELOG.markdown
CHANGED
@@ -1,5 +1,10 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## 1.5.0
|
4
|
+
* Dropped support for Ruby 1.8.3
|
5
|
+
* Added support for modern Ruby versions (Tested up to 2.4.0)
|
6
|
+
* Added utility method to validate expressions
|
7
|
+
|
3
8
|
## 1.4.1
|
4
9
|
* Fix binary exit code to be 1 when no matches are found (Mickaël Canévet)
|
5
10
|
|
data/README.markdown
CHANGED
@@ -1,12 +1,12 @@
|
|
1
1
|
JGrep is a command line tool and API for parsing JSON documents based on logical expressions.
|
2
2
|
|
3
|
-
###Installation
|
3
|
+
### Installation:
|
4
4
|
|
5
5
|
jgrep is available as a gem:
|
6
6
|
|
7
7
|
gem install jgrep
|
8
8
|
|
9
|
-
###JGrep binary usage
|
9
|
+
### JGrep binary usage:
|
10
10
|
|
11
11
|
jgrep [expression] -i foo.json
|
12
12
|
|
@@ -14,7 +14,7 @@ or
|
|
14
14
|
|
15
15
|
cat "foo.json" | jgrep [expression]
|
16
16
|
|
17
|
-
###Flags
|
17
|
+
### Flags:
|
18
18
|
|
19
19
|
-s, --simple [FIELDS] : Greps the JSON and only returns the value of the field(s) specified
|
20
20
|
-c, --compat : Returns the JSON in its non-pretty flat form
|
@@ -26,7 +26,7 @@ or
|
|
26
26
|
--start FIELD : Starts the grep at a specific key in the document
|
27
27
|
--slice [RANGE] : A range of the form 'n' or 'n..m', indicating which documents to extract from the final output
|
28
28
|
|
29
|
-
###Expressions
|
29
|
+
### Expressions:
|
30
30
|
|
31
31
|
JGrep uses the following logical symbols to define expressions.
|
32
32
|
|
@@ -62,7 +62,7 @@ JGrep uses the following logical symbols to define expressions.
|
|
62
62
|
|
63
63
|
Performs the operations inside the perentheses first.
|
64
64
|
|
65
|
-
###Statements
|
65
|
+
### Statements:
|
66
66
|
|
67
67
|
A statement is defined as some value in a json document compared to another value.
|
68
68
|
Available comparison operators are '=', '<', '>', '<=', '>='
|
@@ -73,7 +73,7 @@ Examples:
|
|
73
73
|
foo.bar>0
|
74
74
|
foo.bar<=1.3
|
75
75
|
|
76
|
-
###Complex expressions
|
76
|
+
### Complex expressions:
|
77
77
|
|
78
78
|
Given a json document, {"foo":1, "bar":null}, the following are examples of valid expressions
|
79
79
|
|
@@ -95,12 +95,12 @@ Examples:
|
|
95
95
|
|
96
96
|
... returns true
|
97
97
|
|
98
|
-
###CLI missing an expression
|
98
|
+
### CLI missing an expression:
|
99
99
|
|
100
100
|
If JGrep is executed without a set expression, it will return an unmodified JSON document. The
|
101
101
|
-s flag can still be applied to the result.
|
102
102
|
|
103
|
-
###In document comparison
|
103
|
+
### In document comparison:
|
104
104
|
|
105
105
|
If a document contains an array, the '[' and ']' operators can be used to define a comparison where
|
106
106
|
statements are checked for truth on a per element basis which will then be combined.
|
@@ -152,7 +152,7 @@ will return
|
|
152
152
|
|
153
153
|
**Note**: In document comparison cannot be nested.
|
154
154
|
|
155
|
-
###The -s flag
|
155
|
+
### The -s flag:
|
156
156
|
|
157
157
|
The s flag simplifies the output returned by JGrep. Given a JSON document
|
158
158
|
|
@@ -167,7 +167,7 @@ will output
|
|
167
167
|
1
|
168
168
|
|
169
169
|
The s flag can also be used with multiple field, which will return JSON as output which only contain the specified fields.
|
170
|
-
**Note**: Separate fields by a space and enclose all fields in quotes (see example below)
|
170
|
+
**Note**: Separate fields by a space and enclose all fields in quotes (see example below)
|
171
171
|
|
172
172
|
Given:
|
173
173
|
|
@@ -190,7 +190,7 @@ will output
|
|
190
190
|
}
|
191
191
|
]
|
192
192
|
|
193
|
-
###The --start flag
|
193
|
+
### The --start flag:
|
194
194
|
|
195
195
|
Some documents do not comply to our expected format, they might have an array embedded deep in a field. The --start
|
196
196
|
flag lets you pick a starting point for the grep.
|
@@ -230,7 +230,7 @@ With the --stream or -n flag, jgrep will process multiple JSON inputs (newline
|
|
230
230
|
separated) until standard input is closed. Each JSON input will be processed
|
231
231
|
as usual, but the output immediately printed.
|
232
232
|
|
233
|
-
###JGrep Gem usage
|
233
|
+
### JGrep Gem usage:
|
234
234
|
|
235
235
|
require 'jgrep'
|
236
236
|
|
@@ -242,3 +242,4 @@ as usual, but the output immediately printed.
|
|
242
242
|
sflags = "foo"
|
243
243
|
|
244
244
|
JGrep::jgrep(json, expression, sflags)
|
245
|
+
|
data/Rakefile
CHANGED
data/bin/jgrep
CHANGED
@@ -1,153 +1,139 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
2
|
|
3
|
-
require
|
4
|
-
require
|
3
|
+
require "jgrep"
|
4
|
+
require "optparse"
|
5
5
|
|
6
|
-
@options = {:
|
6
|
+
@options = {flat: false, start: nil, field: [], slice: nil}
|
7
7
|
|
8
8
|
def print_json(result)
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
9
|
+
if @options[:flat]
|
10
|
+
puts(result.first.to_json)
|
11
|
+
else
|
12
|
+
result = result.first if @options[:stream]
|
13
|
+
puts(JSON.pretty_generate(result))
|
14
|
+
end
|
15
15
|
end
|
16
16
|
|
17
17
|
def do_grep(json, expression)
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
18
|
+
if @options[:field].empty?
|
19
|
+
result = JGrep.jgrep(json, expression, nil, @options[:start])
|
20
|
+
result = result.slice(@options[:slice]) if @options[:slice]
|
21
|
+
|
22
|
+
exit 1 if result == []
|
23
|
+
|
24
|
+
print_json(result) unless @options[:quiet] == true
|
25
|
+
elsif @options[:field].size > 1
|
26
|
+
JGrep.validate_filters(@options[:field])
|
27
|
+
result = JGrep.jgrep(json, expression, @options[:field], @options[:start])
|
28
|
+
result = result.slice(@options[:slice]) if @options[:slice]
|
29
|
+
|
30
|
+
exit 1 if result == []
|
31
|
+
|
32
|
+
print_json(result) unless @options[:quiet] == true
|
33
|
+
|
34
|
+
else
|
35
|
+
JGrep.validate_filters(@options[:field][0])
|
36
|
+
result = JGrep.jgrep(json, expression, @options[:field][0], @options[:start])
|
37
|
+
result = result.slice(@options[:slice]) if @options[:slice]
|
38
|
+
exit 1 if result == []
|
39
|
+
if result.is_a?(Array) && !(result.first.is_a?(Hash) || result.flatten.first.is_a?(Hash))
|
40
|
+
unless @options[:quiet] == true
|
41
|
+
result.map {|x| puts x unless x.nil?}
|
42
|
+
end
|
25
43
|
else
|
26
|
-
|
27
|
-
JGrep::validate_filters(@options[:field])
|
28
|
-
result = JGrep::jgrep((json), expression, @options[:field], @options[:start])
|
29
|
-
result = result.slice(@options[:slice]) if @options[:slice]
|
30
|
-
exit 1 if result == []
|
31
|
-
unless @options[:quiet] == true
|
32
|
-
print_json(result)
|
33
|
-
end
|
34
|
-
|
35
|
-
else
|
36
|
-
JGrep::validate_filters(@options[:field][0])
|
37
|
-
result = JGrep::jgrep((json), expression, @options[:field][0], @options[:start])
|
38
|
-
result = result.slice(@options[:slice]) if @options[:slice]
|
39
|
-
exit 1 if result == []
|
40
|
-
if result.is_a?(Array) && !(result.first.is_a?(Hash) || result.flatten.first.is_a?(Hash))
|
41
|
-
unless @options[:quiet] == true
|
42
|
-
result.map{|x| puts x unless x.nil?}
|
43
|
-
end
|
44
|
-
else
|
45
|
-
unless @options[:quiet] == true
|
46
|
-
print_json(result)
|
47
|
-
end
|
48
|
-
end
|
49
|
-
end
|
44
|
+
print_json(result) unless @options[:quiet] == true
|
50
45
|
end
|
46
|
+
end
|
51
47
|
end
|
52
48
|
|
53
49
|
begin
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
|
63
|
-
|
64
|
-
|
65
|
-
@options[:flat] = true
|
66
|
-
end
|
67
|
-
|
68
|
-
opts.on("-n", "--stream", "Display continuous output from continuous input") do
|
69
|
-
@options[:stream] = true
|
70
|
-
end
|
71
|
-
|
72
|
-
opts.on("-f", "--flatten", "Makes output as flat as possible") do
|
73
|
-
JGrep::flatten_on
|
74
|
-
end
|
75
|
-
|
76
|
-
opts.on("-i", "--input [FILENAME]", "Specify input file to parse") do |filename|
|
77
|
-
@options[:file] = filename
|
78
|
-
end
|
79
|
-
|
80
|
-
opts.on("-q", "--quiet", "Quiet; don't write to stdout. Exit with zero status if match found.") do
|
81
|
-
@options[:quiet] = true
|
82
|
-
end
|
83
|
-
|
84
|
-
opts.on("-v", "--verbose", "Verbose output") do
|
85
|
-
JGrep::verbose_on
|
86
|
-
end
|
87
|
-
|
88
|
-
opts.on("--start [FIELD]", "Where in the data to start from") do |field|
|
89
|
-
@options[:start] = field
|
90
|
-
end
|
91
|
-
|
92
|
-
opts.on("--slice [RANGE]", "A range of the form 'n' or 'n..m', indicating which documents to extract from the final output") do |field|
|
93
|
-
range_nums = field.split('..').map{ |x| x.to_i }
|
94
|
-
@options[:slice] = range_nums.length == 1 ? range_nums[0] : Range.new(*range_nums)
|
95
|
-
end
|
96
|
-
end.parse!
|
97
|
-
rescue OptionParser::InvalidOption => e
|
98
|
-
puts e.to_s.capitalize
|
99
|
-
exit 1
|
50
|
+
OptionParser.new do |opts|
|
51
|
+
opts.banner = "Usage: jgrep [options] \"expression\""
|
52
|
+
opts.on("-s", "--simple [FIELDS]", "Display only one or more fields from each of the resulting json documents") do |field|
|
53
|
+
raise "-s flag requires a field value" if field.nil?
|
54
|
+
|
55
|
+
@options[:field].concat(field.split(" "))
|
56
|
+
end
|
57
|
+
|
58
|
+
opts.on("-c", "--compact", "Display non pretty json") do
|
59
|
+
@options[:flat] = true
|
60
|
+
end
|
100
61
|
|
101
|
-
|
102
|
-
|
103
|
-
|
62
|
+
opts.on("-n", "--stream", "Display continuous output from continuous input") do
|
63
|
+
@options[:stream] = true
|
64
|
+
end
|
65
|
+
|
66
|
+
opts.on("-f", "--flatten", "Makes output as flat as possible") do
|
67
|
+
JGrep.flatten_on
|
68
|
+
end
|
69
|
+
|
70
|
+
opts.on("-i", "--input [FILENAME]", "Specify input file to parse") do |filename|
|
71
|
+
@options[:file] = filename
|
72
|
+
end
|
73
|
+
|
74
|
+
opts.on("-q", "--quiet", "Quiet; don't write to stdout. Exit with zero status if match found.") do
|
75
|
+
@options[:quiet] = true
|
76
|
+
end
|
77
|
+
|
78
|
+
opts.on("-v", "--verbose", "Verbose output") do
|
79
|
+
JGrep.verbose_on
|
80
|
+
end
|
81
|
+
|
82
|
+
opts.on("--start [FIELD]", "Where in the data to start from") do |field|
|
83
|
+
@options[:start] = field
|
84
|
+
end
|
85
|
+
|
86
|
+
opts.on("--slice [RANGE]", "A range of the form 'n' or 'n..m', indicating which documents to extract from the final output") do |field|
|
87
|
+
range_nums = field.split("..").map(&:to_i)
|
88
|
+
@options[:slice] = range_nums.length == 1 ? range_nums[0] : Range.new(*range_nums)
|
89
|
+
end
|
90
|
+
end.parse!
|
91
|
+
rescue OptionParser::InvalidOption => e
|
92
|
+
puts e.to_s.capitalize
|
93
|
+
exit 1
|
94
|
+
rescue Exception => e # rubocop:disable Lint/RescueException
|
95
|
+
puts e
|
96
|
+
exit 1
|
104
97
|
end
|
105
98
|
|
106
99
|
begin
|
107
|
-
|
108
|
-
|
109
|
-
|
110
|
-
|
111
|
-
|
112
|
-
|
113
|
-
|
114
|
-
end
|
100
|
+
expression = nil
|
101
|
+
|
102
|
+
# Identify the expression from command line arguments
|
103
|
+
ARGV.each do |argument|
|
104
|
+
if argument =~ /<|>|=|\+|-/
|
105
|
+
expression = argument
|
106
|
+
ARGV.delete(argument)
|
115
107
|
end
|
108
|
+
end
|
116
109
|
|
117
|
-
|
110
|
+
expression = "" if expression.nil?
|
118
111
|
|
119
|
-
|
120
|
-
|
121
|
-
|
112
|
+
# Continuously gets if inputstream in constant
|
113
|
+
# Load json from standard input if tty is false
|
114
|
+
# else find and load file from command line arugments
|
122
115
|
|
123
|
-
|
124
|
-
|
125
|
-
while json = gets
|
126
|
-
do_grep(json, expression)
|
127
|
-
end
|
128
|
-
else
|
129
|
-
raise "No json input specified"
|
130
|
-
end
|
131
|
-
else
|
132
|
-
if @options[:file]
|
133
|
-
json = File.read(@options[:file])
|
134
|
-
do_grep(json, expression)
|
135
|
-
elsif ! STDIN.tty?
|
136
|
-
json = STDIN.read
|
137
|
-
do_grep(json, expression)
|
138
|
-
else
|
139
|
-
raise "No json input specified"
|
140
|
-
end
|
141
|
-
end
|
116
|
+
if @options[:stream]
|
117
|
+
raise "No json input specified" if STDIN.tty?
|
142
118
|
|
143
|
-
|
144
|
-
|
145
|
-
exit 1
|
146
|
-
rescue Exception => e
|
147
|
-
if e.is_a?(SystemExit)
|
148
|
-
exit e.status
|
149
|
-
else
|
150
|
-
STDERR.puts "Error - #{e}"
|
151
|
-
exit 1
|
119
|
+
while json = gets
|
120
|
+
do_grep(json, expression)
|
152
121
|
end
|
122
|
+
elsif @options[:file]
|
123
|
+
json = File.read(@options[:file])
|
124
|
+
do_grep(json, expression)
|
125
|
+
elsif !STDIN.tty?
|
126
|
+
json = STDIN.read
|
127
|
+
do_grep(json, expression)
|
128
|
+
else
|
129
|
+
raise "No json input specified"
|
130
|
+
end
|
131
|
+
rescue Interrupt
|
132
|
+
STDERR.puts "Exiting..."
|
133
|
+
exit 1
|
134
|
+
rescue SystemExit
|
135
|
+
exit e.status
|
136
|
+
rescue Exception => e # rubocop:disable Lint/RescueException
|
137
|
+
STDERR.puts "Error - #{e}"
|
138
|
+
exit 1
|
153
139
|
end
|
data/lib/jgrep.rb
CHANGED
@@ -1,446 +1,355 @@
|
|
1
|
-
|
2
|
-
|
3
|
-
require
|
4
|
-
require
|
5
|
-
require 'rubygems'
|
6
|
-
require 'json'
|
7
|
-
#require 'yajl/json_gem'
|
1
|
+
require "parser/parser.rb"
|
2
|
+
require "parser/scanner.rb"
|
3
|
+
require "rubygems"
|
4
|
+
require "json"
|
8
5
|
|
9
6
|
module JGrep
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
|
41
|
-
|
42
|
-
|
43
|
-
|
44
|
-
|
45
|
-
|
46
|
-
|
47
|
-
|
48
|
-
|
49
|
-
|
50
|
-
else
|
51
|
-
errors = "One or more the json documents could not be parsed. Run jgrep -v for to display documents"
|
52
|
-
end
|
53
|
-
end
|
54
|
-
end
|
7
|
+
@verbose = false
|
8
|
+
@flatten = false
|
9
|
+
|
10
|
+
def self.verbose_on
|
11
|
+
@verbose = true
|
12
|
+
end
|
13
|
+
|
14
|
+
def self.flatten_on
|
15
|
+
@flatten = true
|
16
|
+
end
|
17
|
+
|
18
|
+
# Parse json and return documents that match the logical expression
|
19
|
+
# Filters define output by limiting it to only returning a the listed keys.
|
20
|
+
# Start allows you to move the pointer indicating where parsing starts.
|
21
|
+
# Default is the first key in the document heirarchy
|
22
|
+
def self.jgrep(json, expression, filters = nil, start = nil)
|
23
|
+
errors = ""
|
24
|
+
|
25
|
+
begin
|
26
|
+
JSON.create_id = nil
|
27
|
+
json = JSON.parse(json)
|
28
|
+
json = [json] if json.is_a?(Hash)
|
29
|
+
|
30
|
+
json = filter_json(json, start).flatten if start
|
31
|
+
|
32
|
+
result = []
|
33
|
+
|
34
|
+
if expression == ""
|
35
|
+
result = json
|
36
|
+
else
|
37
|
+
call_stack = Parser.new(expression).execution_stack
|
38
|
+
|
39
|
+
json.each do |document|
|
40
|
+
begin
|
41
|
+
result << document if eval_statement(document, call_stack)
|
42
|
+
rescue Exception => e # rubocop:disable Lint/RescueException
|
43
|
+
if @verbose
|
44
|
+
require "pp"
|
45
|
+
pp document
|
46
|
+
STDERR.puts "Error - #{e} \n\n"
|
55
47
|
else
|
56
|
-
|
48
|
+
errors = "One or more the json documents could not be parsed. Run jgrep -v for to display documents"
|
57
49
|
end
|
50
|
+
end
|
51
|
+
end
|
52
|
+
end
|
58
53
|
|
59
|
-
|
60
|
-
puts errors
|
61
|
-
end
|
54
|
+
puts errors unless errors == ""
|
62
55
|
|
63
|
-
|
64
|
-
return result
|
65
|
-
else
|
66
|
-
filter_json(result, filters)
|
67
|
-
end
|
56
|
+
return result unless filters
|
68
57
|
|
69
|
-
|
70
|
-
|
71
|
-
|
58
|
+
filter_json(result, filters)
|
59
|
+
rescue JSON::ParserError
|
60
|
+
STDERR.puts "Error. Invalid JSON given"
|
72
61
|
end
|
73
|
-
|
74
|
-
|
75
|
-
|
76
|
-
|
77
|
-
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
|
62
|
+
end
|
63
|
+
|
64
|
+
# Validates an expression, true when no errors are found else a string representing the issues
|
65
|
+
def self.validate_expression(expression)
|
66
|
+
Parser.new(expression)
|
67
|
+
true
|
68
|
+
rescue
|
69
|
+
$!.message
|
70
|
+
end
|
71
|
+
|
72
|
+
# Strips filters from json documents and returns those values as a less bloated json document
|
73
|
+
def self.filter_json(documents, filters)
|
74
|
+
result = []
|
75
|
+
|
76
|
+
if filters.is_a? Array
|
77
|
+
documents.each do |doc|
|
78
|
+
tmp_json = {}
|
79
|
+
|
80
|
+
filters.each do |filter|
|
81
|
+
filtered_result = dig_path(doc, filter)
|
82
|
+
unless (filtered_result == doc) || filtered_result.nil?
|
83
|
+
tmp_json[filter] = filtered_result
|
84
|
+
end
|
84
85
|
end
|
86
|
+
result << tmp_json
|
87
|
+
end
|
88
|
+
else
|
89
|
+
documents.each do |r|
|
90
|
+
filtered_result = dig_path(r, filters)
|
91
|
+
|
92
|
+
unless (filtered_result == r) || filtered_result.nil?
|
93
|
+
result << filtered_result
|
94
|
+
end
|
95
|
+
end
|
96
|
+
end
|
85
97
|
|
86
|
-
|
87
|
-
|
88
|
-
begin
|
89
|
-
for i in 0..(documents.size - 1) do
|
90
|
-
tmp = documents[i]
|
91
|
-
unless mark == ""
|
92
|
-
mark.split(".").each_with_index do |m,i|
|
93
|
-
tmp = tmp[m] unless i == mark.split(".").size - 1
|
94
|
-
end
|
95
|
-
end
|
98
|
+
result.flatten if @flatten == true && result.size == 1
|
96
99
|
|
97
|
-
|
98
|
-
|
100
|
+
result
|
101
|
+
end
|
99
102
|
|
100
|
-
|
101
|
-
|
102
|
-
|
103
|
-
|
103
|
+
# Validates if filters do not match any of the parser's logical tokens
|
104
|
+
def self.validate_filters(filters)
|
105
|
+
if filters.is_a? Array
|
106
|
+
filters.each do |filter|
|
107
|
+
if filter =~ /=|<|>|^and$|^or$|^!$|^not$/
|
108
|
+
raise "Invalid field for -s filter : '#{filter}'"
|
104
109
|
end
|
105
|
-
|
106
|
-
|
107
|
-
|
110
|
+
end
|
111
|
+
elsif filters =~ /=|<|>|^and$|^or$|^!$|^not$/
|
112
|
+
raise "Invalid field for -s filter : '#{filters}'"
|
108
113
|
end
|
109
114
|
|
110
|
-
|
111
|
-
|
112
|
-
|
113
|
-
|
114
|
-
|
115
|
-
|
116
|
-
|
117
|
-
|
118
|
-
|
119
|
-
|
120
|
-
|
115
|
+
nil
|
116
|
+
end
|
117
|
+
|
118
|
+
# Correctly format values so we can do the correct type of comparison
|
119
|
+
def self.format(kvalue, value)
|
120
|
+
if kvalue.to_s =~ /^\d+$/ && value.to_s =~ /^\d+$/
|
121
|
+
[Integer(kvalue), Integer(value)]
|
122
|
+
elsif kvalue.to_s =~ /^\d+.\d+$/ && value.to_s =~ /^\d+.\d+$/
|
123
|
+
[Float(kvalue), Float(value)]
|
124
|
+
else
|
125
|
+
[kvalue, value]
|
126
|
+
end
|
127
|
+
end
|
128
|
+
|
129
|
+
# Check if the json key that is defined by statement is defined in the json document
|
130
|
+
def self.present?(document, statement)
|
131
|
+
statement.split(".").each do |key|
|
132
|
+
if document.is_a? Hash
|
133
|
+
if document.value?(nil)
|
134
|
+
document.each do |k, _|
|
135
|
+
document[k] = "null" if document[k].nil?
|
136
|
+
end
|
121
137
|
end
|
138
|
+
end
|
122
139
|
|
123
|
-
|
140
|
+
if document.is_a? Array
|
141
|
+
rval = false
|
142
|
+
document.each do |doc|
|
143
|
+
rval ||= present?(doc, key)
|
144
|
+
end
|
145
|
+
return rval
|
146
|
+
end
|
124
147
|
|
125
|
-
|
126
|
-
for i in 0..(documents.size - 1) do
|
127
|
-
tmp = documents[i]
|
128
|
-
unless mark == ""
|
129
|
-
mark.split(".").each_with_index do |m,i|
|
130
|
-
tmp = tmp[m] unless i == mark.split(".").size - 1
|
131
|
-
end
|
132
|
-
end
|
148
|
+
document = document[key]
|
133
149
|
|
134
|
-
|
135
|
-
|
150
|
+
return false if document.nil?
|
151
|
+
end
|
136
152
|
|
137
|
-
|
138
|
-
|
139
|
-
STDERR.puts "Error. Invalid position specified in JSON document"
|
140
|
-
exit!
|
141
|
-
end
|
153
|
+
true
|
154
|
+
end
|
142
155
|
|
143
|
-
|
156
|
+
# Check if key=value is present in document
|
157
|
+
def self.has_object?(document, statement)
|
158
|
+
key, value = statement.split(/<=|>=|=|<|>/)
|
144
159
|
|
160
|
+
if statement =~ /(<=|>=|<|>|=)/
|
161
|
+
op = $1
|
162
|
+
else
|
163
|
+
op = statement
|
145
164
|
end
|
146
165
|
|
147
|
-
|
148
|
-
def self.filter_json(documents, filters)
|
149
|
-
result = []
|
166
|
+
tmp = dig_path(document, key)
|
150
167
|
|
151
|
-
|
152
|
-
documents.each do |doc|
|
153
|
-
tmp_json = {}
|
154
|
-
filters.each do |filter|
|
155
|
-
filtered_result = dig_path(doc, filter)
|
156
|
-
unless (filtered_result == doc) || filtered_result.nil?
|
157
|
-
tmp_json[filter] = filtered_result
|
158
|
-
end
|
159
|
-
end
|
160
|
-
result << tmp_json
|
161
|
-
end
|
168
|
+
tmp = tmp.first if tmp.is_a?(Array) && tmp.size == 1
|
162
169
|
|
163
|
-
|
164
|
-
return result
|
170
|
+
tmp, value = format(tmp, (value.gsub(/"|'/, "") unless value.nil?)) # rubocop:disable Style/FormatString
|
165
171
|
|
166
|
-
|
167
|
-
|
168
|
-
filtered_result = dig_path(r, filters)
|
169
|
-
unless (filtered_result == r) || filtered_result.nil?
|
170
|
-
result << filtered_result
|
171
|
-
end
|
172
|
-
end
|
172
|
+
# Deal with null comparison
|
173
|
+
return true if tmp.nil? && value == "null"
|
173
174
|
|
174
|
-
|
175
|
-
|
176
|
-
|
177
|
-
end
|
175
|
+
# Deal with booleans
|
176
|
+
return true if tmp == true && value == "true"
|
177
|
+
return true if tmp == false && value == "false"
|
178
178
|
|
179
|
-
#
|
180
|
-
|
181
|
-
|
182
|
-
filters.each do |filter|
|
183
|
-
if filter =~ /=|<|>|^and$|^or$|^!$|^not$/
|
184
|
-
raise "Invalid field for -s filter : '#{filter}'"
|
185
|
-
end
|
186
|
-
end
|
187
|
-
else
|
188
|
-
if filters =~ /=|<|>|^and$|^or$|^!$|^not$/
|
189
|
-
raise "Invalid field for -s filter : '#{filters}'"
|
190
|
-
end
|
191
|
-
end
|
192
|
-
return
|
179
|
+
# Deal with regex matching
|
180
|
+
if !tmp.nil? && value =~ /^\/.*\/$/
|
181
|
+
tmp.match(Regexp.new(value.delete("/"))) ? (return true) : (return false)
|
193
182
|
end
|
194
183
|
|
195
|
-
#
|
196
|
-
|
197
|
-
|
198
|
-
|
199
|
-
|
200
|
-
|
201
|
-
|
202
|
-
|
203
|
-
|
184
|
+
# Deal with everything else
|
185
|
+
case op
|
186
|
+
when "="
|
187
|
+
return tmp == value
|
188
|
+
when "<="
|
189
|
+
return tmp <= value
|
190
|
+
when ">="
|
191
|
+
return tmp >= value
|
192
|
+
when ">"
|
193
|
+
return tmp > value
|
194
|
+
when "<"
|
195
|
+
return tmp < value
|
204
196
|
end
|
197
|
+
end
|
205
198
|
|
199
|
+
# Check if key=value is present in a sub array
|
200
|
+
def self.is_object_in_array?(document, statement)
|
201
|
+
document.each do |item|
|
202
|
+
return true if has_object?(item, statement)
|
203
|
+
end
|
206
204
|
|
207
|
-
|
208
|
-
|
209
|
-
|
210
|
-
|
211
|
-
|
212
|
-
|
213
|
-
|
214
|
-
|
215
|
-
|
216
|
-
|
217
|
-
|
218
|
-
|
219
|
-
|
220
|
-
|
221
|
-
|
222
|
-
|
223
|
-
rval ||= present?(doc, key)
|
224
|
-
end
|
225
|
-
return rval
|
226
|
-
end
|
227
|
-
|
228
|
-
document = document[key]
|
229
|
-
if document.nil?
|
230
|
-
return false
|
231
|
-
end
|
232
|
-
end
|
233
|
-
return true
|
205
|
+
false
|
206
|
+
end
|
207
|
+
|
208
|
+
# Check if complex statement (defined as [key=value...]) is
|
209
|
+
# present over an array of key value pairs
|
210
|
+
def self.has_complex?(document, compound)
|
211
|
+
field = ""
|
212
|
+
tmp = document
|
213
|
+
result = []
|
214
|
+
fresult = []
|
215
|
+
|
216
|
+
compound.each do |token|
|
217
|
+
if token[0] == "statement"
|
218
|
+
field = token
|
219
|
+
break
|
220
|
+
end
|
234
221
|
end
|
235
222
|
|
236
|
-
|
237
|
-
def self.has_object?(document, statement)
|
223
|
+
field = field[1].split(/=|<|>/).first
|
238
224
|
|
239
|
-
|
225
|
+
field.split(".").each_with_index do |item, _|
|
226
|
+
tmp = tmp[item]
|
240
227
|
|
241
|
-
|
242
|
-
op = $1
|
243
|
-
else
|
244
|
-
op = statement
|
245
|
-
end
|
228
|
+
return false if tmp.nil?
|
246
229
|
|
247
|
-
|
230
|
+
next unless tmp.is_a?(Array)
|
248
231
|
|
249
|
-
|
250
|
-
|
251
|
-
end
|
252
|
-
|
253
|
-
tmp, value = format(tmp, (value.gsub(/"|'/, "") unless value.nil?))
|
232
|
+
tmp.each do |doc|
|
233
|
+
result = []
|
254
234
|
|
255
|
-
|
256
|
-
|
257
|
-
|
235
|
+
compound.each do |token|
|
236
|
+
case token[0]
|
237
|
+
when "and"
|
238
|
+
result << "&&"
|
239
|
+
when "or"
|
240
|
+
result << "||"
|
241
|
+
when /not|\!/
|
242
|
+
result << "!"
|
243
|
+
when "statement"
|
244
|
+
op = token[1].match(/.*<=|>=|=|<|>/)
|
245
|
+
left = token[1].split(op[0]).first.split(".").last
|
246
|
+
right = token[1].split(op[0]).last
|
247
|
+
new_statement = left + op[0] + right
|
248
|
+
result << has_object?(doc, new_statement)
|
249
|
+
end
|
258
250
|
end
|
259
251
|
|
260
|
-
#
|
261
|
-
|
262
|
-
|
263
|
-
elsif tmp == false and value == 'false'
|
264
|
-
return true
|
265
|
-
end
|
252
|
+
fresult << eval(result.join(" ")) # rubocop:disable Security/Eval
|
253
|
+
(fresult << "||") unless doc == tmp.last
|
254
|
+
end
|
266
255
|
|
267
|
-
|
268
|
-
if ((value =~ /^\/.*\/$/) && tmp != nil)
|
269
|
-
(tmp.match(Regexp.new(value.gsub("/", "")))) ? (return true) : (return false)
|
270
|
-
end
|
271
|
-
|
272
|
-
#Deal with everything else
|
273
|
-
case op
|
274
|
-
when "="
|
275
|
-
(tmp == value) ? (return true) : (return false)
|
276
|
-
when "<="
|
277
|
-
(tmp <= value) ? (return true) : (return false)
|
278
|
-
when ">="
|
279
|
-
(tmp >= value) ? (return true) : (return false)
|
280
|
-
when ">"
|
281
|
-
(tmp > value) ? (return true) : (return false)
|
282
|
-
when "<"
|
283
|
-
(tmp < value) ? (return true) : (return false)
|
284
|
-
end
|
256
|
+
return eval(fresult.join(" ")) # rubocop:disable Security/Eval
|
285
257
|
end
|
286
|
-
|
287
|
-
|
288
|
-
|
289
|
-
|
290
|
-
|
291
|
-
|
292
|
-
|
293
|
-
|
258
|
+
end
|
259
|
+
|
260
|
+
# Evaluates the call stack en returns true of selected document
|
261
|
+
# matches logical expression
|
262
|
+
def self.eval_statement(document, callstack)
|
263
|
+
result = []
|
264
|
+
|
265
|
+
callstack.each do |expression|
|
266
|
+
case expression.keys.first
|
267
|
+
when "statement"
|
268
|
+
if expression.values.first.is_a?(Array)
|
269
|
+
result << has_complex?(document, expression.values.first)
|
270
|
+
else
|
271
|
+
result << has_object?(document, expression.values.first)
|
294
272
|
end
|
295
|
-
|
296
|
-
|
273
|
+
when "+"
|
274
|
+
result << present?(document, expression.values.first)
|
275
|
+
when "-"
|
276
|
+
result << !present?(document, expression.values.first)
|
277
|
+
when "and"
|
278
|
+
result << "&&"
|
279
|
+
when "or"
|
280
|
+
result << "||"
|
281
|
+
when "("
|
282
|
+
result << "("
|
283
|
+
when ")"
|
284
|
+
result << ")"
|
285
|
+
when "not"
|
286
|
+
result << "!"
|
287
|
+
end
|
297
288
|
end
|
298
289
|
|
299
|
-
|
300
|
-
|
301
|
-
def self.has_complex?(document, compound)
|
302
|
-
field = ""
|
303
|
-
tmp = document
|
304
|
-
result = []
|
305
|
-
fresult = []
|
290
|
+
eval(result.join(" ")) # rubocop:disable Security/Eval
|
291
|
+
end
|
306
292
|
|
307
|
-
|
308
|
-
|
309
|
-
|
310
|
-
|
311
|
-
end
|
312
|
-
end
|
313
|
-
field = field[1].split(/=|<|>/).first
|
293
|
+
# Digs to a specific path in the json document and returns the value
|
294
|
+
def self.dig_path(json, path)
|
295
|
+
index = nil
|
296
|
+
path = path.gsub(/^\./, "")
|
314
297
|
|
315
|
-
|
316
|
-
|
317
|
-
|
318
|
-
return false
|
319
|
-
end
|
320
|
-
if tmp.is_a? Array
|
321
|
-
tmp.each do |doc|
|
322
|
-
result = []
|
323
|
-
compound.each do |token|
|
324
|
-
case token[0]
|
325
|
-
when "and"
|
326
|
-
result << "&&"
|
327
|
-
when "or"
|
328
|
-
result << "||"
|
329
|
-
when /not|\!/
|
330
|
-
result << "!"
|
331
|
-
when "statement"
|
332
|
-
op = token[1].match(/.*<=|>=|=|<|>/)
|
333
|
-
left = token[1].split(op[0]).first.split(".").last
|
334
|
-
right = token[1].split(op[0]).last
|
335
|
-
new_statement = left + op[0] + right
|
336
|
-
result << has_object?(doc, new_statement)
|
337
|
-
end
|
338
|
-
end
|
339
|
-
fresult << eval(result.join(" "))
|
340
|
-
(fresult << "||") unless doc == tmp.last
|
341
|
-
end
|
342
|
-
return eval(fresult.join(" "))
|
343
|
-
end
|
344
|
-
end
|
298
|
+
if path =~ /(.*)\[(.*)\]/
|
299
|
+
path = $1
|
300
|
+
index = $2
|
345
301
|
end
|
346
302
|
|
347
|
-
|
348
|
-
#matches logical expression
|
349
|
-
def self.eval_statement(document, callstack)
|
350
|
-
result = []
|
351
|
-
callstack.each do |expression|
|
352
|
-
case expression.keys.first
|
353
|
-
when "statement"
|
354
|
-
if expression.values.first.is_a? Array
|
355
|
-
result << has_complex?(document, expression.values.first)
|
356
|
-
else
|
357
|
-
result << has_object?(document, expression.values.first)
|
358
|
-
end
|
359
|
-
when "+"
|
360
|
-
result << present?(document, expression.values.first)
|
361
|
-
when "-"
|
362
|
-
result << !(present?(document, expression.values.first))
|
363
|
-
when "and"
|
364
|
-
result << "&&"
|
365
|
-
when "or"
|
366
|
-
result << "||"
|
367
|
-
when "("
|
368
|
-
result << "("
|
369
|
-
when ")"
|
370
|
-
result << ")"
|
371
|
-
when "not"
|
372
|
-
result << "!"
|
373
|
-
end
|
374
|
-
end
|
303
|
+
return json if path == ""
|
375
304
|
|
376
|
-
|
305
|
+
if json.is_a? Hash
|
306
|
+
json.keys.each do |k|
|
307
|
+
if path.start_with?(k) && k.include?(".")
|
308
|
+
return dig_path(json[k], path.gsub(k, ""))
|
309
|
+
end
|
310
|
+
end
|
377
311
|
end
|
378
312
|
|
379
|
-
|
380
|
-
def self.dig_path(json, path)
|
381
|
-
index = nil
|
382
|
-
path = path.gsub(/^\./, "")
|
313
|
+
path_array = path.split(".")
|
383
314
|
|
384
|
-
|
385
|
-
|
386
|
-
index = $2
|
387
|
-
end
|
315
|
+
if path_array.first == "*"
|
316
|
+
tmp = []
|
388
317
|
|
389
|
-
|
390
|
-
|
391
|
-
|
318
|
+
json.each do |j|
|
319
|
+
tmp << dig_path(j[1], path_array.drop(1).join("."))
|
320
|
+
end
|
392
321
|
|
393
|
-
|
394
|
-
|
395
|
-
if path.start_with?(k) && k.include?('.')
|
396
|
-
return dig_path(json[k], path.gsub(k, ""))
|
397
|
-
end
|
398
|
-
end
|
399
|
-
end
|
322
|
+
return tmp
|
323
|
+
end
|
400
324
|
|
401
|
-
|
325
|
+
json = json[path_array.first] if json.is_a? Hash
|
402
326
|
|
403
|
-
|
404
|
-
|
405
|
-
|
406
|
-
tmp << dig_path(j[1], path_array.drop(1).join("."))
|
407
|
-
end
|
408
|
-
return tmp
|
327
|
+
if json.is_a? Hash
|
328
|
+
return json if path == path_array.first
|
329
|
+
return dig_path(json, path.include?(".") ? path_array.drop(1).join(".") : path)
|
409
330
|
|
410
|
-
|
331
|
+
elsif json.is_a? Array
|
332
|
+
if path == path_array.first && (json.first.is_a?(Hash) && !json.first.keys.include?(path))
|
333
|
+
return json
|
334
|
+
end
|
411
335
|
|
412
|
-
|
413
|
-
|
414
|
-
if json.is_a? Hash
|
415
|
-
if path == path_array.first
|
416
|
-
return json
|
417
|
-
else
|
418
|
-
return dig_path(json, (path.include?('.') ? path_array.drop(1).join(".") : path))
|
419
|
-
end
|
336
|
+
tmp = []
|
420
337
|
|
421
|
-
|
422
|
-
|
423
|
-
|
424
|
-
|
425
|
-
tmp = []
|
426
|
-
json.each do |j|
|
427
|
-
tmp_path = dig_path(j, (path.include?('.') ? path_array.drop(1).join(".") : path))
|
428
|
-
unless tmp_path.nil?
|
429
|
-
tmp << tmp_path
|
430
|
-
end
|
431
|
-
end
|
432
|
-
unless tmp.empty?
|
433
|
-
(index) ? (return tmp.flatten[index.to_i]) : (return tmp)
|
434
|
-
end
|
435
|
-
end
|
338
|
+
json.each do |j|
|
339
|
+
tmp_path = dig_path(j, (path.include?(".") ? path_array.drop(1).join(".") : path))
|
340
|
+
tmp << tmp_path unless tmp_path.nil?
|
341
|
+
end
|
436
342
|
|
437
|
-
|
438
|
-
|
343
|
+
unless tmp.empty?
|
344
|
+
return index ? tmp.flatten[index.to_i] : tmp
|
345
|
+
end
|
439
346
|
|
440
|
-
|
441
|
-
|
347
|
+
elsif json.nil?
|
348
|
+
return nil
|
442
349
|
|
443
|
-
|
350
|
+
else
|
351
|
+
return json
|
444
352
|
|
445
353
|
end
|
354
|
+
end
|
446
355
|
end
|