enumpath 0.1.1 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +196 -42
- data/lib/enumpath/logger.rb +14 -11
- data/lib/enumpath/operator/base.rb +25 -10
- data/lib/enumpath/operator/child.rb +2 -2
- data/lib/enumpath/operator/filter_expression.rb +32 -26
- data/lib/enumpath/operator/recursive_descent.rb +7 -7
- data/lib/enumpath/operator/slice.rb +2 -2
- data/lib/enumpath/operator/subscript_expression.rb +16 -14
- data/lib/enumpath/operator/union.rb +1 -1
- data/lib/enumpath/operator/wildcard.rb +1 -1
- data/lib/enumpath/operator.rb +47 -26
- data/lib/enumpath/path/normalized_path.rb +24 -15
- data/lib/enumpath/path.rb +19 -10
- data/lib/enumpath/resolver/object_inspector.rb +70 -0
- data/lib/enumpath/resolver/property.rb +227 -8
- data/lib/enumpath/resolver/simple.rb +35 -1
- data/lib/enumpath/version.rb +1 -1
- data/lib/enumpath.rb +21 -10
- metadata +67 -39
- data/.gitignore +0 -15
- data/.rspec +0 -3
- data/.travis.yml +0 -5
- data/.yardopts +0 -1
- data/CONTRIBUTING.md +0 -91
- data/Gemfile +0 -6
- data/Gemfile.lock +0 -56
- data/Rakefile +0 -11
- data/bin/console +0 -14
- data/bin/setup +0 -8
- data/enumpath.gemspec +0 -42
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 470c663d2026145fc65af532db8c91cc8735d780b1e04a9a6504df3258555ff1
|
|
4
|
+
data.tar.gz: 48033720373583fc8eeb7f93a788ba25b719ec5491b76505bb4448da04114ed8
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 69aa4c2efc793ddd006cde16adae4d71619d1fb32d7f27f0c588f58f1525951173ac1dc084dd33907234d4dd1fb8164dd7903f3754e19439e1174ce2e4e14ef0
|
|
7
|
+
data.tar.gz: e9cf4de00bd088bd96004955132e2b04009cdf61d90467087c849d32e8c0b5ddbfd553f662b8336d0f5ef5c27bdddc251f227a97d015c2de8db4f6636ef1972a
|
data/README.md
CHANGED
|
@@ -1,14 +1,18 @@
|
|
|
1
1
|
# Enumpath
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Navigate nested Ruby objects with JSONPath-style path expressions, safely by default.
|
|
4
4
|
|
|
5
5
|
## Introduction
|
|
6
6
|
|
|
7
|
-
Enumpath
|
|
7
|
+
Enumpath lets you describe a path through nested Ruby objects as a string, using [JSONPath][jsonpath]-style syntax, and pull out whatever it matches. It works on the data you actually have in memory: hashes with symbol, string or integer keys, arrays, `Struct`s, plain Ruby objects and `OpenStruct`s, in any mix. It is like Ruby's native `dig`, but a path is a plain string that can use wildcards, filters, slices and recursive descent, so it can live in configuration or a database and change without a deploy. (`dig(*keys)` also takes keys at runtime, but only as a fixed chain: it can't say "every item" or "items where the price is over 100".) That makes it a good fit for flexible ETL (Extract, Transform, Load) processes and anywhere else a path should be data instead of code.
|
|
8
|
+
|
|
9
|
+
Paths are safe by default: a path can read your data but cannot call mutating or destructive methods such as `clear` or `destroy`. See [`:property_resolver`](#property_resolver) to allow more. See [Object traversal](#object-traversal) for examples, and [Alternatives](#alternatives) for when another gem is the better choice.
|
|
10
|
+
|
|
11
|
+
Only accept paths from sources you trust. The default resolver stops a path from calling methods that change your data, but a hostile path can still be expensive to evaluate.
|
|
8
12
|
|
|
9
13
|
Enumpath path expressions look like this:
|
|
10
14
|
|
|
11
|
-
```
|
|
15
|
+
```plain
|
|
12
16
|
$.pets.cats.0.name
|
|
13
17
|
|
|
14
18
|
$.pets[cats,dogs].*.name
|
|
@@ -22,13 +26,13 @@ pets..name
|
|
|
22
26
|
pets.cats.-1
|
|
23
27
|
```
|
|
24
28
|
|
|
25
|
-
Enumpath has the following benefits over vanilla `
|
|
29
|
+
Enumpath has the following benefits over vanilla `dig`:
|
|
26
30
|
|
|
27
31
|
- Paths can be described as simple strings
|
|
28
|
-
- It's smart enough to figure out which path segments are integer indexes versus symbolic keys versus string keys
|
|
32
|
+
- It's smart enough to figure out which path segments are integer indexes versus symbolic keys versus string keys versus struct members versus object readers
|
|
29
33
|
- It enables the use of wildcard, recursive descent, filter, subscript, union, and slice operators to describe complex paths through the data
|
|
30
34
|
|
|
31
|
-
Like `
|
|
35
|
+
Like `dig`, Enumpath protects against missing path segments and returns safely if the full path cannot be resolved.
|
|
32
36
|
|
|
33
37
|
## Installation
|
|
34
38
|
|
|
@@ -40,25 +44,29 @@ gem 'enumpath'
|
|
|
40
44
|
|
|
41
45
|
And then execute:
|
|
42
46
|
|
|
43
|
-
|
|
47
|
+
```shell
|
|
48
|
+
bundle
|
|
49
|
+
```
|
|
44
50
|
|
|
45
51
|
Or install it yourself as:
|
|
46
52
|
|
|
47
|
-
|
|
53
|
+
```shell
|
|
54
|
+
gem install enumpath
|
|
55
|
+
```
|
|
48
56
|
|
|
49
57
|
## Usage
|
|
50
58
|
|
|
51
59
|
Enumpath exposes a simple interface via `Enumpath.apply` that takes a path and an enumerable.
|
|
52
60
|
|
|
53
61
|
```ruby
|
|
54
|
-
party = { food: %w
|
|
62
|
+
party = { food: %w[pizza tacos] }
|
|
55
63
|
Enumpath.apply('food.0', party) # => ["pizza"]
|
|
56
64
|
```
|
|
57
65
|
|
|
58
66
|
The result of `Enumpath.apply` is an array of values that were extracted according to the path. Technically it's an instance of `Enumpath::Results` which is an Array-like object that allows you to chain further calls to `.apply` like this:
|
|
59
67
|
|
|
60
68
|
```ruby
|
|
61
|
-
party = { food: %w
|
|
69
|
+
party = { food: %w[pizza tacos] }
|
|
62
70
|
results = Enumpath.apply('food.*', party) # => ["pizza", "tacos"]
|
|
63
71
|
results.apply("[?(@ == 'pizza')]") # => ["pizza"]
|
|
64
72
|
```
|
|
@@ -66,12 +74,70 @@ results.apply("[?(@ == 'pizza')]") # => ["pizza"]
|
|
|
66
74
|
In the event that the path doesn't match anything, an empty results set is returned:
|
|
67
75
|
|
|
68
76
|
```ruby
|
|
69
|
-
party = { food: %w
|
|
77
|
+
party = { food: %w[pizza tacos] }
|
|
70
78
|
Enumpath.apply("drinks.*", party) # => []
|
|
71
79
|
```
|
|
72
80
|
|
|
73
81
|
> This is a thoughtful deviation from the original JSONPath spec which would return `false` on no matches.
|
|
74
82
|
|
|
83
|
+
## Object traversal
|
|
84
|
+
|
|
85
|
+
The same path syntax works on whatever shape your data has. These examples are mirrored by `spec/integration_tests/object_traversal_spec.rb`, so they stay accurate.
|
|
86
|
+
|
|
87
|
+
A hash with symbol keys, using a wildcard and a filter:
|
|
88
|
+
|
|
89
|
+
```ruby
|
|
90
|
+
order = { customer: { name: 'Ada' }, items: [{ sku: 'A1', qty: 2 }, { sku: 'B2', qty: 1 }] }
|
|
91
|
+
Enumpath.apply('items.*.sku', order) # => ["A1", "B2"]
|
|
92
|
+
Enumpath.apply('items[?(@.qty > 1)].sku', order) # => ["A1"]
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
A `Struct` inside a hash, with a relative index:
|
|
96
|
+
|
|
97
|
+
```ruby
|
|
98
|
+
Point = Struct.new(:x, :y)
|
|
99
|
+
route = { stops: [Point.new(1, 2), Point.new(3, 4)] }
|
|
100
|
+
Enumpath.apply('stops.*.x', route) # => [1, 3]
|
|
101
|
+
Enumpath.apply('stops.-1.y', route) # => [4]
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
A plain Ruby object. Under the default `:safe` policy its `attr_reader`s resolve, and so do read-only methods such as `length` on the values they return:
|
|
105
|
+
|
|
106
|
+
```ruby
|
|
107
|
+
class Author
|
|
108
|
+
attr_reader :name, :books
|
|
109
|
+
|
|
110
|
+
def initialize(name, books)
|
|
111
|
+
@name = name
|
|
112
|
+
@books = books
|
|
113
|
+
end
|
|
114
|
+
|
|
115
|
+
def label
|
|
116
|
+
"#{name} (#{books.length} books)"
|
|
117
|
+
end
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
library = { authors: [Author.new('Le Guin', %w[Earthsea Dispossessed]), Author.new('Butler', %w[Kindred])] }
|
|
121
|
+
Enumpath.apply('authors.*.name', library) # => ["Le Guin", "Butler"]
|
|
122
|
+
Enumpath.apply('authors.0.books.length', library) # => [2]
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
A computed reader such as `label` is a method you wrote, not an `attr_reader`, so `:safe` does not call it. Ask for it with `property_resolver: :readers`, which also runs methods from installed gems and is not read-only, so use it on data you trust:
|
|
126
|
+
|
|
127
|
+
```ruby
|
|
128
|
+
Enumpath.apply('authors.*.label', library) # => []
|
|
129
|
+
Enumpath.apply('authors.*.label', library, property_resolver: :readers)
|
|
130
|
+
# => ["Le Guin (2 books)", "Butler (1 books)"]
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
An `OpenStruct`:
|
|
134
|
+
|
|
135
|
+
```ruby
|
|
136
|
+
require 'ostruct'
|
|
137
|
+
config = OpenStruct.new(db: OpenStruct.new(host: 'localhost', port: 5432))
|
|
138
|
+
Enumpath.apply('db.host', config) # => ["localhost"]
|
|
139
|
+
```
|
|
140
|
+
|
|
75
141
|
## Operator Reference
|
|
76
142
|
|
|
77
143
|
Enumpath currently implements the following path operators:
|
|
@@ -96,16 +162,16 @@ Child operators match on an index, key, member, or property of the enumerable. I
|
|
|
96
162
|
1. as an integer key or index (if the segment is integer-like),
|
|
97
163
|
2. then as a string key,
|
|
98
164
|
3. then as a symbol key,
|
|
99
|
-
4. and finally as a public property (i.e. a public method of the target that expects no arguments)
|
|
165
|
+
4. and finally as a public property (i.e. a public method of the target that expects no arguments), if the [`:property_resolver`](#property_resolver) option allows it. By default (`:safe`) only read-only access is allowed: allowlisted methods such as `length`, `size`, `count`, `first`, `keys` or `upcase` on core data (arrays, hashes, strings, symbols, numbers, structs, `Data` objects, `nil`, `true`, `false`), and `attr_reader`s on plain Ruby objects. A path segment like `clear`, `pop`, `sort!` or `destroy` matches nothing. Use `property_resolver: :readers` to also allow computed readers and ActiveRecord attributes (this also runs methods from installed gems and is not read-only; see [`:property_resolver`](#property_resolver)).
|
|
100
166
|
|
|
101
|
-
#### Examples
|
|
167
|
+
#### Operator Examples
|
|
102
168
|
|
|
103
|
-
```
|
|
169
|
+
```ruby
|
|
104
170
|
Car = Struct.new(:color, :transmition, :owners)
|
|
105
|
-
|
|
171
|
+
hyundai = Car.new('blue', :automatic, [{ name: 'Bill' }, { name: 'Ted' }])
|
|
106
172
|
subaru = Car.new('gold', :standard, [{ name: 'Kate' }])
|
|
107
173
|
jeep = Car.new('black', :automatic, [])
|
|
108
|
-
garages = [{ 'cars' => [
|
|
174
|
+
garages = [{ 'cars' => [hyundai, subaru] }, { 'cars' => [jeep] }]
|
|
109
175
|
Enumpath.apply('1', garages) # => [{"cars"=>[#<struct Car color="black", transmition=:automatic, owners=[]>]}]
|
|
110
176
|
Enumpath.apply('0.cars.-1', garages) # => [#<struct Car color="gold", transmition=:standard, owners=[{:name=>"Kate"}]>]
|
|
111
177
|
Enumpath.apply('1.cars.0.owners.length', garages) # => [0]
|
|
@@ -141,7 +207,7 @@ The operator accepts a mixed bag of argument combinations. For instance, these a
|
|
|
141
207
|
|
|
142
208
|
Syntax: `[child1,child2,...]`
|
|
143
209
|
|
|
144
|
-
The union operator combines the results of two or more child operators. There is no limit to the number of child operators you can specify in a single union. Each child operator is separated by a comma (`,`). White space is stripped from around each child operator. Child operators can optionally be wrapped in single quotes. Bracket notation is not
|
|
210
|
+
The union operator combines the results of two or more child operators. There is no limit to the number of child operators you can specify in a single union. Each child operator is separated by a comma (`,`). White space is stripped from around each child operator. Child operators can optionally be wrapped in single quotes. Bracket notation is not supported in this context.
|
|
145
211
|
|
|
146
212
|
The following are all valid union operators:
|
|
147
213
|
|
|
@@ -161,7 +227,7 @@ The following are all valid filter expressions:
|
|
|
161
227
|
- `[?(isbn)]`: equivalent to the previous example
|
|
162
228
|
- `[?(@.price == 8)]`: any member whose `price` value is equal to 8
|
|
163
229
|
- `[?(@.price == 8 || @.price == 10)]`: members with a `price` of 8 _or_ 10
|
|
164
|
-
- `[?(@.price > 2 && @.price < 10)]`: members with a `price` greater than
|
|
230
|
+
- `[?(@.price > 2 && @.price < 10)]`: members with a `price` greater than 2 _and_ less than 10
|
|
165
231
|
- `[?(@.name =~ /bob/i || @.name == 'Mark')]`: any member whose `name` matches the regex `/bob/i` or equals `'Mark'`
|
|
166
232
|
|
|
167
233
|
> Regular expression operands are safely parsed using the `to_regexp` gem
|
|
@@ -170,15 +236,15 @@ The following are all valid filter expressions:
|
|
|
170
236
|
|
|
171
237
|
Syntax: `[(expression)]`
|
|
172
238
|
|
|
173
|
-
A subscript expression is made up of a
|
|
239
|
+
A subscript expression is made up of a single expression that consists of a child operator (`@.child` or `child`; the leading `@.` is optional), plus an optional pair of arithmetic operator and operand. The arithmetic operator can be any one of `+`, `-`, `**`, `*`, `/`, or `%`. The operand can be a string (`'some string'`), symbol (`:some_symbol`), or numeric value (`10` or `1.0`). The expression is evaluated and the result becomes the subscript. If an operator and operand are not included in an expression then the value located at the child operator is used as the subscript. If the subscript represents a valid child path for the enumerable, the value of that member will be passed along for further processing of the path.
|
|
174
240
|
|
|
175
|
-
The following are all valid
|
|
241
|
+
The following are all valid subscript expressions:
|
|
176
242
|
|
|
177
243
|
- `[(@.length - 1)]`: the subscript becomes the length of the current enumerable, minus 1
|
|
178
244
|
- `[(length / 2)]`: the subscript becomes the index at half the length of the enumerable
|
|
179
245
|
- `[(@.type)]`: the subscript becomes the value at the `type` key, member, or property of the enumerable
|
|
180
246
|
|
|
181
|
-
## Examples
|
|
247
|
+
## Usage Examples
|
|
182
248
|
|
|
183
249
|
Given the same store example from the JSONPath project:
|
|
184
250
|
|
|
@@ -373,12 +439,51 @@ Enumpath.apply("$..*", store_info)
|
|
|
373
439
|
By default, Enumpath returns the values that match the path expression. Like the original JSONPath implementation, Enumpath also supports returning path results instead of values. This can be useful for collecting static paths from dynamic paths.
|
|
374
440
|
|
|
375
441
|
```ruby
|
|
376
|
-
party = { food: %w
|
|
442
|
+
party = { food: %w[pizza tacos] }
|
|
377
443
|
Enumpath.apply("food.*", party, result_type: :path) # => ["$['food'][0]", "$['food'][1]"]
|
|
378
444
|
```
|
|
379
445
|
|
|
380
446
|
Each returned path is a valid path expression that can be used in calls to `Enumpath.apply`. If you want to be explicit about returning values instead of paths you can specify that with the option `result_type: :value`.
|
|
381
447
|
|
|
448
|
+
### :property_resolver
|
|
449
|
+
|
|
450
|
+
When a path segment is not an index, key, or struct or `Data` member, Enumpath can resolve it by calling a method on the object (step 4 of the [child operator](#child-operator)). Wildcards, filters and recursive descent also use this option when they need the keys of an object that is not an array, hash, struct or `Data` object (they call its `to_h` only if the option allows it). The `:property_resolver` option controls what a path is allowed to call:
|
|
451
|
+
|
|
452
|
+
value | what resolves | read-only?
|
|
453
|
+
--- | --- | ---
|
|
454
|
+
`:safe` (default) | allowlisted read-only methods on core data, `attr_reader`s on plain objects | yes
|
|
455
|
+
`:readers` | `:safe`, plus public no-argument methods written in Ruby, including installed gems (Ruby's built-in and standard library methods excluded), and ActiveModel-style `attributes` | no
|
|
456
|
+
`:none` | nothing; only indexes, keys and members | yes
|
|
457
|
+
`:all` | any public method that expects no arguments (the behavior before this option existed) | no
|
|
458
|
+
a callable | whatever the callable allows | up to you
|
|
459
|
+
|
|
460
|
+
- `:safe` (the default): read-only access. On core data (arrays, hashes, strings, symbols, numbers, structs, `Data` objects, `nil`, `true`, `false`) Enumpath calls a fixed allowlist of non-mutating methods such as `length`, `size`, `count`, `first`, `last`, `min`, `max`, `sum`, `keys`, `values`, `to_a`, `to_h`, `to_s`, `empty?`, `zero?`, `upcase` or `strip`, and only when the receiver's class does not override them. On plain Ruby objects it reads instance variables that have a public reader of the same name (`attr_reader`s); the instance variable is read directly, so the method itself never runs. Names ending in `!` or `=` are never called, and neither is any method that requires arguments.
|
|
461
|
+
- `:readers`: everything `:safe` allows, plus, on objects that are not core data, any public method that expects no arguments and is written in Ruby, except Ruby's built-in methods, methods in the Ruby standard library directory, and a short denylist of persistence verbs (`destroy`, `delete`, `save`, `update`, `reload`, `touch`, `delete_all`, `destroy_all`, `toggle`). It also reads the keys of an ActiveModel-style `attributes` Hash. So computed readers like `full_name`, `attr_reader`s and ActiveRecord columns like `email` work. Methods implemented in C (core classes, `StringIO`, C-extension gems) are never called; for objects of C-extension classes use a callable or `:all`. Core data is treated exactly like `:safe`. **This is not read-only.** It includes methods from installed gems, including default and bundled gems such as Logger and CSV, and ActiveModel or ActiveRecord: for example `Logger#close` would run, and so would a model method like `archive` or `cancel`. The denylist is a safety net, not a guarantee. Use `:readers` for data you trust or objects you built; for untrusted paths use `:safe` or a callable.
|
|
462
|
+
- `:none`: never call methods.
|
|
463
|
+
- `:all`: call any public method that expects no arguments. This is how Enumpath behaved before this option existed, and it lets a path run methods such as `clear`, `pop`, `sort!` or `destroy`. Only use it with paths you trust.
|
|
464
|
+
- a callable, such as a lambda: called with the object and the method name (a String). The method is called only if it returns true. The callable has full control: none of the checks above are applied, so call `Enumpath::Resolver::Property.safe?(object, name)` or `.reader?(object, name)` from it if you want to keep them and allow more.
|
|
465
|
+
|
|
466
|
+
```ruby
|
|
467
|
+
party = { food: %w[pizza tacos] }
|
|
468
|
+
Enumpath.apply('food.pop', party) # => [] and party is unchanged
|
|
469
|
+
Enumpath.apply('food.length', party) # => [2]
|
|
470
|
+
Enumpath.apply('food.length', party, property_resolver: :none) # => []
|
|
471
|
+
|
|
472
|
+
# ActiveRecord models you trust: computed readers and columns
|
|
473
|
+
Enumpath.apply('[?(@.published)].title', Post.all.to_a, property_resolver: :readers)
|
|
474
|
+
|
|
475
|
+
# Recommended for objects you do not control: keep the safe defaults and allow specific methods per class
|
|
476
|
+
ALLOWED = { 'Invoice' => %w[total due_on], 'Customer' => %w[name email] }.freeze
|
|
477
|
+
policy = lambda do |object, name|
|
|
478
|
+
Enumpath::Resolver::Property.safe?(object, name) || ALLOWED.fetch(object.class.name, []).include?(name)
|
|
479
|
+
end
|
|
480
|
+
Enumpath.apply('invoices.*.total', account, property_resolver: policy)
|
|
481
|
+
```
|
|
482
|
+
|
|
483
|
+
The option applies to a single call. A chained `results.apply(...)` uses the default unless you pass the option again. Any other value raises an `ArgumentError`.
|
|
484
|
+
|
|
485
|
+
To decide whether an object has a method, Enumpath asks Ruby (`respond_to?` and `method`), which calls the object's `respond_to_missing?` if it defines one. So even under `:safe`, a path can run a little of your application's code that way.
|
|
486
|
+
|
|
382
487
|
### :verbose
|
|
383
488
|
|
|
384
489
|
Seeing how your path expression is being applied to an enumerable can be helpful in understanding the path expression syntax. Enumpath has a built-in logger to assist with this. It can be enabled by simply passing `verbose: true` as an option on `Enumpath.apply`. By default this will log debugging information to STDOUT, however you can provide your own logger.
|
|
@@ -391,7 +496,7 @@ Enumpath.logger.logger = ::Logger.new('log/enumpath.log')
|
|
|
391
496
|
|
|
392
497
|
Once enabled, it will log debugging information like so:
|
|
393
498
|
|
|
394
|
-
```
|
|
499
|
+
```plain
|
|
395
500
|
Enumpath.apply('$.store.book', store_info, verbose: true)
|
|
396
501
|
|
|
397
502
|
--------------------------------------
|
|
@@ -428,7 +533,7 @@ You can also control verbose mode via `Enumpath.verbose = true` and `Enumpath.ve
|
|
|
428
533
|
|
|
429
534
|
## Path normalization
|
|
430
535
|
|
|
431
|
-
When you give a string path to Enumpath it will automatically normalize it to an array of path segments. You can also pass it an array of path segments to avoid the normalization, for instance if the normalization process is having trouble parsing your path, or you happen to have a pre-normalized path already. For example the path `['pets']..[?(@.age > 10)].name` is represented in normalized form as `['pets', '..', '?(@.age > 10)', 'name']`. For the most part you should stick with string paths and let Enumpath normalize on its own.
|
|
536
|
+
When you give a string path to Enumpath it will automatically normalize it to an array of path segments. You can also pass it an array of path segments to avoid the normalization, for instance if the normalization process is having trouble parsing your path, or you happen to have a pre-normalized path already. For example the path `['pets']..[?(@.age > 10)].name` is represented in normalized form as `['pets', '..', '?(@.age > 10)', 'name']`. The segments of an array path are converted to strings and the array is used as given, without normalizing or caching it. A path that is neither a String nor an Array raises an `ArgumentError`. For the most part you should stick with string paths and let Enumpath normalize on its own.
|
|
432
537
|
|
|
433
538
|
### Normalized path caching
|
|
434
539
|
|
|
@@ -437,20 +542,52 @@ To save a little bit of time on consecutive calls Enumpath caches the normalized
|
|
|
437
542
|
## Deviations from the Original JSONPath Spec
|
|
438
543
|
|
|
439
544
|
1. The JSONPath spec required that `false` be returned when no matches were found, but Enumpath will return an empty result set (`[]`) instead. This is a thoughtful divergence based on the principle of least astonishment and the robustness principle.
|
|
440
|
-
|
|
441
545
|
2. Enumpath supports relative child indexes, which the original implementation did not support. For instance:
|
|
442
546
|
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
547
|
+
```ruby
|
|
548
|
+
# Get the last element. Both are equivalent to `$..book[-1:]`
|
|
549
|
+
Enumpath.apply('$..book.-1', store_info)
|
|
550
|
+
Enumpath.apply('$..book[-1]', store_info)
|
|
551
|
+
```
|
|
552
|
+
|
|
553
|
+
3. The original implementations of JSONPath allowed unchecked evaluation of filter and subscript expressions. Enumpath limits those expressions to a reasonable subset of operations as detailed in the [Operator Reference](#operator-reference) section. It never uses `eval`: the comparison and arithmetic operators are called by name from a fixed list, and a path can only call methods on your data as the [`:property_resolver`](#property_resolver) option allows (read-only by default).
|
|
554
|
+
4. The original JSONPath spec did not include support for using logical operators to chain expressions in filter expression operators. This addition was inspired by [Gergely Brautigam's][replace-eval-with-object-send-and-a-parser] work on [joshbuddy/jsonpath][jsonpath-gh]
|
|
555
|
+
|
|
556
|
+
## Known limitations
|
|
557
|
+
|
|
558
|
+
These are tracked as issues, and contributions are welcome.
|
|
559
|
+
|
|
560
|
+
- A quoted bracket child cannot contain a dot: `['a.b']` matches nothing ([issue 41](https://github.com/chrisbloom7/enumpath/issues/41)).
|
|
561
|
+
- `&&` and `||` inside a quoted string in a filter expression split the expression ([issue 42](https://github.com/chrisbloom7/enumpath/issues/42)).
|
|
562
|
+
- A key whose value is `nil` returns the same empty result as a missing key ([issue 43](https://github.com/chrisbloom7/enumpath/issues/43)).
|
|
563
|
+
- A negative slice step such as `[::-1]` matches nothing ([issue 44](https://github.com/chrisbloom7/enumpath/issues/44)).
|
|
564
|
+
- Malformed paths are not rejected: `a[` is read as `a` ([issue 45](https://github.com/chrisbloom7/enumpath/issues/45)).
|
|
565
|
+
- Recursive descent raises `SystemStackError` on data nested a few thousand levels deep ([issue 46](https://github.com/chrisbloom7/enumpath/issues/46)).
|
|
566
|
+
- Subscript arithmetic with `*` and `**` can use a lot of CPU or memory, so only accept paths you trust ([issue 52](https://github.com/chrisbloom7/enumpath/issues/52)).
|
|
446
567
|
|
|
447
|
-
|
|
568
|
+
## Alternatives
|
|
448
569
|
|
|
449
|
-
|
|
570
|
+
Enumpath is not the only JSONPath library for Ruby, and for some data it is not the best one:
|
|
571
|
+
|
|
572
|
+
- [`json_p3`][json-p3] follows [RFC 9535][rfc-9535] (the JSONPath standard) and works on parsed JSON only, by design. It is the standards-compliant choice and faster than Enumpath (about 2 to 4 times in a benchmark on a 10,000-element store example).
|
|
573
|
+
- [`jsonpath`][jsonpath-gh] is the long-standing de facto Ruby gem. It was about as fast as Enumpath in the same benchmark.
|
|
574
|
+
- Enumpath is **not** RFC 9535 compliant. It implements the original JSONPath spec with some deviations (see [above](#deviations-from-the-original-jsonpath-spec)).
|
|
575
|
+
|
|
576
|
+
If your data is parsed JSON with string keys, either alternative works and `json_p3` is the standards-compliant, faster choice. Enumpath is for data that is not plain JSON, in particular symbol keys (`jsonpath` loses track of objects nested inside a symbol-keyed Hash) and paths that should only be allowed to read, not call arbitrary methods (see [`:property_resolver`](#property_resolver)). The same queries behave like this across the three gems (checked with `jsonpath` 1.1.5 and `json_p3` 1.0.1 on Ruby 3.4):
|
|
577
|
+
|
|
578
|
+
data | Enumpath | `jsonpath` | `json_p3`
|
|
579
|
+
--- | --- | --- | ---
|
|
580
|
+
Hash and Array with string keys | works | works | works
|
|
581
|
+
Hash with symbol keys | works | no match | no match
|
|
582
|
+
`Struct` | works | works for top-level members; no match once a `Struct` is inside a symbol-keyed Hash | no match
|
|
583
|
+
Plain Ruby object with readers | works (`attr_reader`s under `:safe`, computed readers under `:readers`) | works (it calls any method), except inside a symbol-keyed Hash | no match
|
|
584
|
+
`OpenStruct` | works | works, except inside a symbol-keyed Hash | no match
|
|
585
|
+
|
|
586
|
+
If you only need a fixed path into data you already know, plain `dig` is simpler and much faster (about 50 times faster than Enumpath on a simple path).
|
|
450
587
|
|
|
451
588
|
## Requirements
|
|
452
589
|
|
|
453
|
-
Enumpath requires Ruby 2.
|
|
590
|
+
Enumpath requires Ruby 2.6.0 or higher.
|
|
454
591
|
|
|
455
592
|
## Development
|
|
456
593
|
|
|
@@ -458,35 +595,52 @@ After checking out the repo, run `bin/setup` to install dependencies. Then, run
|
|
|
458
595
|
|
|
459
596
|
## Contributing
|
|
460
597
|
|
|
461
|
-
Bug reports and pull requests are welcome on GitHub at [
|
|
598
|
+
Bug reports and pull requests are welcome on GitHub at [chrisbloom7/enumpath][enumpath-gh]. Please read [CONTRIBUTING.md][contributing] for details on our code of conduct, and the process for submitting pull requests to us.
|
|
462
599
|
|
|
463
600
|
## Versioning
|
|
464
601
|
|
|
465
|
-
We use [SemVer]
|
|
602
|
+
We use [SemVer][semver] for versioning. For the versions available, see the [tags][tags] on this repository.
|
|
466
603
|
|
|
467
604
|
## Authors
|
|
468
605
|
|
|
469
|
-
- [Chris Bloom]
|
|
606
|
+
- [Chris Bloom][chris-bloom-gh]
|
|
470
607
|
|
|
471
|
-
See also the list of [contributors]
|
|
608
|
+
See also the list of [contributors][contributors] who participated in this project.
|
|
472
609
|
|
|
473
610
|
## License
|
|
474
611
|
|
|
475
|
-
Copyright 2018 YouEarnedIt.com
|
|
476
|
-
|
|
477
612
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
|
|
478
613
|
|
|
479
|
-
[www.apache.org/licenses/LICENSE-2.0]
|
|
614
|
+
[www.apache.org/licenses/LICENSE-2.0][apache-license-2-0]
|
|
480
615
|
|
|
481
616
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
|
|
482
617
|
|
|
483
618
|
## Acknowledgements
|
|
484
619
|
|
|
485
|
-
This project
|
|
620
|
+
This project originated from an internal project at [YouEarnedIt][youearnedit], an employee engagement and performance metrics platform, headquartered in Austin, TX.
|
|
486
621
|
|
|
487
622
|
Enumpath is based on [Stefan Goessner's JSONPath spec][jsonpath], and was inspired by several similar libraries:
|
|
488
623
|
|
|
489
|
-
- [nickcharlton/keypath-ruby]
|
|
490
|
-
- [joshbuddy/jsonpath]
|
|
491
|
-
|
|
624
|
+
- [nickcharlton/keypath-ruby][keypath-ruby]
|
|
625
|
+
- [joshbuddy/jsonpath][jsonpath-gh]
|
|
626
|
+
|
|
627
|
+
[apache-license-2-0]: http://www.apache.org/licenses/LICENSE-2.0
|
|
628
|
+
[chris-bloom-gh]: https://github.com/chrisbloom7
|
|
629
|
+
[ci-image]: https://circleci.com/gh/chrisbloom7/enumpath.svg?style=svg
|
|
630
|
+
[ci]: https://circleci.com/gh/chrisbloom7/enumpath
|
|
631
|
+
[contributing]: https://github.com/chrisbloom7/enumpath/blob/main/CONTRIBUTING.md
|
|
632
|
+
[contributors]: https://github.com/chrisbloom7/enumpath/graphs/contributors
|
|
633
|
+
[enumpath-gh]: https://github.com/chrisbloom7/enumpath
|
|
634
|
+
[hound-badge-image]: https://img.shields.io/badge/Reviewed_by-Hound-8E64B0.svg
|
|
635
|
+
[hound]: https://houndci.com
|
|
636
|
+
[jsonpath-gh]: https://github.com/joshbuddy/jsonpath
|
|
637
|
+
[json-p3]: https://github.com/jg-rp/ruby-json-p3
|
|
492
638
|
[jsonpath]: http://goessner.net/articles/JsonPath/
|
|
639
|
+
[keypath-ruby]: https://github.com/nickcharlton/keypath-ruby
|
|
640
|
+
[replace-eval-with-object-send-and-a-parser]: https://skarlso.github.io/2017/05/28/replace-eval-with-object-send-and-a-parser/
|
|
641
|
+
[rfc-9535]: https://www.rfc-editor.org/rfc/rfc9535
|
|
642
|
+
[semver]: http://semver.org/
|
|
643
|
+
[tags]: https://github.com/chrisbloom7/enumpath/tags
|
|
644
|
+
[version-image]: https://badge.fury.io/rb/enumpath.svg
|
|
645
|
+
[version]: https://badge.fury.io/rb/enumpath
|
|
646
|
+
[youearnedit]: http://youearnedit.com
|
data/lib/enumpath/logger.rb
CHANGED
|
@@ -6,7 +6,7 @@ module Enumpath
|
|
|
6
6
|
# A logger for providing debugging information while evaluating path expressions
|
|
7
7
|
# @private
|
|
8
8
|
class Logger
|
|
9
|
-
PAD =
|
|
9
|
+
PAD = ' '
|
|
10
10
|
|
|
11
11
|
SEPARATOR = '--------------------------------------'
|
|
12
12
|
|
|
@@ -31,17 +31,11 @@ module Enumpath
|
|
|
31
31
|
# @yield A lazily evaluated hash of key/value pairs to include in the log message
|
|
32
32
|
def log(title)
|
|
33
33
|
return unless Enumpath.verbose
|
|
34
|
+
|
|
34
35
|
append_log "#{padding}#{SEPARATOR}\n"
|
|
35
36
|
append_log "#{padding}Enumpath: #{title}\n"
|
|
36
|
-
if block_given?
|
|
37
|
-
|
|
38
|
-
vars = yield
|
|
39
|
-
return unless vars.is_a?(Hash)
|
|
40
|
-
label_size = vars.keys.map(&:size).max
|
|
41
|
-
vars.each do |label, value|
|
|
42
|
-
append_log "#{padding}#{label.to_s.ljust(label_size)}: #{massaged_value(value)}\n"
|
|
43
|
-
end
|
|
44
|
-
end
|
|
37
|
+
append_log "#{padding}#{SEPARATOR}\n" if block_given?
|
|
38
|
+
log_vars(yield) if block_given?
|
|
45
39
|
end
|
|
46
40
|
|
|
47
41
|
private
|
|
@@ -50,7 +44,16 @@ module Enumpath
|
|
|
50
44
|
logger << message
|
|
51
45
|
end
|
|
52
46
|
|
|
53
|
-
def
|
|
47
|
+
def log_vars(vars)
|
|
48
|
+
return unless vars.is_a?(Hash)
|
|
49
|
+
|
|
50
|
+
label_size = vars.keys.map(&:size).max
|
|
51
|
+
vars.each do |label, value|
|
|
52
|
+
append_log "#{padding}#{label.to_s.ljust(label_size)}: #{massaged_value(value)}\n"
|
|
53
|
+
end
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
def massaged_value(value) # rubocop:disable Metrics/MethodLength
|
|
54
57
|
if value.is_a?(Enumerable)
|
|
55
58
|
enum_for_log(value)
|
|
56
59
|
elsif value.is_a?(TrueClass)
|
|
@@ -47,7 +47,7 @@ module Enumpath
|
|
|
47
47
|
# @yieldparam enum [Enumerable] the new enum after applying the operator
|
|
48
48
|
# @yieldparam resolved_path [Array] the new resolved_path after applying the operator
|
|
49
49
|
# @yieldreturn [void]
|
|
50
|
-
def apply(remaining_path, enum, resolved_path
|
|
50
|
+
def apply(remaining_path, enum, resolved_path)
|
|
51
51
|
raise NotImplementedError
|
|
52
52
|
end
|
|
53
53
|
|
|
@@ -62,22 +62,37 @@ module Enumpath
|
|
|
62
62
|
private
|
|
63
63
|
|
|
64
64
|
# Returns a set of keys, member names, or indices for a given enumerable. Useful for operators that need to
|
|
65
|
-
# iterate over each member of an enumerable.
|
|
66
|
-
#
|
|
67
|
-
#
|
|
65
|
+
# iterate over each member of an enumerable.
|
|
66
|
+
#
|
|
67
|
+
# - Arrays return their indexes, Hashes their keys, and Structs and Data objects their member names. These are
|
|
68
|
+
# read through Ruby's own implementations, so a subclass that overrides them is not called.
|
|
69
|
+
# - Any other object returns the keys of its `to_h`, but only if the current property resolver policy (see
|
|
70
|
+
# {Enumpath::Resolver::Property}) allows calling `to_h` and it returns a Hash. Under the default `:safe`
|
|
71
|
+
# policy a plain object's own `to_h` is never called, so iterating over it cannot run its code or consume an
|
|
72
|
+
# Enumerator.
|
|
73
|
+
# - Everything else returns an empty array.
|
|
68
74
|
#
|
|
69
75
|
# @param enum [Enumerable] the object to detect keys, member names, or indices on.
|
|
70
76
|
# @return [Array] a set of keys, member names, or indices for the object, or an empty set if they could not be
|
|
71
77
|
# determined.
|
|
72
78
|
def keys(enum)
|
|
73
|
-
|
|
74
|
-
return
|
|
79
|
+
core_keys = core_keys(enum)
|
|
80
|
+
return core_keys unless core_keys.nil?
|
|
75
81
|
|
|
76
|
-
|
|
77
|
-
|
|
82
|
+
hash = Enumpath::Resolver::Property.resolve('to_h', enum)
|
|
83
|
+
hash.is_a?(Hash) ? hash.keys : []
|
|
84
|
+
end
|
|
78
85
|
|
|
79
|
-
|
|
80
|
-
|
|
86
|
+
def core_keys(enum)
|
|
87
|
+
if enum.is_a?(Array)
|
|
88
|
+
(0...Array.instance_method(:length).bind(enum).call).to_a
|
|
89
|
+
elsif enum.is_a?(Hash)
|
|
90
|
+
Hash.instance_method(:keys).bind(enum).call
|
|
91
|
+
elsif enum.is_a?(Struct)
|
|
92
|
+
Struct.instance_method(:members).bind(enum).call
|
|
93
|
+
elsif Enumpath::Resolver::Simple.data?(enum)
|
|
94
|
+
Enumpath::Resolver::Simple.data_members(enum)
|
|
95
|
+
end
|
|
81
96
|
end
|
|
82
97
|
end
|
|
83
98
|
end
|
|
@@ -24,10 +24,10 @@ module Enumpath
|
|
|
24
24
|
# @yieldparam remaining_path [Array] remaining_path
|
|
25
25
|
# @yieldparam enum [Enumerable] the resolved value of the enumerable
|
|
26
26
|
# @yieldparam resolved_path [Array] resolved_path plus the child operator
|
|
27
|
-
def apply(remaining_path, enum, resolved_path
|
|
27
|
+
def apply(remaining_path, enum, resolved_path)
|
|
28
28
|
value = Enumpath::Resolver::Simple.resolve(operator, enum)
|
|
29
29
|
value = Enumpath::Resolver::Property.resolve(operator, enum) if value.nil?
|
|
30
|
-
yield(remaining_path, value, resolved_path + [operator])
|
|
30
|
+
yield(remaining_path, value, resolved_path + [operator]) unless value.nil?
|
|
31
31
|
end
|
|
32
32
|
end
|
|
33
33
|
end
|