enumpath 0.1.2 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 369ec833e0f348946ea91ec684b9cccbc3e53a5bc7bae96e45b262a0988d8631
4
- data.tar.gz: d73ff78feb24cb35c16ed43b0a7d3e8772bcfcd65b56a7a9a273341541b1eca6
3
+ metadata.gz: 4a759aab90deef7e5fe16fb2373738f61d26247efc4d20553740b86c025e1e29
4
+ data.tar.gz: f6dddd9c820e26fdde6eedd8999dae7fd94ab5bf578f33c81f60d56f08b0c559
5
5
  SHA512:
6
- metadata.gz: cbb24d41e81f8c4673e765e28e9dbea07c3fa41a3745357369fea4430f0c83a6d3cbb3eb942784c4177060ad9fe9ddcad4c0ddfcf87ba4eb6cb1854a33775b84
7
- data.tar.gz: 785fb0a118331ba56750117c50c85126c3f5846ba1e403b6376b9fdaf2102267e06e54614f3c0b45eafd34a28295690b0d14357c1ad41f22433b5cc903914f8e
6
+ metadata.gz: 3984dc1111b7435e76868a400685144b7eb7467e1a61b66fe65b1447aac3c82c93c722fa6a8ff1ff84c8c0cef1dfabf15c6f1b7b291a0b678e733ae2fb435be4
7
+ data.tar.gz: 6be4ed326e7dcd4fb9c00d2cefda0ded4efc8ac57d1e0535bc307ca289bb30ad0a705d5d4f4654c955f5d6443c549c1b395bed331a3ddcc4ea68edf1cc1af03c
data/README.md CHANGED
@@ -1,14 +1,18 @@
1
- # Enumpath [![Build Status][ci-image]][ci] [![Gem Version][version-image]][version] [![Reviewed by Hound][hound-badge-image]][hound]
1
+ # Enumpath
2
2
 
3
- A JSONPath-compatible library for safely navigating nested Ruby objects using path expressions.
3
+ Navigate nested Ruby objects with JSONPath-style path expressions, safely by default.
4
4
 
5
5
  ## Introduction
6
6
 
7
- Enumpath is an implementation of the [JSONPath][jsonpath] spec for Ruby objects, plus some added sugar. It's like Ruby's native `Enumerable#dig` method, but fancier. It is designed for situations where you need to provide a dynamic way of describing a complex path through nested enumerable objects. This makes it exceptionally well suited for flexible ETL (Extract, Transform, Load) processes by allowing you to define paths through your data in a simple, easily readable, easily storable syntax.
7
+ Enumpath lets you describe a path through nested Ruby objects as a string, using [JSONPath][jsonpath]-style syntax, and pull out whatever it matches. It works on the data you actually have in memory: hashes with symbol, string or integer keys, arrays, `Struct`s, plain Ruby objects and `OpenStruct`s, in any mix. It is like Ruby's native `dig`, but a path is a plain string that can use wildcards, filters, slices and recursive descent, so it can live in configuration or a database and change without a deploy. (`dig(*keys)` also takes keys at runtime, but only as a fixed chain: it can't say "every item" or "items where the price is over 100".) That makes it a good fit for flexible ETL (Extract, Transform, Load) processes and anywhere else a path should be data instead of code.
8
+
9
+ Paths are safe by default: a path can read your data but cannot call mutating or destructive methods such as `clear` or `destroy`. See [`:property_resolver`](#property_resolver) to allow more. See [Object traversal](#object-traversal) for examples, and [Alternatives](#alternatives) for when another gem is the better choice.
10
+
11
+ Only accept paths from sources you trust. The default resolver stops a path from calling methods that change your data, but a hostile path can still be expensive to evaluate: a regular expression in a filter can take exponential time to match (see [Known limitations](#known-limitations)).
8
12
 
9
13
  Enumpath path expressions look like this:
10
14
 
11
- ```
15
+ ```plain
12
16
  $.pets.cats.0.name
13
17
 
14
18
  $.pets[cats,dogs].*.name
@@ -22,13 +26,13 @@ pets..name
22
26
  pets.cats.-1
23
27
  ```
24
28
 
25
- Enumpath has the following benefits over vanilla `Enumerable#dig`:
29
+ Enumpath has the following benefits over vanilla `dig`:
26
30
 
27
31
  - Paths can be described as simple strings
28
- - It's smart enough to figure out which path segments are integer indexes versus symbolic keys versus string keys
32
+ - It's smart enough to figure out which path segments are integer indexes versus symbolic keys versus string keys versus struct members versus object readers
29
33
  - It enables the use of wildcard, recursive descent, filter, subscript, union, and slice operators to describe complex paths through the data
30
34
 
31
- Like `Enumerable#dig`, Enumpath protects against missing path segments and returns safely if the full path cannot be resolved.
35
+ Like `dig`, Enumpath protects against missing path segments and returns safely if the full path cannot be resolved.
32
36
 
33
37
  ## Installation
34
38
 
@@ -40,11 +44,15 @@ gem 'enumpath'
40
44
 
41
45
  And then execute:
42
46
 
43
- $ bundle
47
+ ```shell
48
+ bundle
49
+ ```
44
50
 
45
51
  Or install it yourself as:
46
52
 
47
- $ gem install enumpath
53
+ ```shell
54
+ gem install enumpath
55
+ ```
48
56
 
49
57
  ## Usage
50
58
 
@@ -72,6 +80,64 @@ Enumpath.apply("drinks.*", party) # => []
72
80
 
73
81
  > This is a thoughtful deviation from the original JSONPath spec which would return `false` on no matches.
74
82
 
83
+ ## Object traversal
84
+
85
+ The same path syntax works on whatever shape your data has. These examples are mirrored by `spec/integration_tests/object_traversal_spec.rb`, so they stay accurate.
86
+
87
+ A hash with symbol keys, using a wildcard and a filter:
88
+
89
+ ```ruby
90
+ order = { customer: { name: 'Ada' }, items: [{ sku: 'A1', qty: 2 }, { sku: 'B2', qty: 1 }] }
91
+ Enumpath.apply('items.*.sku', order) # => ["A1", "B2"]
92
+ Enumpath.apply('items[?(@.qty > 1)].sku', order) # => ["A1"]
93
+ ```
94
+
95
+ A `Struct` inside a hash, with a relative index:
96
+
97
+ ```ruby
98
+ Point = Struct.new(:x, :y)
99
+ route = { stops: [Point.new(1, 2), Point.new(3, 4)] }
100
+ Enumpath.apply('stops.*.x', route) # => [1, 3]
101
+ Enumpath.apply('stops.-1.y', route) # => [4]
102
+ ```
103
+
104
+ A plain Ruby object. Under the default `:safe` policy its `attr_reader`s resolve, and so do read-only methods such as `length` on the values they return:
105
+
106
+ ```ruby
107
+ class Author
108
+ attr_reader :name, :books
109
+
110
+ def initialize(name, books)
111
+ @name = name
112
+ @books = books
113
+ end
114
+
115
+ def label
116
+ "#{name} (#{books.length} books)"
117
+ end
118
+ end
119
+
120
+ library = { authors: [Author.new('Le Guin', %w[Earthsea Dispossessed]), Author.new('Butler', %w[Kindred])] }
121
+ Enumpath.apply('authors.*.name', library) # => ["Le Guin", "Butler"]
122
+ Enumpath.apply('authors.0.books.length', library) # => [2]
123
+ ```
124
+
125
+ A computed reader such as `label` is a method you wrote, not an `attr_reader`, so `:safe` does not call it. Ask for it with `property_resolver: :readers`, which also runs methods from installed gems and is not read-only, so use it on data you trust:
126
+
127
+ ```ruby
128
+ Enumpath.apply('authors.*.label', library) # => []
129
+ Enumpath.apply('authors.*.label', library, property_resolver: :readers)
130
+ # => ["Le Guin (2 books)", "Butler (1 books)"]
131
+ ```
132
+
133
+ An `OpenStruct`:
134
+
135
+ ```ruby
136
+ require 'ostruct'
137
+ config = OpenStruct.new(db: OpenStruct.new(host: 'localhost', port: 5432))
138
+ Enumpath.apply('db.host', config) # => ["localhost"]
139
+ ```
140
+
75
141
  ## Operator Reference
76
142
 
77
143
  Enumpath currently implements the following path operators:
@@ -96,11 +162,11 @@ Child operators match on an index, key, member, or property of the enumerable. I
96
162
  1. as an integer key or index (if the segment is integer-like),
97
163
  2. then as a string key,
98
164
  3. then as a symbol key,
99
- 4. and finally as a public property (i.e. a public method of the target that expects no arguments)
165
+ 4. and finally as a public property (i.e. a public method of the target that expects no arguments), if the [`:property_resolver`](#property_resolver) option allows it. By default (`:safe`) only read-only access is allowed: allowlisted methods such as `length`, `size`, `count`, `first`, `keys` or `upcase` on core data (arrays, hashes, strings, symbols, numbers, structs, `Data` objects, `nil`, `true`, `false`), and `attr_reader`s on plain Ruby objects. A path segment like `clear`, `pop`, `sort!` or `destroy` matches nothing. Use `property_resolver: :readers` to also allow computed readers and ActiveRecord attributes (this also runs methods from installed gems and is not read-only; see [`:property_resolver`](#property_resolver)).
100
166
 
101
- #### Examples
167
+ #### Operator Examples
102
168
 
103
- ```
169
+ ```ruby
104
170
  Car = Struct.new(:color, :transmition, :owners)
105
171
  hyundai = Car.new('blue', :automatic, [{ name: 'Bill' }, { name: 'Ted' }])
106
172
  subaru = Car.new('gold', :standard, [{ name: 'Kate' }])
@@ -141,7 +207,7 @@ The operator accepts a mixed bag of argument combinations. For instance, these a
141
207
 
142
208
  Syntax: `[child1,child2,...]`
143
209
 
144
- The union operator combines the results of two or more child operators. There is no limit to the number of child operators you can specify in a single union. Each child operator is separated by a comma (`,`). White space is stripped from around each child operator. Child operators can optionally be wrapped in single quotes. Bracket notation is not supporter in this context.
210
+ The union operator combines the results of two or more child operators. There is no limit to the number of child operators you can specify in a single union. Each child operator is separated by a comma (`,`). White space is stripped from around each child operator. Child operators can optionally be wrapped in single quotes. Bracket notation is not supported in this context.
145
211
 
146
212
  The following are all valid union operators:
147
213
 
@@ -161,7 +227,7 @@ The following are all valid filter expressions:
161
227
  - `[?(isbn)]`: equivalent to the previous example
162
228
  - `[?(@.price == 8)]`: any member whose `price` value is equal to 8
163
229
  - `[?(@.price == 8 || @.price == 10)]`: members with a `price` of 8 _or_ 10
164
- - `[?(@.price > 2 && @.price < 10)]`: members with a `price` greater than 8 _and_ less than 10
230
+ - `[?(@.price > 2 && @.price < 10)]`: members with a `price` greater than 2 _and_ less than 10
165
231
  - `[?(@.name =~ /bob/i || @.name == 'Mark')]`: any member whose `name` matches the regex `/bob/i` or equals `'Mark'`
166
232
 
167
233
  > Regular expression operands are safely parsed using the `to_regexp` gem
@@ -170,15 +236,24 @@ The following are all valid filter expressions:
170
236
 
171
237
  Syntax: `[(expression)]`
172
238
 
173
- A subscript expression is made up of a singe expression that consists of a child operator (`@.child` or `child`; the leading `@.` is optional), plus an optional pair of arithmetic operator and operand. The arithmetic operator can be any one of `+`, `-`, `**`, `*`, `/`, or `%`. The operand can be a string (`'some string'`), symbol (`:some_symbol`), or numeric value (`10` or `1.0`). The expression is evaluated and the result becomes the subscript. If an operator and operand are not included in an expression then the value located at the child operator is used as the subscript. If the subscript represents a valid child path for the enumerable, the value of that member will be passed along for further processing of the path.
239
+ A subscript expression is made up of a single expression that consists of a child operator (`@.child` or `child`; the leading `@.` is optional), plus an optional pair of arithmetic operator and operand. The arithmetic operator can be any one of `+`, `-`, `**`, `*`, `/`, or `%`. The operand can be a string (`'some string'`), symbol (`:some_symbol`), or numeric value (`10` or `1.0`). The expression is evaluated and the result becomes the subscript. If an operator and operand are not included in an expression then the value located at the child operator is used as the subscript. If the subscript represents a valid child path for the enumerable, the value of that member will be passed along for further processing of the path.
174
240
 
175
- The following are all valid filter expressions:
241
+ The following are all valid subscript expressions:
176
242
 
177
243
  - `[(@.length - 1)]`: the subscript becomes the length of the current enumerable, minus 1
178
244
  - `[(length / 2)]`: the subscript becomes the index at half the length of the enumerable
179
245
  - `[(@.type)]`: the subscript becomes the value at the `type` key, member, or property of the enumerable
180
246
 
181
- ## Examples
247
+ An expression that cannot be evaluated against the data, such as a division by zero or a string plus a number, matches nothing instead of raising an error. So does an expression that could use an unbounded amount of CPU or memory, because the operand comes from the path. The limits are:
248
+
249
+ - an integer operand must fit in 64 bits,
250
+ - repeating a string or array with `*` may produce at most 10,000 characters or elements,
251
+ - an integer raised to a power with `**` must produce a result of at most 1,024 bits, and
252
+ - `%` only applies to numbers (on a string it would be `format`, which can build a huge string).
253
+
254
+ These are far above anything useful for an index or a key.
255
+
256
+ ## Usage Examples
182
257
 
183
258
  Given the same store example from the JSONPath project:
184
259
 
@@ -379,6 +454,45 @@ Enumpath.apply("food.*", party, result_type: :path) # => ["$['food'][0]", "$['fo
379
454
 
380
455
  Each returned path is a valid path expression that can be used in calls to `Enumpath.apply`. If you want to be explicit about returning values instead of paths you can specify that with the option `result_type: :value`.
381
456
 
457
+ ### :property_resolver
458
+
459
+ When a path segment is not an index, key, or struct or `Data` member, Enumpath can resolve it by calling a method on the object (step 4 of the [child operator](#child-operator)). Wildcards, filters and recursive descent also use this option when they need the keys of an object that is not an array, hash, struct or `Data` object (they call its `to_h` only if the option allows it). The `:property_resolver` option controls what a path is allowed to call:
460
+
461
+ value | what resolves | read-only?
462
+ --- | --- | ---
463
+ `:safe` (default) | allowlisted read-only methods on core data, `attr_reader`s on plain objects | yes
464
+ `:readers` | `:safe`, plus public no-argument methods written in Ruby, including installed gems (Ruby's built-in and standard library methods excluded), and ActiveModel-style `attributes` | no
465
+ `:none` | nothing; only indexes, keys and members | yes
466
+ `:all` | any public method that expects no arguments (the behavior before this option existed) | no
467
+ a callable | whatever the callable allows | up to you
468
+
469
+ - `:safe` (the default): read-only access. On core data (arrays, hashes, strings, symbols, numbers, structs, `Data` objects, `nil`, `true`, `false`) Enumpath calls a fixed allowlist of non-mutating methods such as `length`, `size`, `count`, `first`, `last`, `min`, `max`, `sum`, `keys`, `values`, `to_a`, `to_h`, `to_s`, `empty?`, `zero?`, `upcase` or `strip`, and only when the receiver's class does not override them. On plain Ruby objects it reads instance variables that have a public reader of the same name (`attr_reader`s); the instance variable is read directly, so the method itself never runs. Names ending in `!` or `=` are never called, and neither is any method that requires arguments.
470
+ - `:readers`: everything `:safe` allows, plus, on objects that are not core data, any public method that expects no arguments and is written in Ruby, except Ruby's built-in methods, methods in the Ruby standard library directory, and a short denylist of persistence verbs (`destroy`, `delete`, `save`, `update`, `reload`, `touch`, `delete_all`, `destroy_all`, `toggle`). It also reads the keys of an ActiveModel-style `attributes` Hash. So computed readers like `full_name`, `attr_reader`s and ActiveRecord columns like `email` work. Methods implemented in C (core classes, `StringIO`, C-extension gems) are never called; for objects of C-extension classes use a callable or `:all`. Core data is treated exactly like `:safe`. **This is not read-only.** It includes methods from installed gems, including default and bundled gems such as Logger and CSV, and ActiveModel or ActiveRecord: for example `Logger#close` would run, and so would a model method like `archive` or `cancel`. The denylist is a safety net, not a guarantee. Use `:readers` for data you trust or objects you built; for untrusted paths use `:safe` or a callable.
471
+ - `:none`: never call methods.
472
+ - `:all`: call any public method that expects no arguments. This is how Enumpath behaved before this option existed, and it lets a path run methods such as `clear`, `pop`, `sort!` or `destroy`. Only use it with paths you trust.
473
+ - a callable, such as a lambda: called with the object and the method name (a String). The method is called only if it returns true. The callable has full control: none of the checks above are applied, so call `Enumpath::Resolver::Property.safe?(object, name)` or `.reader?(object, name)` from it if you want to keep them and allow more.
474
+
475
+ ```ruby
476
+ party = { food: %w[pizza tacos] }
477
+ Enumpath.apply('food.pop', party) # => [] and party is unchanged
478
+ Enumpath.apply('food.length', party) # => [2]
479
+ Enumpath.apply('food.length', party, property_resolver: :none) # => []
480
+
481
+ # ActiveRecord models you trust: computed readers and columns
482
+ Enumpath.apply('[?(@.published)].title', Post.all.to_a, property_resolver: :readers)
483
+
484
+ # Recommended for objects you do not control: keep the safe defaults and allow specific methods per class
485
+ ALLOWED = { 'Invoice' => %w[total due_on], 'Customer' => %w[name email] }.freeze
486
+ policy = lambda do |object, name|
487
+ Enumpath::Resolver::Property.safe?(object, name) || ALLOWED.fetch(object.class.name, []).include?(name)
488
+ end
489
+ Enumpath.apply('invoices.*.total', account, property_resolver: policy)
490
+ ```
491
+
492
+ The option applies to a single call. A chained `results.apply(...)` uses the default unless you pass the option again. Any other value raises an `ArgumentError`.
493
+
494
+ To decide whether an object has a method, Enumpath asks Ruby (`respond_to?` and `method`), which calls the object's `respond_to_missing?` if it defines one. So even under `:safe`, a path can run a little of your application's code that way.
495
+
382
496
  ### :verbose
383
497
 
384
498
  Seeing how your path expression is being applied to an enumerable can be helpful in understanding the path expression syntax. Enumpath has a built-in logger to assist with this. It can be enabled by simply passing `verbose: true` as an option on `Enumpath.apply`. By default this will log debugging information to STDOUT, however you can provide your own logger.
@@ -391,7 +505,7 @@ Enumpath.logger.logger = ::Logger.new('log/enumpath.log')
391
505
 
392
506
  Once enabled, it will log debugging information like so:
393
507
 
394
- ```
508
+ ```plain
395
509
  Enumpath.apply('$.store.book', store_info, verbose: true)
396
510
 
397
511
  --------------------------------------
@@ -428,7 +542,7 @@ You can also control verbose mode via `Enumpath.verbose = true` and `Enumpath.ve
428
542
 
429
543
  ## Path normalization
430
544
 
431
- When you give a string path to Enumpath it will automatically normalize it to an array of path segments. You can also pass it an array of path segments to avoid the normalization, for instance if the normalization process is having trouble parsing your path, or you happen to have a pre-normalized path already. For example the path `['pets']..[?(@.age > 10)].name` is represented in normalized form as `['pets', '..', '?(@.age > 10)', 'name']`. For the most part you should stick with string paths and let Enumpath normalize on its own.
545
+ When you give a string path to Enumpath it will automatically normalize it to an array of path segments. You can also pass it an array of path segments to avoid the normalization, for instance if the normalization process is having trouble parsing your path, or you happen to have a pre-normalized path already. For example the path `['pets']..[?(@.age > 10)].name` is represented in normalized form as `['pets', '..', '?(@.age > 10)', 'name']`. The segments of an array path are converted to strings and the array is used as given, without normalizing or caching it. A path that is neither a String nor an Array raises an `ArgumentError`. For the most part you should stick with string paths and let Enumpath normalize on its own.
432
546
 
433
547
  ### Normalized path caching
434
548
 
@@ -437,20 +551,52 @@ To save a little bit of time on consecutive calls Enumpath caches the normalized
437
551
  ## Deviations from the Original JSONPath Spec
438
552
 
439
553
  1. The JSONPath spec required that `false` be returned when no matches were found, but Enumpath will return an empty result set (`[]`) instead. This is a thoughtful divergence based on the principle of least astonishment and the robustness principle.
440
-
441
554
  2. Enumpath supports relative child indexes, which the original implementation did not support. For instance:
442
555
 
443
- # Get the last element. Both are equivalent to `$..book[-1:]`
444
- Enumpath.apply('$..book.-1', store_info)
445
- Enumpath.apply('$..book[-1]', store_info)
446
-
447
- 3. The original implementations of JSONPath allowed unchecked evaluation of filter and subscript expressions. Enumpath limits those expressions to a reasonable subset of operations as detailed in the [Operator Reference](#operator-reference) section and uses `public_send` rather than `eval` to resolve expressions as necessary.
556
+ ```ruby
557
+ # Get the last element. Both are equivalent to `$..book[-1:]`
558
+ Enumpath.apply('$..book.-1', store_info)
559
+ Enumpath.apply('$..book[-1]', store_info)
560
+ ```
448
561
 
562
+ 3. The original implementations of JSONPath allowed unchecked evaluation of filter and subscript expressions. Enumpath limits those expressions to a reasonable subset of operations as detailed in the [Operator Reference](#operator-reference) section. It never uses `eval`: the comparison and arithmetic operators are called by name from a fixed list, and a path can only call methods on your data as the [`:property_resolver`](#property_resolver) option allows (read-only by default).
449
563
  4. The original JSONPath spec did not include support for using logical operators to chain expressions in filter expression operators. This addition was inspired by [Gergely Brautigam's][replace-eval-with-object-send-and-a-parser] work on [joshbuddy/jsonpath][jsonpath-gh]
450
564
 
565
+ ## Known limitations
566
+
567
+ These are tracked as issues, and contributions are welcome.
568
+
569
+ - A quoted bracket child cannot contain a dot: `['a.b']` matches nothing ([issue 41](https://github.com/chrisbloom7/enumpath/issues/41)).
570
+ - `&&` and `||` inside a quoted string in a filter expression split the expression ([issue 42](https://github.com/chrisbloom7/enumpath/issues/42)).
571
+ - A key whose value is `nil` returns the same empty result as a missing key ([issue 43](https://github.com/chrisbloom7/enumpath/issues/43)).
572
+ - A negative slice step such as `[::-1]` matches nothing ([issue 44](https://github.com/chrisbloom7/enumpath/issues/44)).
573
+ - Malformed paths are not rejected: `a[` is read as `a` ([issue 45](https://github.com/chrisbloom7/enumpath/issues/45)).
574
+ - Recursive descent raises `SystemStackError` on data nested a few thousand levels deep ([issue 46](https://github.com/chrisbloom7/enumpath/issues/46)).
575
+ - A regular expression in a filter can take exponential time to match, so only accept paths you trust ([issue 60](https://github.com/chrisbloom7/enumpath/issues/60)).
576
+
577
+ ## Alternatives
578
+
579
+ Enumpath is not the only JSONPath library for Ruby, and for some data it is not the best one:
580
+
581
+ - [`json_p3`][json-p3] follows [RFC 9535][rfc-9535] (the JSONPath standard) and works on parsed JSON only, by design. It is the standards-compliant choice and faster than Enumpath (about 2 to 4 times in a benchmark on a 10,000-element store example).
582
+ - [`jsonpath`][jsonpath-gh] is the long-standing de facto Ruby gem. It was about as fast as Enumpath in the same benchmark.
583
+ - Enumpath is **not** RFC 9535 compliant. It implements the original JSONPath spec with some deviations (see [above](#deviations-from-the-original-jsonpath-spec)).
584
+
585
+ If your data is parsed JSON with string keys, either alternative works and `json_p3` is the standards-compliant, faster choice. Enumpath is for data that is not plain JSON, in particular symbol keys (`jsonpath` loses track of objects nested inside a symbol-keyed Hash) and paths that should only be allowed to read, not call arbitrary methods (see [`:property_resolver`](#property_resolver)). The same queries behave like this across the three gems (checked with `jsonpath` 1.1.5 and `json_p3` 1.0.1 on Ruby 3.4):
586
+
587
+ data | Enumpath | `jsonpath` | `json_p3`
588
+ --- | --- | --- | ---
589
+ Hash and Array with string keys | works | works | works
590
+ Hash with symbol keys | works | no match | no match
591
+ `Struct` | works | works for top-level members; no match once a `Struct` is inside a symbol-keyed Hash | no match
592
+ Plain Ruby object with readers | works (`attr_reader`s under `:safe`, computed readers under `:readers`) | works (it calls any method), except inside a symbol-keyed Hash | no match
593
+ `OpenStruct` | works | works, except inside a symbol-keyed Hash | no match
594
+
595
+ If you only need a fixed path into data you already know, plain `dig` is simpler and much faster (about 50 times faster than Enumpath on a simple path).
596
+
451
597
  ## Requirements
452
598
 
453
- Enumpath requires Ruby 2.3.0 or higher.
599
+ Enumpath requires Ruby 2.6.0 or higher.
454
600
 
455
601
  ## Development
456
602
 
@@ -458,7 +604,7 @@ After checking out the repo, run `bin/setup` to install dependencies. Then, run
458
604
 
459
605
  ## Contributing
460
606
 
461
- Bug reports and pull requests are welcome on GitHub at [chrisbloom7/enumpath][enumpath-gh]. Please read [CONTRIBUTING.md]() for details on our code of conduct, and the process for submitting pull requests to us.
607
+ Bug reports and pull requests are welcome on GitHub at [chrisbloom7/enumpath][enumpath-gh]. Please read [CONTRIBUTING.md][contributing] for details on our code of conduct, and the process for submitting pull requests to us.
462
608
 
463
609
  ## Versioning
464
610
 
@@ -489,18 +635,21 @@ Enumpath is based on [Stefan Goessner's JSONPath spec][jsonpath], and was inspir
489
635
 
490
636
  [apache-license-2-0]: http://www.apache.org/licenses/LICENSE-2.0
491
637
  [chris-bloom-gh]: https://github.com/chrisbloom7
492
- [ci]: https://circleci.com/gh/chrisbloom7/enumpath
493
638
  [ci-image]: https://circleci.com/gh/chrisbloom7/enumpath.svg?style=svg
639
+ [ci]: https://circleci.com/gh/chrisbloom7/enumpath
640
+ [contributing]: https://github.com/chrisbloom7/enumpath/blob/main/CONTRIBUTING.md
494
641
  [contributors]: https://github.com/chrisbloom7/enumpath/graphs/contributors
495
642
  [enumpath-gh]: https://github.com/chrisbloom7/enumpath
496
- [hound]: https://houndci.com
497
643
  [hound-badge-image]: https://img.shields.io/badge/Reviewed_by-Hound-8E64B0.svg
498
- [jsonpath]: http://goessner.net/articles/JsonPath/
644
+ [hound]: https://houndci.com
499
645
  [jsonpath-gh]: https://github.com/joshbuddy/jsonpath
646
+ [json-p3]: https://github.com/jg-rp/ruby-json-p3
647
+ [jsonpath]: http://goessner.net/articles/JsonPath/
500
648
  [keypath-ruby]: https://github.com/nickcharlton/keypath-ruby
501
649
  [replace-eval-with-object-send-and-a-parser]: https://skarlso.github.io/2017/05/28/replace-eval-with-object-send-and-a-parser/
650
+ [rfc-9535]: https://www.rfc-editor.org/rfc/rfc9535
502
651
  [semver]: http://semver.org/
503
652
  [tags]: https://github.com/chrisbloom7/enumpath/tags
504
- [version]: https://badge.fury.io/rb/enumpath
505
653
  [version-image]: https://badge.fury.io/rb/enumpath.svg
654
+ [version]: https://badge.fury.io/rb/enumpath
506
655
  [youearnedit]: http://youearnedit.com
@@ -16,7 +16,7 @@ module Enumpath
16
16
  # @param enum [Enumerable] an enum that can be used to assist in detection. Not all subclasses require an enum
17
17
  # for detection.
18
18
  # @return [true, false] whether the operator param appears to represent the operator class
19
- def detect?(_operator, _enum = nil)
19
+ def detect?(operator, enum = nil)
20
20
  raise NotImplementedError
21
21
  end
22
22
  end
@@ -47,7 +47,7 @@ module Enumpath
47
47
  # @yieldparam enum [Enumerable] the new enum after applying the operator
48
48
  # @yieldparam resolved_path [Array] the new resolved_path after applying the operator
49
49
  # @yieldreturn [void]
50
- def apply(_remaining_path, _enum, _resolved_path)
50
+ def apply(remaining_path, enum, resolved_path)
51
51
  raise NotImplementedError
52
52
  end
53
53
 
@@ -62,22 +62,37 @@ module Enumpath
62
62
  private
63
63
 
64
64
  # Returns a set of keys, member names, or indices for a given enumerable. Useful for operators that need to
65
- # iterate over each member of an enumerable. If the object is an array then it will return an array of indexes.
66
- # If the object can be converted to a hash (Hash, Struct, or anything responding to {#to_h}) then it will be
67
- # forced to a hash and the hash keys will be returned. For all other objects, an empty array will be returned.
65
+ # iterate over each member of an enumerable.
66
+ #
67
+ # - Arrays return their indexes, Hashes their keys, and Structs and Data objects their member names. These are
68
+ # read through Ruby's own implementations, so a subclass that overrides them is not called.
69
+ # - Any other object returns the keys of its `to_h`, but only if the current property resolver policy (see
70
+ # {Enumpath::Resolver::Property}) allows calling `to_h` and it returns a Hash. Under the default `:safe`
71
+ # policy a plain object's own `to_h` is never called, so iterating over it cannot run its code or consume an
72
+ # Enumerator.
73
+ # - Everything else returns an empty array.
68
74
  #
69
75
  # @param enum [Enumerable] the object to detect keys, member names, or indices on.
70
76
  # @return [Array] a set of keys, member names, or indices for the object, or an empty set if they could not be
71
77
  # determined.
72
78
  def keys(enum)
73
- # Arrays
74
- return (0...enum.length).to_a if enum.is_a?(Array)
79
+ core_keys = core_keys(enum)
80
+ return core_keys unless core_keys.nil?
75
81
 
76
- # Other Enumerables
77
- return enum.to_h.keys if enum.respond_to?(:to_h)
82
+ hash = Enumpath::Resolver::Property.resolve('to_h', enum)
83
+ hash.is_a?(Hash) ? hash.keys : []
84
+ end
78
85
 
79
- # Fallback
80
- []
86
+ def core_keys(enum)
87
+ if enum.is_a?(Array)
88
+ (0...Array.instance_method(:length).bind(enum).call).to_a
89
+ elsif enum.is_a?(Hash)
90
+ Hash.instance_method(:keys).bind(enum).call
91
+ elsif enum.is_a?(Struct)
92
+ Struct.instance_method(:members).bind(enum).call
93
+ elsif Enumpath::Resolver::Simple.data?(enum)
94
+ Enumpath::Resolver::Simple.data_members(enum)
95
+ end
81
96
  end
82
97
  end
83
98
  end
@@ -8,6 +8,21 @@ module Enumpath
8
8
  ARITHMETIC_OPERATOR_REGEX = %r{(\+|-|\*\*|\*|\/|%)}
9
9
  OPERATOR_REGEX = /^\((.*)\)$/
10
10
 
11
+ # The operand of an arithmetic expression comes from the path, so an expression that could use an unbounded
12
+ # amount of CPU or memory is not evaluated and matches nothing. A subscript is an index or a key, so none of
13
+ # these limits get in the way of a useful expression.
14
+ #
15
+ # The widest integer operand allowed, in bits
16
+ MAX_OPERAND_BITS = 64
17
+ # The most elements or characters that repeating a string or array (`*`) may produce
18
+ MAX_REPEAT_SIZE = 10_000
19
+ # The widest result allowed for an integer raised to a power (`**`), in bits
20
+ MAX_POWER_BITS = 1_024
21
+
22
+ # Errors from evaluating an expression against data that does not fit it, such as a string plus a number or a
23
+ # division by zero. The expression matches nothing in these cases.
24
+ EVALUATION_ERRORS = [NoMethodError, TypeError, ArgumentError, ZeroDivisionError, RangeError].freeze
25
+
11
26
  class << self
12
27
  # Whether the operator matches {Enumpath::Operator::SubscriptExpression::OPERATOR_REGEX}
13
28
  #
@@ -58,13 +73,18 @@ module Enumpath
58
73
  else
59
74
  evaluate_with_operator(operator, operand, value)
60
75
  end
61
- rescue NoMethodError
76
+ rescue *EVALUATION_ERRORS
62
77
  Enumpath.log('Subscript could not be evaluated') { { subscript: nil } }
63
78
  nil
64
79
  end
65
80
 
66
81
  def evaluate_with_operator(operator, operand, value)
67
82
  typecast_operand = variable_typecaster(operand)
83
+ unless within_limits?(operator, typecast_operand, value)
84
+ Enumpath.log('Subscript exceeds the arithmetic limits') { { operator: operator, operand: typecast_operand } }
85
+ return nil
86
+ end
87
+
68
88
  result = value.public_send(operator.to_sym, typecast_operand)
69
89
  Enumpath.log('Evaluated subscript') do
70
90
  { value: value, operator: operator, operand: typecast_operand, result: result }
@@ -72,6 +92,35 @@ module Enumpath
72
92
  result
73
93
  end
74
94
 
95
+ # Whether evaluating `value <operator> operand` stays inside the limits above, before any work is done
96
+ def within_limits?(operator, operand, value)
97
+ return false if operand.is_a?(Integer) && operand.abs.bit_length > MAX_OPERAND_BITS
98
+
99
+ case operator
100
+ when '*' then repetition_within_limit?(value, operand)
101
+ when '**' then power_within_limit?(value, operand)
102
+ when '%' then value.is_a?(Numeric) # `String#%` is `format`, which can build a huge string
103
+ else true
104
+ end
105
+ end
106
+
107
+ # Repeating a string or array by an integer multiplies its size. (`Array#*` with a string joins instead.)
108
+ def repetition_within_limit?(value, operand)
109
+ return true unless operand.is_a?(Integer) && (value.is_a?(String) || value.is_a?(Array))
110
+
111
+ operand >= 0 && value.size * operand <= MAX_REPEAT_SIZE
112
+ end
113
+
114
+ # A float overflows to Infinity at once. An integer power is only allowed if its result fits in MAX_POWER_BITS,
115
+ # estimated as the bit length of the base times the exponent. Any other kind of number is not allowed.
116
+ def power_within_limit?(value, operand)
117
+ case value
118
+ when Integer then operand.is_a?(Integer) && value.abs.bit_length * operand.abs <= MAX_POWER_BITS
119
+ when Float then true
120
+ else false
121
+ end
122
+ end
123
+
75
124
  def variable_typecaster(variable)
76
125
  if variable =~ /\A('|").+\1\z/ || variable =~ /^:.+/
77
126
  # It quacks like a string or symbol
data/lib/enumpath/path.rb CHANGED
@@ -11,7 +11,8 @@ module Enumpath
11
11
  # @return [Enumpath::Results] the current results array
12
12
  attr_reader :results
13
13
 
14
- # @param path [String, Array<String>] the path expression to apply to the enumerable
14
+ # @param path [String, Array<String>] the path expression to apply to the enumerable, or an array of path segments
15
+ # @raise [ArgumentError] if the path is neither a String nor an Array
15
16
  # @param result_type (see Enumpath::Results#initialize)
16
17
  def initialize(path, result_type: nil)
17
18
  @path = path
@@ -57,7 +58,7 @@ module Enumpath
57
58
 
58
59
  def apply_segments(path_segments, enum, resolved_path, nesting_level)
59
60
  segment = path_segments.first
60
- remaining_path = path_segments[1..-1]
61
+ remaining_path = path_segments[1..]
61
62
  operator = Enumpath::Operator.detect(segment, enum)
62
63
  operator&.apply(remaining_path, enum, resolved_path) do |s, e, c|
63
64
  trace(s, e, c, nesting_level + 1)
@@ -68,8 +69,14 @@ module Enumpath
68
69
  @cache ||= Enumpath.path_cache
69
70
  end
70
71
 
72
+ # String paths are normalized once and cached. An array is already a list of path segments, so it only has its
73
+ # segments converted to strings and never touches the cache.
71
74
  def normalize!
72
- @path = cache.get_or_set(@path) { Enumpath::Path::NormalizedPath.new(@path) }
75
+ @path = case @path
76
+ when String then cache.get_or_set(@path) { Enumpath::Path::NormalizedPath.new(@path) }
77
+ when Array then Enumpath::Path::NormalizedPath.new(@path.map(&:to_s))
78
+ else raise ArgumentError, 'path must be a String or an Array of path segments'
79
+ end
73
80
  end
74
81
  end
75
82
  end
@@ -0,0 +1,70 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'rbconfig'
4
+
5
+ module Enumpath
6
+ module Resolver
7
+ # Helpers for inspecting an object without trusting it. Kernel methods are called through
8
+ # `Kernel.instance_method(...).bind`, so an object that overrides `is_a?`, `respond_to?`, `method` or the instance
9
+ # variable methods cannot change the answer.
10
+ #
11
+ # @private
12
+ module ObjectInspector
13
+ # The Kernel methods {.kernel} can call
14
+ KERNEL = %i[is_a? respond_to? method instance_variable_defined? instance_variable_get].map do |name|
15
+ [name, Kernel.instance_method(name)]
16
+ end.to_h.freeze
17
+
18
+ # Ruby's standard library directory (`RbConfig::CONFIG['rubylibdir']`). Installed gems, including default and
19
+ # bundled gems such as Logger or CSV, live elsewhere and are not covered by it.
20
+ RUBY_LIBRARY_DIR = File.join(RbConfig::CONFIG['rubylibdir'], '')
21
+
22
+ module_function
23
+
24
+ # Calls one of the {KERNEL} methods on an object
25
+ #
26
+ # @raise [TypeError] if the object does not include Kernel (a BasicObject)
27
+ def kernel(object, method_name, *args)
28
+ KERNEL[method_name].bind(object).call(*args)
29
+ end
30
+
31
+ # The public method of that name, if the object has one that expects no arguments
32
+ #
33
+ # @return [Method, nil]
34
+ def public_method_without_required_args(object, name)
35
+ return unless kernel(object, :respond_to?, name)
36
+
37
+ method = kernel(object, :method, name)
38
+ method unless method.parameters.any? { |type, _| %i[req keyreq].include?(type) }
39
+ rescue NameError, TypeError
40
+ nil
41
+ end
42
+
43
+ # Whether `:readers` may call a method: it is written in Ruby (it has a source location, so it is not
44
+ # implemented in C), it is not one of Ruby's built-in methods (`<internal:...>`), and it is not in the Ruby
45
+ # standard library directory ({RUBY_LIBRARY_DIR}). Methods from installed gems pass this check, including
46
+ # default and bundled gems such as Logger and CSV, and so do attr_readers and methods made with
47
+ # `define_method` or `class_eval`.
48
+ #
49
+ # @param method [Method, nil]
50
+ # @return [true, false]
51
+ def application_method?(method)
52
+ location = method&.source_location
53
+ return false if location.nil? || location.empty?
54
+
55
+ file = location.first.to_s
56
+ !file.start_with?('<internal:') && !file.start_with?(RUBY_LIBRARY_DIR)
57
+ end
58
+
59
+ # Whether the object is an instance of any of the classes. Objects that do not include Kernel count as a match,
60
+ # so callers treat them as off limits.
61
+ #
62
+ # @return [true, false]
63
+ def instance_of_any?(object, classes)
64
+ classes.any? { |klass| kernel(object, :is_a?, klass) }
65
+ rescue TypeError
66
+ true
67
+ end
68
+ end
69
+ end
70
+ end