janeway-jsonpath 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 23479bd7bd49f2ac64ce5e8f27399df191efd5c7efe76aaca4fd5615cf429319
-  data.tar.gz: b1dea6ae594f5ab51f0105fb32f81ad03a5028223214f38e01c05cae0fad011b
+  metadata.gz: ee00e4f5c1c77042409d76e6de5a287dfe0d5bcdaa2c150db2d11fa7e4037228
+  data.tar.gz: 7b95f568aa555bb5b9959a2ced3677d7ab7b0e877c69e34db29abbdff5211e50
 SHA512:
-  metadata.gz: 46d9ea3bfe2b012d538f53abcad7f29f604ce5c54de51f7f0354d78aa9be4871955cf4666898fdf0ba0e6262147b4d9f29570ca394046b1473597b2c8118f023
-  data.tar.gz: f323676b0cfeb66a2cbbad9331ff782dbd2691d42258ca607d9e6aa4bf511f6cc6e439962d40720b75a0aceee746f5ed224f4c3f4fbd68c29a2d093395807830
+  metadata.gz: 57a64e5946f8375ef13bc634dd9e860588fd7ff7d2c6b2f3b6c14de640ba1ba26e4a4fcc12ba71958853728626d846ed067392ca876efd94f02f0a41c2d2ac5b
+  data.tar.gz: 238e8cdb81ddab8674c22eb2150c5b7522e7012eac965f264486017c7e51328a0eea23d3d352698c72eeb0275a1924a1b89575104eaf64d3796a52cdb76499dd

data/README.md

@@ -1,6 +1,6 @@
 # Janeway JSONPath parser
 
-This is a [JsonPath](https://goessner.net/articles/JsonPath/) parser.
+This is a [JSONPath](https://goessner.net/articles/JsonPath/) parser.
 It strictly follows [RFC 9535](https://www.rfc-editor.org/rfc/rfc9535.html) and passes the [JSONPath Compliance Test Suite](https://github.com/jsonpath-standard/jsonpath-compliance-test-suite).
 
 It reads a JSON input file and a query, and uses the query to find and return a set of matching values from the input.
@@ -8,8 +8,8 @@ This does for JSON the same job that XPath does for XML.
 
 This project includes:
 
-    * command-line tool to run jsonpath queries on a JSON input
     * ruby library to run jsonpath queries on a JSON input
+    * command-line tool to do the same
 
 **Contents**
 
@@ -23,13 +23,13 @@ This project includes:
 
 Install the gem from the command-line:
 ```
-    gem install janeway-jsonpath`
+    gem install janeway-jsonpath
 ```
 
 or add it to your Gemfile:
 
 ```
-    gem 'janeway-jsonpath', '~> 0.2.0'
+    gem 'janeway-jsonpath', '~> 0.4.0'
 ```
 
 ### Usage
@@ -41,45 +41,92 @@ Use single quotes around the JSON query to avoid shell interaction.
 Example:
 
 ```
-    $ janeway '$..book[?(@.price<10)]' example.json
-    [
-      {
-        "category": "reference",
-        "author": "Nigel Rees",
-        "title": "Sayings of the Century",
-        "price": 8.95
-      },
-      {
-        "category": "fiction",
-        "author": "Herman Melville",
-        "title": "Moby Dick",
-        "isbn": "0-553-21311-3",
-        "price": 8.99
+    $ cat store.json
+    { "store": {
+        "book": [
+          { "category": "reference",
+            "author": "Nigel Rees",
+            "title": "Sayings of the Century",
+            "price": 8.95
+          },
+          { "category": "fiction",
+            "author": "Evelyn Waugh",
+            "title": "Sword of Honour",
+            "price": 12.99
+          },
+          { "category": "fiction",
+            "author": "Herman Melville",
+            "title": "Moby Dick",
+            "isbn": "0-553-21311-3",
+            "price": 8.99
+          },
+          { "category": "fiction",
+            "author": "J. R. R. Tolkien",
+            "title": "The Lord of the Rings",
+            "isbn": "0-395-19395-8",
+            "price": 22.99,
+            "value": true
+          }
+        ],
+        "bicycle": {
+          "color": "red",
+          "price": 399
+        }
       }
+    }
+
+
+    ### Find and return matching values
+    $ janeway '$..book[?(@["price"] == 8.95 || @["price"] == 8.99)].title' store.json
+    [
+      "Sayings of the Century",
+      "Moby Dick"
     ]
+
+    ### Delete matching values from the input, print what remains
+    $ janeway -d '$.store.book' store.json
+    {
+      "store": {
+        "bicycle": {
+          "color": "red",
+          "price": 399
+        }
+      }
+    }
 ```
 
 You can also pipe JSON into it:
 ```
-    $ cat example.json | janeway '$..book[?(@.price<10)]'
+    $ cat store.json | janeway '$..book[?(@.price<10)]'
 ```
 
 See the help message for more capabilities: `janeway --help`
 
 #### Janeway ruby libarary
 
-Here's an example of using Janeway to execute a JSONPath query in ruby code:
+Here's an example of ruby code using Janeway to find values from a JSON document:
+To use the Janeway library in ruby code, providing a jsonpath query and an input object (Hash or Array) to search.
 ```ruby
-require 'janeway'
-require 'json'
+    require 'janeway'
+    require 'json'
 
-data = JSON.parse(File.read(ARGV.first))
-results = Janeway.find_all('$..book[?(@.price<10)]', data)
+    data = JSON.parse(File.read(ARGV.first))
+    Janeway.enum_for('$.store.book[0].title', data)
 ```
+This returns an Enumerator, which offers instance methods for using the query to operate on matching values in the input object.
+
+Following are examples showing how to work with the Enumerator methods.
 
-Alternatively, compile the query once, and share it between threads or ractors.
+##### #search
 
-The Janeway::AST::Query object is not modified after parsing, so it is easy to freeze and share concurrently.
+Returns all values that match the query.
+
+```ruby
+    results = Janeway.enum_for('$..book[?(@.price<10)]', data).search
+    # Returns every book in the store cheaper than $10
+```
+
+Alternatively, compile the query once, and share it between threads or ractors with different data sources:
 
 ```ruby
     # Create ractors with their own data sources
@@ -88,7 +135,7 @@ The Janeway::AST::Query object is not modified after parsing, so it is easy to f
         Ractor.new(index) do |i|
           query = receive
           data = JSON.parse File.read("input-file-#{i}.json")
-          puts query.find_all(data)
+          puts query.enum_for(data).search
         end
       end
 
@@ -97,6 +144,132 @@ The Janeway::AST::Query object is not modified after parsing, so it is easy to f
     ractors.each { |ractor| ractor.send(query).take }
 ```
 
+##### #each
+
+Iterates through matches one by one, without holding the entire set in memory.
+Janeway's #each iteration is particularly powerful, as it provides context for each match.
+The matched value is yielded:
+```ruby
+    data = {
+        'birds' => ['eagle', 'stork', 'cormorant'] }
+        'dogs' => ['poodle', 'pug', 'saint bernard'] },
+    }
+    Janeway.enum_for('$.birds.*', data).each do |bird|
+      puts "the bird is a #{bird}"
+    end
+```
+
+This allows the matched value to be modified in place:
+```ruby
+    Janeway.enum_for('$.birds[? @=="storck"]', data).each do |bird|
+      bird.gsub!('ck', 'k') # Fix a typo: "storck" => "stork"
+    end
+    # input list is now ['eagle', 'stork', 'cormorant']
+```
+
+However, this is not enough to replace the matched value:
+```ruby
+    Janeway.enum_for('$.birds', data).each do |bird|
+      bird = "bald eagle" if bird == 'eagle'
+      # local variable 'bird' now points to a new value, but the original list is unchanged
+    end
+    # input list is still ['eagle', 'stork', 'cormorant']
+```
+
+The second and third yield parameters are the object that contains the value, and the array index or hash key of the value.
+This allows the list or hash to be modified:
+```ruby
+    Janeway.enum_for('$.birds[? @=="eagle"]', data).each do |_bird, parent, index|
+      parent[index] = "golden eagle"
+    end
+    # input list is now ['golden eagle', 'stork', 'cormorant']
+```
+
+Lastly, the #each iterator's fourth yield parameter is the [normalized path](https://www.rfc-editor.org/rfc/rfc9535.html#name-normalized-paths) to the matched value.
+This is a jsonpath query string that uniquely points to the matched value.
+
+```ruby
+    # Collect the normalized path of every object in the input, at all levels
+    paths = []
+    Janeway.enum_for('$..*', data).each do |_bird, _parent, _index, path| do
+        paths << path
+    end
+    paths
+    # ["$['birds']", "$['dogs']", "$['birds'][0]", "$['birds'][1]", "$['birds'][2]", "$['dogs'][0]", "$['dogs'][1]", "$['dogs'][2]"]
+```
+
+##### #delete
+
+The '#delete' method deletes matched values from the input.
+```ruby
+    # delete any bird whose name is "eagle" or starts with "s"
+    Janeway.enum_for('$.birds[? @=="eagle" || search(@, "^s")]', data).delete
+    # input bird list is now ['cormorant']
+
+    # delete all dog names
+    Janeway.enum_for('$.dogs.*', data).delete
+    # dog list is now []
+```
+
+
+The `Janeway.enum_for` and `Janeway::Query#on` methods return an enumerator, so you can use the usual ruby enumerator methods, such as:
+
+#####  #map
+Return the matched elements, as modified by the block.
+
+```ruby
+    # take a dollar off every price in the store
+    sale_prices = Janeway.enum_for('$.store..price', data).map { |price| price - 1 }
+    # [7.95, 11.99, 7.99, 21.99, 398]
+```
+
+#####  #select (alias #find_all)
+
+Return only values that match the JSONPath query and also return a truthy value from the block.
+This solves a common JSON problem: You want to do a numeric comparison on a value in your JSON, but the JSON value is stored as a string type.
+
+```ruby
+    # Poorly serialized, the prices are strings
+    data =
+      { "store" => {
+          "book" => [
+             { "title" => "Sayings of the Century", "price" => "8.95" },
+             { "title" => "Sword of Honour", "price" => "12.99" },
+             { "title" => "Moby Dick", "price" => "8.99" },
+             { "title" => "The Lord of the Rings", "price" => "22.99" },
+          ]
+        }
+      }
+
+    # Can't use a filter query with a numeric comparison on a string price
+    Janeway.enum_for('$.store.book[? @.price > 10.00]', data).find_all
+    # result: []
+
+    # Solve the problem with ruby by filtering with #select and converting string to number:
+    Janeway.enum_for('$.store.book.*', data).select { |book| book['price'].to_f > 10 }
+    # result: [{"title" => "Sword of Honour", "price" => "12.99"}, {"title" => "The Lord of the Rings", "price" => "22.99"}]
+```
+
+#####  #reject
+
+Return only values that match the JSONPath query and also return false from the block.
+
+#####  #filter_map
+
+Combines #select and #map.
+Return values that match the jsonpath query and return truthy values from the block.
+Instead of returning the value from the data, return the result of the block.
+
+#####  #find 
+
+Return the first value that matches the jsonpath query and also returns a truthy value from the block
+```ruby
+    Janeway.enum_for('$.store.book[? @.price >= 10.99]', data).find { |book| book['title'].start_with?('T') }
+    # [ "The Lord of the Rings" ]
+```
+
+There are many other Enumerable methods too, see the ruby Enumerable module documenation for more.
+
 ### Related Projects
 
 - [joshbuddy/jsonpath](https://github.com/joshbuddy/jsonpath)
@@ -121,7 +294,7 @@ Also there are many non-ruby implementations of RFC 9535, here are just a few:
 * don't use `eval`, which is known to be an attack vector
 * be simple and fast with minimal dependencies
 * provide ruby-like accessors (eg. #each, #delete_if) for processing results
-* modern, linted ruby 3 code with frozen string literals
+* idiomatic, linted ruby 3 code with frozen string literals everywhere
 
 ### Non-goals
 

data/bin/janeway

@@ -15,14 +15,15 @@ HELP = <<~HELP_TEXT.freeze
     #{SCRIPT_NAME} [QUERY] [FILENAME]
 
   Purpose:
-    Print the result of applying a JsonPath query to a JSON input.
+    Print the result of applying a JSONPath query to a JSON input.
 
-    QUERY is a JsonPath query. Quote it with single quotes to avoid shell errors.
+    QUERY is a JSONPath query. Quote it with single quotes to avoid shell errors.
 
     FILENAME is the path to a JSON file to use as input.
     Alternately, input JSON can be provided on STDIN.
+    If input is not provided then the query is just checked for correctness.
 
-    For an introduction to JsonPath, see https://goessner.net/articles/JsonPath/
+    For an introduction to JSONPath, see https://goessner.net/articles/JsonPath/
     For the complete reference, see https://www.rfc-editor.org/info/rfc9535
 
   Examples:
@@ -32,7 +33,7 @@ HELP = <<~HELP_TEXT.freeze
 HELP_TEXT
 
 # Command-line options
-Options = Struct.new(:query, :query_file, :input, :compact_output, :verbose)
+Options = Struct.new(:query, :query_file, :input, :compact_output, :delete, :verbose)
 
 # Parse the command-line arguments.
 # This includes both bare words and hyphenated options.
@@ -60,6 +61,7 @@ def parse_options(argv)
 
     opts.on('-q', '--query FILE', 'Read jsonpath query from file') { |o| options.query_file = o }
     opts.on('-c', '--compact', 'Express result in compact json format') { options.compact_output = true }
+    opts.on('-d', '--delete', 'Print the input, with matching values deleted') { options.delete = true }
     opts.on('--version', 'Show version number') { abort(Janeway::VERSION) }
     opts.on('-h', '--help', 'Show this help message') { abort(opts.to_s) }
   end
@@ -74,8 +76,9 @@ end
 def read_query(path, argv)
   return File.read(path).strip if path
 
-  query = argv.find { |arg| arg.start_with?('$') }
-  abort('No JsonPath query received, provide one on the command line.') unless query
+  # Assume query is the first arg that is not a filename
+  query = argv.reject { File.exist?(_1) }.first
+  abort('No JSONPath query received, provide one on the command line.') unless query
 
   argv.delete(query)
   query
@@ -91,7 +94,7 @@ def read_input(path)
     elsif !$stdin.tty?
       $stdin.read
     else
-      abort('No input JSON provided. Provide a filename or pipe it to STDIN.')
+      return # no input json provided
     end
   parse_json(json)
 end
@@ -108,9 +111,31 @@ rescue JSON::JSONError => e
   abort "Input is not valid JSON: #{msg}"
 end
 
+# Just pares the query ane then exit.
+# Useful for testing whether a query is valid.
+# @param query [String] jsonpath
+def parse_query_and_exit(query)
+  Janeway.compile(query)
+  puts 'Query is valid. Provide input json to run the query.'
+  exit(0)
+end
+
 # @param options [Options]
 def main(options)
-  results = Janeway.find_all(options.query, options.input)
+  parse_query_and_exit(options.query) unless options.input
+
+  enum = Janeway.enum_for(options.query, options.input)
+  if options.delete
+    results = enum.delete
+    if options.compact_output
+      puts JSON.generate(options.input)
+    else
+      puts JSON.pretty_generate(options.input)
+    end
+    exit(0)
+  else
+    results = enum.search
+  end
 
   if options.compact_output
     puts JSON.generate(results)
@@ -123,7 +148,9 @@ begin
   options = parse_args(ARGV.dup)
   main(options)
 rescue Janeway::Error => e
-  abort e.detailed_message
+  warn "Error: #{e.message}\nQuery: #{e.query}\n"
+  warn "       #{' ' * e.location.col}^" if e.location
+  exit(1)
 rescue Interrupt, Errno::EPIPE
   abort("\n")
 end

data/lib/janeway/ast/expression.rb

@@ -1,5 +1,4 @@
 # frozen_string_literal: true
-
 require_relative 'helpers'
 require_relative 'error'
 
@@ -23,7 +22,7 @@ module Janeway
       def initialize(val = nil)
         # don't set the instance variable if unused, because it makes the
         # "#inspect" output cleaner in rspec test failures
-        @value = val unless val.nil? # false must be stored though!
+        @value = val unless val.nil? # literal false must be stored though!
       end
 
       # @return [String]

data/lib/janeway/enumerator.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+module Janeway
+  # Enumerator combines a parsed JSONpath query with input.
+  # It provides enumerator methods.
+  class Enumerator
+    include Enumerable
+
+    # @param query [Janeway::Query] @param input [Array, Hash]
+    def initialize(query, input)
+      @query = query
+      @input = input
+
+      raise ArgumentError, "expect Janeway::Query, got #{query.inspect}" unless query.is_a?(Query)
+    end
+
+    # Return a list of values from the input data that match the jsonpath query
+    #
+    # @return [Array] all matched objects
+    def search
+      Janeway::Interpreter.new(@query).interpret(@input)
+    end
+
+    # Iterate through each value matched by the JSONPath query.
+    #
+    # @yieldparam [Object] value matched by query
+    # @yieldparam [Array, Hash] parent object that contains the value
+    # @yieldparam [String, Integer] hash key or array index of the value within the parent object
+    # @yieldparam [String] normalized jsonpath that uniqely points to this value
+    # @return [void]
+    def each(&block)
+      return enum_for(:each) unless block_given?
+
+      Janeway::Interpreter.new(@query, as: :iterator, &block).interpret(@input)
+    end
+
+    # Delete each value matched by the JSONPath query.
+    # @return [Array, Hash]
+    def delete
+      Janeway::Interpreter.new(@query, as: :deleter).interpret(@input)
+    end
+  end
+end

data/lib/janeway/error.rb

@@ -3,7 +3,10 @@
 require_relative 'location'
 
 module Janeway
-  # Base class for JSONPath query errors
+  # Error class for Janeway.
+  # Contains a copy of the jsonpath query string.
+  # Lexer errors may also include the index into the query string that
+  # points at which token was being lexed at the time of the error.
   class Error < StandardError
     # @return [String]
     attr_reader :query
@@ -14,16 +17,10 @@ module Janeway
     # @param message [String] error message
     # @param query [String] entire query string
     # @param location [Location] location of error
-    def initialize(message, query = nil, location = nil)
-      super("Jsonpath query #{query} - #{message}")
+    def initialize(msg, query = nil, location = nil)
+      super("Jsonpath query #{query} - #{msg}")
       @query = query
       @location = location
     end
-
-    def detailed_message
-      msg = "Error: #{message}\nQuery: #{query}\n"
-      msg += "#{' ' * location.col}^" if @location
-      msg
-    end
   end
 end

data/lib/janeway/functions/length.rb

@@ -19,7 +19,7 @@ module Janeway
       unless arg.singular_query? || arg.literal?
         raise Error, "Invalid parameter - length() expects literal value or singular query, got #{arg.value.inspect}"
       end
-      raise Error, 'Too many parameters for length() function call' unless current.type == :group_end
+      raise err('Too many parameters for length() function call') unless current.type == :group_end
 
       # Meaning of return value depends on the JSON type:
       #   * string - number of Unicode scalar values in the string.