janeway-jsonpath 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 07de626741809ed0f0c9d62f626a45a1eb91513cfda3b536160060428e23cda0
+  data.tar.gz: 70953f9cee1de853145ff4eb56a80ed535a93d6126211982b7adaada9438c9a0
+SHA512:
+  metadata.gz: 003fbc4fe8e79e174f8e06147f4790554ca2577ed0b29503f1ef61619a33ef082a7e2f2ee3a39a2eb412e0b28778ed90b7ad3c9b6205d67888b2121f9f32e488
+  data.tar.gz: ee42a2d1601aa765d748a5a806f79653622a7cfc83c1fdbe87a04cd0c3f7effb67b5a8c3a88fa6733bfb0c50fb138fa0b152aed58c5a28b646032a6951309b84

data/LICENSE

@@ -0,0 +1,7 @@
+Copyright (c) 2025 Fraser Hanson
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,43 @@
+# Janeway JSONPath parser
+
+### Purpose
+
+This is a [JsonPath](https://goessner.net/articles/JsonPath/) parser.
+
+It reads a JSON input file and a query.
+It uses the query to find and return a set of matching values from the input.
+This does for JSON the same job that XPath does for XML.
+
+This project includes:
+    * command-line tool to run jsonpath queries on a JSON input
+    * ruby library to run jsonpath queries on a JSON input
+
+### Goals
+
+* parse Goessner JSONPath, similar to https://github.com/joshbuddy/jsonpath
+* implement all of [IETF RFC 9535](https://github.com/ietf-wg-jsonpath)
+* don't use regular expressions for parsing, for performance
+* don't use `eval`, which is known to be an attack vector
+* be simple and fast with minimal dependencies
+* use helpful query parse errors which help understand and improve queries, rather than describing issues in the code
+* modern, linted ruby 3 code with frozen string literals
+
+### Non-goals
+
+* Changing behavior to follow [other implementations] (https://cburgmer.github.io/json-path-comparison/)
+
+The JSONPath RFC was in draft status for a long time and has seen many changes.
+There are many implementations based on older drafts, or which add features that were never in the RFC at all.
+
+The goal here is perfect adherence to the finalized [RFC 9535](https://github.com/ietf-wg-jsonpath) rather than adding features that are in other implementations.
+
+The RFC was finalized in 2024, and it has a rigorous [suite of compliance tests.](https://github.com/jsonpath-standard/jsonpath-compliance-test-suite)
+
+With these tools it is possible to have JSONPath implementations in many languages with identical behavior.
+
+### Implementation
+
+Functionality is based on [IETF RFC 9535, "JSONPath: Query Expressions for JSON"](https://www.rfc-editor.org/rfc/rfc9535.html#filter-selector)
+The examples in the RFC have been implemented as unit tests.
+
+For details not covered in the RFC, it does the most reasonable thing based on [what other JSONPath parsers do.](https://cburgmer.github.io/json-path-comparison/). However, this is always secondary to following the RFC. Many of the recommended behaviors there contradict the RFC, so it is one or the other.

data/bin/janeway

@@ -0,0 +1,130 @@
+#!/usr/bin/env ruby
+
+# frozen_string_literal: true
+
+require 'json'
+require 'optparse'
+
+# FIXME: delete from final version?
+$LOAD_PATH << "#{__dir__}/../lib/"
+require 'janeway'
+
+SCRIPT_NAME = File.basename($PROGRAM_NAME)
+HELP = <<~HELP_TEXT.freeze
+  Usage:
+    #{SCRIPT_NAME} [QUERY] [FILENAME]
+
+  Purpose:
+    Print the result of applying a JsonPath query to a JSON input.
+
+    QUERY is a JsonPath query. Quote it with single quotes to avoid shell errors.
+
+    FILENAME is the path to a JSON file to use as input.
+    Alternately, input JSON can be provided on STDIN.
+
+    For an introduction to JsonPath, see https://goessner.net/articles/JsonPath/
+    For the complete reference, see https://www.rfc-editor.org/info/rfc9535
+
+  Examples:
+    #{SCRIPT_NAME} '$.store.book[*].author' input.json
+    cat input.json | #{SCRIPT_NAME} '$.store.book[*].author'
+
+HELP_TEXT
+
+# Command-line options
+Options = Struct.new(:query, :query_file, :input, :compact_output, :verbose)
+
+# Parse the command-line arguments.
+# This includes both bare words and hyphenated options.
+#
+# @param argv [Array<String>]
+def parse_args(argv)
+  # parse command-line options
+  argv << '--help' if argv.empty?
+  options = parse_options(argv)
+
+  # Next get jsonpath query and input jsonn
+  options.query = read_query(options.query_file, argv)
+  options.input = read_input(argv.first)
+  options
+end
+
+# Parse the command-line options.
+#
+# @param argv [Array<String>]
+def parse_options(argv)
+  options = Options.new
+  op = OptionParser.new do |opts|
+    opts.banner = HELP
+    opts.separator('Options:')
+
+    opts.on('-q', '--query FILE', 'Read jsonpath query from file') { |o| options.query_file = o }
+    opts.on('-c', '--compact', 'Express result in compact json format') { options.compact_output = true }
+    opts.on('--version', 'Show version number') { abort(Janeway::VERSION) }
+    opts.on('-h', '--help', 'Show this help message') { abort(opts.to_s) }
+  end
+  op.parse!(argv)
+  options
+end
+
+# Read jsonpath query from file or command-line arguments
+#
+# @param path [String,nil] query path, or nil if not provided
+# @param argv [Array<String>] command line arguments
+def read_query(path, argv)
+  return File.read(path).strip if path
+
+  query = argv.find { |arg| arg.start_with?('$') }
+  abort('No JsonPath query received, provide one on the command line.') unless query
+
+  argv.delete(query)
+  query
+end
+
+# Read input json from file or STDIN
+# @param path [String,nil] json file path, or nil if not provided
+# @return [Hash, Array] un-serialized json, as ruby objects
+def read_input(path)
+  json =
+    if path
+      File.read(path)
+    elsif !$stdin.tty?
+      $stdin.read
+    else
+      abort('No input JSON provided. Provide a filename or pipe it to STDIN.')
+    end
+  parse_json(json)
+end
+
+# Parse JSON, and abort if it is invalid
+# @param json [String]
+# @return [Hash, Array] un-serialized json, as ruby objects
+def parse_json(json)
+  JSON.parse(json)
+rescue JSON::JSONError => e
+  # JSON error messages may include the entire input, so limit how much is printed.
+  msg = e.message[0..256]
+  msg += "\n..." if e.message.length > 256
+  abort "Input is not valid JSON: #{msg}"
+end
+
+# @param options [Options]
+def main(options)
+  results = Janeway.find_all(options.query, options.input)
+
+  if options.compact_output
+    puts JSON.generate(results)
+  else
+    puts JSON.pretty_generate(results)
+  end
+end
+
+begin
+  options = parse_args(ARGV.dup)
+  main(options)
+rescue Janeway::Error => e
+  puts
+  abort("Error: #{e}")
+rescue Interrupt, Errno::EPIPE
+  abort("\n")
+end

data/lib/janeway/ast/array_slice_selector.rb

@@ -0,0 +1,104 @@
+# frozen_string_literal: true
+
+require_relative 'selector'
+
+module Janeway
+  module AST
+    # An array slice selects a series of elements from an array.
+    #
+    # It accepts a start and end positions, and a step value that define the range to select.
+    # All of these are optional.
+    #
+    # @example
+    #   $[1:3]
+    #   $[5:]
+    #   $[1:5:2]
+    #   $[5:1:-2]
+    #   $[::-1]
+    #   $[:]
+    #
+    # ArraySliceSelector needs to store "default" arguments differently from
+    # "explicit" arguments, since they're interpreted differently.
+    #
+    class ArraySliceSelector < Janeway::AST::Selector
+      # @param start [Integer, nil]
+      # @param end_ [Integer, nil]
+      # @param step [Integer, nil]
+      def initialize(start = nil, end_ = nil, step = nil)
+        [start, end_, step].each do |arg|
+          next if arg.nil? || arg.is_a?(Integer)
+
+          raise ArgumentError, "Expect Integer or nil, got #{arg.inspect}"
+        end
+        super(nil)
+
+        # Nil values are kept to indicate that the "default" values is to be used.
+        # The interpreter selects the actual values.
+        @start = start
+        @end = end_
+        @step = step
+      end
+
+      # Return the step size to use for stepping through the array.
+      # Defaults to 1 if it was not given to the constructor.
+      #
+      # @return [Integer]
+      def step
+        # The iteration behavior of jsonpath does not match that of ruby Integer#step.
+        # Support the behavior of Integer#step, which needs this:
+        #   1. for stepping forward between positive numbers, use a positive number
+        #   2. for stepping backward between positive numbers, use a negative number
+        #   3. for stepping backward from positive to negative, use a negative number
+        #   4. for stepping backward from negative to negative, use a positive number
+        # Case #4 has to be detected and the sign of step inverted
+        @step || 1
+      end
+
+      # Return the start index to use for stepping through the array, based on a specified array size
+      #
+      # @param input_size [Integer]
+      # @return [Integer]
+      def start_index(input_size)
+        if @start
+          @start.clamp(0, input_size)
+        elsif step.positive?
+          0
+        else # negative step
+          input_size - 1 # last index of input
+        end
+      end
+
+      # Return the end index to use for stepping through the array, based on a specified array size
+      # End index is calculated to omit the final index value, as per the RFC.
+      #
+      # @param input_size [Integer]
+      # @return [Integer]
+      def end_index(input_size)
+        if @end
+          value = @end.clamp(0, input_size)
+          step.positive? ? value - 1 : value + 1 # +/- to exclude the final element
+        elsif step.positive?
+          input_size - 1 # last index of input
+        else
+          0
+        end
+      end
+
+      # ignores the brackets: argument, this always needs surrounding brackets
+      # @return [String]
+      def to_s(*)
+        if @step
+          "[#{@start}:#{@end}:#{@step}]"
+        else
+          "[#{@start}:#{@end}]"
+        end
+      end
+
+      # @param level [Integer]
+      # @return [Array]
+      def tree(level)
+        [indented(level, to_s)]
+      end
+    end
+  end
+end

data/lib/janeway/ast/binary_operator.rb

@@ -0,0 +1,101 @@
+# frozen_string_literal: true
+
+module Janeway
+  module AST
+    class BinaryOperator < Janeway::AST::Expression
+      attr_reader :operator, :left, :right
+
+      def initialize(operator, left = nil, right = nil)
+        super(nil)
+        raise ArgumentError, "expect symbol, got #{operator.inspect}" unless operator.is_a?(Symbol)
+
+        @operator = operator
+        self.left = left if left
+        self.right = right if right
+      end
+
+      # Set the left-hand-side value
+      # @param expr [AST::Expression]
+      def left=(expr)
+        if comparison_operator? && !(expr.literal? || expr.singular_query?)
+          raise Error, "Expression #{expr} does not produce a singular value for #{operator_to_s} comparison"
+        end
+
+        # Compliance test suite requires error for this, but don't have go so far as to bar every literal
+        if expr.is_a?(Boolean) && right.is_a?(Boolean)
+          raise Error, "Literal \"#{expr}\" must be compared to an expression, not another literal (\"#{left}\")"
+        end
+
+        @left = expr
+      end
+
+      # Set the left-hand-side value
+      # @param expr [AST::Expression]
+      def right=(expr)
+        if comparison_operator? && !(expr.literal? || expr.singular_query?)
+          raise Error, "Expression #{expr} does not produce a singular value for #{operator_to_s} comparison"
+        end
+
+        # Compliance test suite requires error for this, but don't have go so far as to bar every literal
+        if expr.is_a?(Boolean) && left.is_a?(Boolean)
+          raise Error, "Literal \"#{expr}\" must be compared to an expression, not another literal (\"#{left}\")"
+        end
+
+        @right = expr
+      end
+
+      def to_s
+        # Make precedence explicit by adding parentheses
+        "(#{@left} #{operator_to_s} #{@right})"
+      end
+
+      private
+
+      def operator_to_s
+        case operator
+        when :and then '&&'
+        when :equal then '=='
+        when :greater_than then '>'
+        when :greater_than_or_equal then '>='
+        when :less_than then '<'
+        when :less_than_or_equal then '<='
+        when :not_equal then '!='
+        when :or then '||'
+        else
+          raise "unknown binary operator #{operator}"
+        end
+      end
+
+      # @param level [Integer]
+      # @return [Array]
+      def tree(level)
+        [
+          indented(level, to_s),
+          @left.tree(level + 1),
+          @right.tree(level + 1),
+        ]
+      end
+
+      # True if this operator is a comparison operator
+      # @return [Boolean]
+      def comparison_operator?
+        operator_type == :comparison
+      end
+
+      # True if this operator is a logical operator
+      # @return [Boolean]
+      def logical_operator?
+        operator_type == :logical
+      end
+
+      def operator_type
+        case operator
+        when :and, :or then :logical
+        when :equal, :not_equal, :greater_than, :greater_than_or_equal, :less_than, :less_than_or_equal then :comparison
+        else
+          raise "unknown binary operator #{operator}"
+        end
+      end
+    end
+  end
+end

data/lib/janeway/ast/boolean.rb

@@ -0,0 +1,24 @@
+# frozen_string_literal: true
+
+module Janeway
+  module AST
+    # Represent keywords true, false
+    class Boolean < Janeway::AST::Expression
+      def to_s
+        @value ? 'true' : 'false'
+      end
+
+      # @param level [Integer]
+      # @return [Array]
+      def tree(level)
+        [indented(level, to_s)]
+      end
+
+      # Return true if this is a literal expression
+      # @return [Boolean]
+      def literal?
+        true
+      end
+    end
+  end
+end

data/lib/janeway/ast/child_segment.rb

@@ -0,0 +1,48 @@
+# frozen_string_literal: true
+
+require 'forwardable'
+
+# A set of selectors within brackets, as a comma-separated list.
+# https://www.rfc-editor.org/rfc/rfc9535.html#child-segment
+#
+# @example
+#    $[*, *]
+#    $[1, 2, 3]
+#    $[name1, [1:10]]
+module Janeway
+  module AST
+    # Represent a union of 2 or more selectors.
+    class ChildSegment < Janeway::AST::Expression
+      extend Forwardable
+      def_delegators :@value, :size, :first, :last, :each, :map, :empty?
+
+      # Subsequent expression that modifies the result of this selector list.
+      # This one is not in the selector list, it comes afterward.
+      attr_accessor :child
+
+      def initialize
+        super([]) # @value holds the expressions in the selector
+        @child = nil # @child is the next expression, which modifies the output of this one
+      end
+
+      # Add a selector to the list
+      def <<(selector)
+        raise ArgumentError, "expect Selector, got #{selector.inspect}" unless selector.is_a?(Selector)
+
+        @value << selector
+      end
+
+      def to_s(with_child: true)
+        str = @value.map { |selector| selector.to_s(brackets: false) }.join(', ')
+        with_child ? "[#{str}]#{@child}" : "[#{str}]"
+      end
+
+      # @param level [Integer]
+      # @return [Array]
+      def tree(level)
+        msg = format('[%s]', @value.map(&:to_s).join(', '))
+        [indented(level, msg), @child&.tree(level + 1)]
+      end
+    end
+  end
+end

data/lib/janeway/ast/current_node.rb

@@ -0,0 +1,71 @@
+# frozen_string_literal: true
+
+module Janeway
+  module AST
+    # CurrentNode addresses the current node of the filter-selector that is directly enclosing the identifier.
+    #
+    # Note: Within nested filter-selectors, there is no syntax to address the
+    # current node of any other than the directly enclosing filter-selector
+    # (i.e., of filter-selectors enclosing the filter-selector that is directly
+    # enclosing the identifier).
+    #
+    # It may be followed by another selector which is applied to it.
+    # Within the IETF examples, I see @ followed by these selectors:
+    #   * name selector (dot notation)
+    #   * filter selector (bracketed)
+    #   * wildcard selector
+    # Probably best to assume that any selector type could appear here.
+    #
+    # It may also not be followed by any selector, and be used directly with a comparison operator.
+    #
+    # @example
+    #   $.a[?@.b == 'kilo']
+    #   $.a[?@.b]
+    #   $[?@.*]
+    #   $[?@[?@.b]]
+    #   $.a[?@<2 || @.b == "k"]
+    #   $.o[?@>1 && @<4]
+    #   $.a[?@ == @]
+    #
+    # Construct accepts an optional Selector which will be applied to the "current" node
+    class CurrentNode < Janeway::AST::Expression
+      def to_s
+        if @value.is_a?(NameSelector) || @value.is_a?(WildcardSelector)
+          "@.#{@value.to_s(brackets: false)}"
+        else
+          "@#{@value}"
+        end
+      end
+
+      # True if this is the root of a singular-query.
+      # @see https://www.rfc-editor.org/rfc/rfc9535.html#name-well-typedness-of-function-
+      #
+      # @return [Boolean]
+      def singular_query?
+        return true unless @value # there are no following selectors
+
+        selector_types = []
+        selector = @value
+        loop do
+          selector_types << selector.class
+          selector = selector.child
+          break unless selector
+        end
+        allowed = [AST::IndexSelector, AST::NameSelector]
+        selector_types.uniq.all? { allowed.include?(_1) }
+      end
+
+      # True if this is a bare current node operator, without a following expression
+      # @return [Boolean]
+      def empty?
+        @value.nil?
+      end
+
+      # @param level [Integer]
+      # @return [Array]
+      def tree(level)
+        [indented(level, '@'), @value.tree(indent + 1)]
+      end
+    end
+  end
+end

data/lib/janeway/ast/descendant_segment.rb

@@ -0,0 +1,44 @@
+# frozen_string_literal: true
+
+require_relative 'selector'
+
+module Janeway
+  module AST
+    # An array slice start:end:step selects a series of elements from
+    # an array, giving a start position, an end position, and an optional step
+    # value that moves the position from the start to the end.
+    #
+    # @example
+    #   $..j      Values of keys equal to 'j'
+    #   $..[0]    First entry of any array
+    #   $..[*]    All values
+    #   $..*      All values
+    #   $..[*, *] All values, twice non-deterministic order
+    #   $..[0, 1] Multiple segments
+    class DescendantSegment < Janeway::AST::Selector
+      attr_accessor :child
+
+      def initialize(selector)
+        super
+
+        @child = nil
+      end
+
+      def to_s
+        "..#{@value}#{@child}"
+      end
+
+      # @return [AST::Selector]
+      #
+      def selector
+        value
+      end
+
+      # @param level [Integer]
+      # @return [Array]
+      def tree(level)
+        [indented(level, "..#{@value}"), child&.tree(level + 1)]
+      end
+    end
+  end
+end

data/lib/janeway/ast/error.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+require_relative '../error'
+
+module Janeway
+  module AST
+    # Error raised during parsing, for use by AST classes
+    class Error < Janeway::Error; end
+  end
+end

data/lib/janeway/ast/expression.rb

@@ -0,0 +1,55 @@
+# frozen_string_literal: true
+
+require_relative 'helpers'
+require_relative 'error'
+
+module Janeway
+  module AST
+    INDENT = '  '
+
+    class Expression
+      attr_accessor :value
+
+      def initialize(val = nil)
+        # don't set the instance variable if unused, because it makes the
+        # "#inspect" output cleaner in rspec test failures
+        @value = val unless val.nil?  # false must be stored though!
+      end
+
+      # @return [String]
+      def type
+        name = self.class.to_s.split('::').last # eg. Janeway::AST::FunctionCall => "FunctionCall"
+        Helpers.camelcase_to_underscore(name) # eg. "FunctionCall" => "function_call"
+      end
+
+      # Return the given message, indented
+      #
+      # @param level [Integer]
+      # @param msg [String]
+      # @return [String]
+      def indented(level, msg)
+        format('%s%s', (INDENT * level), msg)
+      end
+
+      # @param level [Integer]
+      # @return [Array]
+      def tree(level)
+        [indented(level, to_s)]
+      end
+
+      # Return true if this is a literal expression
+      # @return [Boolean]
+      def literal?
+        false
+      end
+
+      # True if this is the root of a singular-query.
+      # @see https://www.rfc-editor.org/rfc/rfc9535.html#name-well-typedness-of-function-
+      #
+      # @return [Boolean]
+      def singular_query?
+        false
+      end
+    end
+  end
+end