rika 2.0.4-java → 2.2.0-java

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1a4cc1c56edb22131c3409bbaee3617febac8d40ba3aed28600cde997d93d465
-  data.tar.gz: 8b38c319ca598ab107762222bc0b097bcf5001aaf53fc1c33e912cff83ff7997
+  metadata.gz: bd45b4d1119a5372921bf75ba95f8b42c4b75b3299065c77c5a1809f30d21a93
+  data.tar.gz: 62f53d53f1b55d08de9f427fa2c7891baa13b0a6406d304fdf6d4fd7673b239c
 SHA512:
-  metadata.gz: 1cfea7c20c8c2b294ff6a5d29877cc4670034ac6c58ac299bd5b3ac716d55bc4f1df39843d7c3a33776186ca73d7652747e8853fc914f114e2ecef16e8b26278
-  data.tar.gz: c6c5aeef86b5e9b2cba4b31f395602f94b8c1094975f5543c388175cb619c34c87258ade2043cc0faa47cff7f5bbb87f85e298179faaa4465e96e47647b1bfe0
+  metadata.gz: 7c66fb4cc8fde420e7a25f35dea66ed3e88ce6b3bc4d79df9c26425569feb72281e9321230ac5c99905cb50bb6acb1724428ffea2f3cd41a7ab59e647b73918a
+  data.tar.gz: 8ad4a5ea3da65eec6a590871855a99699c9cc890ed5f28f331679383cd3fb4531022d695aee112153cba2859a3bc63c8a6daee63fec45dc1a596beb923a3c20b

data/.gitignore

@@ -1,17 +1,20 @@
-*.gem
-*.rbc
-.DS_Store
+_yardoc
 .bundle
 .config
-coverage/
+.DS_Store
 .idea/
+.ruby-version
+.rvmrc
 .yardoc
-Gemfile.lock
-InstalledFiles
-_yardoc
+*.gem
+*.rbc
 coverage
+coverage/
 doc/
+Gemfile.lock
+InstalledFiles
 lib/bundler/man
+pdf/
 pkg
 projectFilesBackup/
 rdoc

data/.rspec

@@ -1,2 +1,2 @@
+--format documentation
 --color
---format progress

data/README.md

@@ -2,11 +2,11 @@
 
 [Rika](https://github.com/keithrbennett/rika) is a [JRuby](https://www.jruby.org) wrapper for
 the [Apache Tika](http://tika.apache.org/) Java library, which extracts text and metadata from files and resources
-of [many different formats](https://tika.apache.org/1.24.1/formats.html).
+of [many different formats](https://tika.apache.org/3.1.0/formats.html).
 
-Rika can be used as a library in your Ruby code, or on the command line.
+Rika can be used as a library in your Ruby code or on the command line using the provided executable.
 
-For class and method level documentation, please use [YARD](https://rubydoc.info/gems/yard).
+For class and method level documentation, please use [YARD](https://yardoc.org/).
 You can `gem install yard`, then run `yard doc` from the project root, 
 and then open the `doc/index.html` file in a browser.
 
@@ -23,12 +23,10 @@ for more advanced needs.
 See the [Other Tika Resources](#other-tika-resources) section of this document for alternatives to
 Rika that may suit more demanding needs.
 
-Rika can be used either as a gem in your own Ruby project, or on the command line using the provided executable.
 
 ## Usage in Your Ruby Code
 
-> [!IMPORTANT]  
-> **It is necessary to call `Rika.init` before using Rika.**  This is because the loading of the Tika library
+> **⚠️ IMPORTANT:** It is necessary to call `Rika.init` before using Rika.  This is because the loading of the Tika library
 has been put in an init method, rather than at load time, so that 'jar file not found or specified' errors 
 do not prevent your application from loading. If you forget to call `Rika.init`, you may see seemingly unrelated
 error messages.
@@ -80,16 +78,16 @@ Rika can also be used on the command line using the `rika` executable.  For exam
 specify one or more filespecs or URL's as arguments:
 
 ```bash
-rika x.pdf https://github.com/keithrbennett/rika
+rika x.pdf https://www.google.com
 ```
-
+  
 > [!NOTE]
 > If running `rika` produces an error indicating that the JRuby interpreter cannot be found, try preceding it with `jruby`, e.g. `jruby rika x.pdf`.
 
 Here is the help text:
 
 ```
-Rika v2.0.2 (Tika v2.9.0) - https://github.com/keithrbennett/rika
+Rika v2.2.0 (Tika v3.1.0) - https://github.com/keithrbennett/rika
 
 Usage: rika [options] <file or url> [...file or url...]
 Output formats are: [a]wesome_print, [t]o_s, [i]nspect, [j]son), [J] for pretty json, and [y]aml.
@@ -98,13 +96,19 @@ Values for the text, metadata, and as_array boolean options may be specified as
   Enable:  +, true,  yes, [empty]
   Disable: -, false, no, [long form option with no- prefix, e.g. --no-metadata]
 
+IMPORTANT: Always quote wildcard patterns when files might contain special characters:
+           - Double quotes: "*.pdf" (allows variable expansion)
+           - Single quotes: '*.pdf' (prevents all shell interpretation)
+           Use -n/--dry-run to preview command execution and check for issues.
+
     -f, --format FORMAT              Output format (default: at)
     -m, --[no-]metadata [FLAG]       Output metadata (default: true)
     -t, --[no-]text [FLAG]           Output text (default: true)
     -k, --[no-]key-sort [FLAG]       Sort metadata keys case insensitively (default: true)
-    -s, --[no-]source [FLAG]         Output document source file or URL (default: false)
+    -s, --[no-]source [FLAG]         Output document source file or URL (default: true)
     -a, --[no-]as-array [FLAG]       Output all parsed results as an array (default: false)
-    -v, --version                    Output version
+    -n, --[no-]dry-run [FLAG]        Show what would be done without executing (default: false)
+    -v, --version                    Output software versions
     -h, --help                       Output help
 ```    
 
@@ -144,6 +148,72 @@ If you find yourself using the same options over and over again, you can put the
 variable. For example, if the default behavior of sorting keys does not work for your language, you can disable it
 for all invocations of the `rika` command by specifying `-k-` in the RIKA_OPTIONS environment variable.
 
+### Using Wildcards for File Specification
+
+Rika now supports in-app expansion of wildcard patterns for file specification. This means you can quote wildcard patterns 
+to prevent the shell from expanding them, and Rika will handle the expansion internally:
+
+```bash
+# Let Rika handle the expansion (no practical limit on number of files)
+rika '**/*.pdf'
+
+# Shell expands wildcards (limited by shell's maximum argument length)
+rika **/*.pdf
+```
+
+> **⚠️ IMPORTANT:** Always quote wildcard patterns (using either single or double quotes) when they might match files containing special characters!
+> 
+> When unquoted, the shell may misinterpret filenames containing spaces, $, *, ?, [], (), {}, &, |, <, >, ;,
+> backticks, quotes, and other shell metacharacters, causing unpredictable behavior:
+>
+> ```bash
+> # PROBLEMATIC - Shell breaks/misinterprets files with special characters
+> rika pdf/*
+> 
+> # CORRECT - Both single and double quotes work to preserve filenames
+> rika "pdf/*"      # Double quotes allow variable expansion within the pattern
+> rika 'pdf/*.pdf'  # Single quotes prevent all shell interpretation
+> ```
+>
+> Use the `-n` (dry-run) option to preview how your command will be processed and to check for issues.
+
+This is particularly useful when dealing with large numbers of files, as shell expansion may hit command line length limits.
+In-app expansion has no practical limit on the number of files that can be processed.
+
+Supported wildcard patterns:
+- `*` - Match any number of characters
+- `?` - Match a single character
+- `[abc]` - Match one character from the set
+- `{a,b,c}` - Match any of the patterns a, b, or c
+- `**` - Recursive directory matching (match all files in all subdirectories)
+
+### Dry Run Mode
+
+You can use the `-n` or `--dry-run` option to see what would happen when running a command without actually executing it:
+
+```bash
+rika -n -f jy README.md
+```
+
+Like other boolean options, dry-run mode can be disabled with various syntax options:
+```bash
+rika -n- README.md            # Hyphen suffix
+rika -n false README.md       # "false" value
+rika -n no README.md          # "no" value
+rika --no-dry-run README.md   # --no- prefix
+```
+
+This will display:
+- All the options that would be used, with human-readable descriptions
+- A list of files that would be processed
+- Any issues that were detected (like non-existent files)
+
+This is useful for:
+- Debugging complex commands
+- Checking what files would be processed when using wildcards
+- Verifying options before running on large sets of files
+- Understanding how different options would affect processing
+
 ### Machine Readable Data Support
 
 If both metadata and text are output, and the same output format is used for both, and that format is JSON

data/RELEASE_NOTES.md

@@ -1,5 +1,21 @@
 ## Release Notes
 
+### v2.2.0
+
+* Switched to TikaInputStream for better memory management and performance
+* Added dry-run mode (-n/--dry-run) to preview command execution
+* Improved error handling and reporting (empty files, symlinks, invalid URLs)
+* Enhanced URL validation and processing
+* Added comprehensive integration tests
+* Moved gem executable from bin/ to exe/, made 'exe' the bindir
+* Changed executable shebang from ruby to jruby
+* Added prominent warning about quoting wildcards
+* Added support for UTF-8 encoding in environment options
+
+### v2.1.0
+
+* Add ability to specify quoted wildcard filespecs on command line so Ruby will expand them.
+
 ### v2.0.4
 
 * Fix uninitialized constant StringIO error (issue #16).

data/{bin → exe}/rika

@@ -1,4 +1,4 @@
-#!/usr/bin/env ruby
+#!/usr/bin/env jruby
 # frozen_string_literal: true
 
 require 'rika/cli/rika_command'

data/lib/rika/cli/args_parser.rb

@@ -1,5 +1,11 @@
 # frozen_string_literal: true
 
+require 'optparse'
+require 'shellwords'
+require 'uri'
+require 'awesome_print'
+require_relative 'rika_command'
+
 # Processes the array of arguments (ARGV by default) and returns the options, targets, and help string.
 class ArgsParser
   attr_reader :args, :options, :option_parser
@@ -12,14 +18,15 @@ class ArgsParser
       metadata: true,
       text: true,
       source: true,
-      key_sort: true
+      key_sort: true,
+      dry_run: false,
     }.freeze
 
   # Parses the command line arguments.
-  # Shorthand for ArgsParser.new.call. This call is recommended to pro tect the caller in case
+  # Shorthand for ArgsParser.new.call. This call is recommended to protect the caller in case
   # this functionality is repackaged as a Module or otherwise modified.
   # @param [Array] args the command line arguments (overridable for testing, etc.)
-  # @return [Array<Hash,String>] [options, targets, help_string],
+  # @return [Array<Hash,Array,String,Hash>] [options, targets, help_string, issues],
   #   or exits if help or version requested or no targets specified.
   def self.call(args = ARGV)
     new.call(args)
@@ -27,7 +34,7 @@ class ArgsParser
 
   # Parses the command line arguments.
   # @param [Array] args the command line arguments (overridable for testing, etc.)
-  # @return [Array<Hash,String>] [options, targets, help_string],
+  # @return [Array<Hash,Array,String,Hash>] [options, targets, help_string, issues],
   #   or exits if help or version requested or no targets specified.
   def call(args = ARGV)
     @args = args
@@ -36,12 +43,16 @@ class ArgsParser
     @option_parser = create_option_parser
     option_parser.parse!(args)
     postprocess_format_options
-    targets = create_target_array
-    [options, targets, option_parser.help]
+    targets, issues = process_args_for_targets
+    [options, targets, option_parser.help, issues]
   end
 
+  # -------------------------------------------------------
+  private
+  # -------------------------------------------------------
+
   # @return [OptionParser]
-  private def create_option_parser
+  def create_option_parser
     OptionParser.new do |opts|
       opts.banner =  <<~BANNER
         Rika v#{Rika::VERSION} (Tika v#{Rika.tika_version}) - #{Rika::PROJECT_URL}
@@ -53,6 +64,11 @@ class ArgsParser
           Enable:  +, true,  yes, [empty]
           Disable: -, false, no, [long form option with no- prefix, e.g. --no-metadata]
 
+        ⚠️ IMPORTANT: Always quote wildcard patterns when files might contain special characters!
+          - Double quotes: "*.pdf" (allows variable expansion)
+          - Single quotes: '*.pdf' (prevents all shell interpretation)
+          Use -n/--dry-run to preview command execution and check for issues.
+
       BANNER
 
       format_message = 'Output format (default: at)'
@@ -72,7 +88,7 @@ class ArgsParser
         options[:key_sort] = (v.nil? ? true : v)
       end
 
-      opts.on('-s', '--[no-]source [FLAG]', TrueClass, 'Output document source file or URL (default: false)') do |v|
+      opts.on('-s', '--[no-]source [FLAG]', TrueClass, 'Output document source file or URL (default: true)') do |v|
         options[:source] = (v.nil? ? true : v)
       end
 
@@ -81,52 +97,138 @@ class ArgsParser
         options[:as_array] = (v.nil? ? true : v)
       end
 
-      opts.on('-v', '--version', 'Output version') do
+      opts.on('-n', '--[no-]dry-run [FLAG]', TrueClass, 'Show what would be done without executing (default: false)') do |v|
+        options[:dry_run] = (v.nil? ? true : v)
+      end
+
+      opts.on('-v', '--version', 'Output software versions') do
         puts versions_string
         exit
       end
 
       opts.on('-h', '--help', 'Output help') do
-        puts opts
+        RikaCommand.output_help_text(opts)
         exit
       end
     end
   end
 
-  # @return [Array] the targets specified on the command line, possibly expanded by the shell,
-  #   and with any directories removed.
-  private def create_target_array
-    targets = args.dup.reject { |arg| File.directory?(arg) }.freeze # reject dirs to handle **/* globbing
-    targets.map(&:freeze)
-  end
-
   # Fills in the second format option character if absent, and removes any excess characters
   # @return [String] format options 2-character value, e.g. 'at'
-  private def postprocess_format_options
+  def postprocess_format_options
     # If only one format letter is specified, use it for both metadata and text.
     options[:format] *= 2 if options[:format].length == 1
 
     # Ignore and remove extra characters after the first two format characters.
     options[:format] = options[:format][0..1]
+    
+    # Validate format characters
+    valid_formats = %w[a i j J t y]
+    format_chars = options[:format].chars
+    
+    if options[:format].strip.empty? || format_chars.any? { |c| !valid_formats.include?(c) }
+      $stderr.puts "Error: Invalid format characters in '#{options[:format]}'. Valid characters are: #{valid_formats.join(', ')}"
+      exit 1
+    end
   end
 
   # If the user wants to specify options in an environment variable ("RIKA_OPTIONS"),
   # then this method will insert those options at the beginning of the `args` array,
   # where they can be overridden by command line arguments.
-  private def prepend_environment_args
+  def prepend_environment_args
     env_opt_string = environment_options
     args_to_prepend = Shellwords.shellsplit(env_opt_string)
     args.unshift(args_to_prepend).flatten!
   end
 
   # @return [String] the value of the RIKA_OPTIONS environment variable if present, else ''.
-  private def environment_options
-    ENV['RIKA_OPTIONS'] || ''
+  def environment_options
+    env_value = ENV['RIKA_OPTIONS'] || ''
+    # Necessary to handle escaped spaces and other special characters consistently:
+    env_value.dup.force_encoding('UTF-8')
   end
 
   # @return [String] string containing versions of Rika and Tika, with labels
-  private def versions_string
+  def versions_string
     java_version = Java::java.lang.System.getProperty("java.version")
     "Versions: Rika: #{Rika::VERSION}, Tika: #{Rika.tika_version}, Java: #{java_version}"
   end
+
+  # Process the command line arguments to find URLs and file specifications
+  # @return [Array<Array,Hash>] [targets, issues] where targets is an array of valid URLs and filespecs
+  #   and issues is a hash of categories to arrays of problematic targets
+  def process_args_for_targets
+    targets = []
+    issues = Hash.new { |hash, key| hash[key] = [] }
+
+    args.each do |arg|
+      if arg.include?('://') 
+        if File.exist?(arg)
+          # Files containing "://" are highly unusual in normal filesystems.
+          # This is a defensive check to prevent misinterpreting valid files as URLs
+          # just because they contain URL-like patterns, which could happen in test
+          # environments or with specially crafted filenames.
+          issues[:file_with_url_characters] << arg
+        else
+          # Otherwise treat it as a URL candidate
+          process_url_candidate(arg, targets, issues)
+        end
+      else
+        process_filespec_candidate(arg, targets, issues)
+      end
+    end
+    
+    [targets, issues]
+  end
+
+  # Determines if a string looks like a URL based on the presence of "://"
+  # @param [String] arg string to check
+  # @return [Boolean] true if the string appears to be a URL
+  def looks_like_url?(arg)
+    arg.include?('://')
+  end
+
+  # Process a candidate URL
+  # @param [String] arg the URL to process
+  # @param [Array] targets array to add valid URLs to
+  # @param [Hash] issues hash to collect issues
+  # @return [void]
+  def process_url_candidate(arg, targets, issues)
+    begin
+      uri = URI.parse(arg)
+      if ['http', 'https'].include?(uri.scheme.downcase)
+        targets << arg
+      else
+        issues[:bad_url_scheme] << arg
+      end
+    rescue URI::InvalidURIError
+      issues[:invalid_url] << arg
+    end
+  end
+
+  # Process a candidate file specification
+  # @param [String] arg the filespec to process
+  # @param [Array] targets array to add valid filespecs to
+  # @param [Hash] issues hash to collect issues
+  # @return [void]
+  def process_filespec_candidate(arg, targets, issues)
+    matching_filespecs = Dir.glob(arg)
+    
+    if matching_filespecs.empty?
+      issues[:non_existent_file] << arg
+      return
+    end
+    
+    matching_filespecs.each do |file|
+      if File.symlink?(file)
+        issues[:is_symlink_wont_process] << file
+      elsif File.directory?(file)
+        # ignore
+      elsif File.empty?(file)
+        issues[:empty_file] << file
+      else
+        targets << file
+      end
+    end
+  end
 end