yara-ffi 3.0.0 → 4.0.0

data/README.md

@@ -1,13 +1,18 @@
 # yara-ffi
 
-A Ruby library for using [libyara](https://yara.readthedocs.io/en/stable/capi.html) via FFI.
+A Ruby library for using [YARA-X](https://virustotal.github.io/yara-x/) via FFI bindings. YARA-X is a modern, Rust-based implementation of YARA that's faster and safer than the original C implementation.
+
+## Requirements
+
+- Ruby 3.0 or later
+- YARA-X C API library (`libyara_x_capi`) installed on your system
 
 ## Installation
 
 Add this line to your application's Gemfile:
 
 ```ruby
-gem "yara"
+gem "yara-ffi"
 ```
 
 And then execute:
@@ -20,20 +25,51 @@ Or install it yourself as:
 
 ## Usage
 
+### Quick scanning with convenience methods
+
 ```ruby
-Yara.start # run before you start using the Yara API.
+rule = <<-RULE
+rule ExampleRule
+{
+  meta:
+    description = "Example rule for testing"
 
+  strings:
+    $text_string = "we were here"
+    $text_regex = /were here/
+
+  condition:
+    $text_string or $text_regex
+}
+RULE
+
+# Test a rule against data
+results = Yara.test(rule, "one day we were here and then we were not")
+puts results.first.match?  # => true
+puts results.first.rule_name  # => "ExampleRule"
+
+# Scan with a block for processing results
+Yara.scan(rule, "sample data") do |result|
+  puts "Matched: #{result.rule_name}"
+end
+```
+
+### Manual scanner usage
+
+```ruby
 rule = <<-RULE
 rule ExampleRule
 {
-meta:
+  meta:
     string_meta = "an example rule for testing"
+    int_meta = 42
+    bool_meta = true
 
-strings:
+  strings:
     $my_text_string = "we were here"
     $my_text_regex = /were here/
 
-condition:
+  condition:
     $my_text_string or $my_text_regex
 }
 RULE
@@ -41,21 +77,100 @@ RULE
 scanner = Yara::Scanner.new
 scanner.add_rule(rule)
 scanner.compile
-result = scanner.call("one day we were here and then we were not")
-result.match?
-# => true
 
-scanner.close   # run when you are done using the scanner API and want to free up memory.
-Yara.stop       # run when you are completely done using the Yara API to free up memory.
+results = scanner.scan("one day we were here and then we were not")
+result = results.first
+
+puts result.match?           # => true
+puts result.rule_name        # => "ExampleRule"
+puts result.rule_meta        # => {:string_meta=>"an example rule for testing", :int_meta=>42, :bool_meta=>true}
+puts result.rule_strings     # => {:"$my_text_string"=>"we were here", :"$my_text_regex"=>"were here"}
+
+scanner.close  # Clean up resources when done
+```
+
+### Block-based scanner usage
+
+```ruby
+# Automatically handles resource cleanup
+Yara::Scanner.open(rule) do |scanner|
+  scanner.compile
+  results = scanner.scan("test data")
+  # scanner is automatically closed when block exits
+end
+```
+
+### Multiple rules
+
+```ruby
+rule1 = <<-RULE
+rule RuleOne
+{
+  strings:
+    $a = "pattern one"
+  condition:
+    $a
+}
+RULE
+
+rule2 = <<-RULE
+rule RuleTwo
+{
+  strings:
+    $b = "pattern two"
+  condition:
+    $b
+}
+RULE
+
+scanner = Yara::Scanner.new
+scanner.add_rule(rule1)
+scanner.add_rule(rule2)
+scanner.compile
+
+results = scanner.scan("text with pattern one and pattern two")
+puts results.map(&:rule_name)  # => ["RuleOne", "RuleTwo"]
+scanner.close
 ```
 
+## API Reference
+
+### Yara module methods
+
+- `Yara.test(rule_string, data)` - Quick test of a rule against data, returns array of ScanResult objects
+- `Yara.scan(rule_string, data, &block)` - Scan data with rule, optionally yielding each result to block
+
+### Scanner class
+
+- `Scanner.new` - Create a new scanner instance
+- `Scanner.open(rule_string, namespace: nil, &block)` - Create scanner with optional rule and namespace, auto-cleanup with block
+- `#add_rule(rule_string, namespace: nil)` - Add a YARA rule to the scanner
+- `#compile` - Compile all added rules (required before scanning)
+- `#scan(data, &block)` - Scan data and return ScanResults, or yield each result to block
+- `#close` - Free scanner resources
+
+### ScanResult class
+
+- `#match?` - Returns true if rule matched
+- `#rule_name` - Name of the matched rule
+- `#rule_meta` - Hash of rule metadata (keys are symbols)
+- `#rule_strings` - Hash of rule strings (keys are symbols with $ prefix)
+
+## Installing YARA-X
+
+You'll need the YARA-X C API library installed on your system. You can:
+
+1. Build from source: https://github.com/VirusTotal/yara-x
+2. Install via package manager (when available)
+3. Use the provided Docker environment
+
 ## Development
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+See [DEVELOPMENT.md](DEVELOPMENT.md) for detailed development setup instructions, testing guidelines, and contribution workflow.
 
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/jonmagic/yara-ffi. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/jonmagic/yara-ffi/blob/master/CODE_OF_CONDUCT.md).
+Bug reports and pull requests are welcome on GitHub at https://github.com/jonmagic/yara-ffi. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/jonmagic/yara-ffi/blob/main/CODE_OF_CONDUCT.md).
 
 ## License
 
@@ -63,4 +178,4 @@ The gem is available as open source under the terms of the [MIT License](https:/
 
 ## Code of Conduct
 
-Everyone interacting in the Yara::Ffi project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/jonmagic/yara-ffi/blob/master/CODE_OF_CONDUCT.md).
+Everyone interacting in the yara-ffi project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/jonmagic/yara-ffi/blob/main/CODE_OF_CONDUCT.md).

data/lib/yara/ffi.rb

@@ -1,116 +1,305 @@
-require_relative "yr_meta"
-require_relative "yr_namespace"
-require_relative "yr_string"
-require_relative "yr_rule"
-require_relative "user_data"
-
 module Yara
-  # FFI bindings to libyara.
+  # Internal: Low-level FFI bindings to the YARA-X C API.
+  #
+  # This module provides direct Ruby FFI bindings to the yara_x_capi library.
+  # It handles dynamic library loading with multiple fallback paths and exposes
+  # the raw C functions for rule compilation, scanning, and resource management.
+  #
+  # The FFI module is primarily used internally by higher-level classes like
+  # Scanner. Direct usage requires careful memory management and error handling.
+  #
+  # Examples
+  #
+  #   # Direct FFI usage (not recommended for normal use)
+  #   rules_ptr = FFI::MemoryPointer.new(:pointer)
+  #   result = Yara::FFI.yrx_compile("rule test { condition: true }", rules_ptr)
+  #   raise "Error: #{Yara::FFI.yrx_last_error}" unless result == Yara::FFI::YRX_SUCCESS
   module FFI
     extend ::FFI::Library
-    ffi_lib "libyara"
-
-    # int yr_initialize(void)
-    attach_function :yr_initialize, [], :int
-
-    # int yr_finalize(void)
-    attach_function :yr_finalize, [], :int
-
-    # Creates a new compiler and assigns a pointer to that compiler
-    # to the pointer passed into the method. To access the complier
-    # get the pointer from the pointer you passed in.
-    #
-    # Usage:
-    # > compiler_pointer = FFI::MemoryPointer.new(:pointer)
-    # > Yara::FFI.yr_compiler_create(compiler_pointer)
-    # > compiler_pointer = compiler_pointer.get_pointer(0)
-    #
-    # int yr_compiler_create(YR_COMPILER** compiler)
-    attach_function :yr_compiler_create, [
-      :pointer, # compiler_pointer*
-    ], :int
-
-    # int yr_compiler_destroy(YR_COMPILER* compiler)
-    attach_function :yr_compiler_destroy, [
-      :pointer, # compiler_pointer
-    ], :void
-
-    # int yr_rules_destroy(YR_RULES* rules)
-    attach_function :yr_rules_destroy, [
-      :pointer, # rules_pointer
-    ], :void
-
-    # void callback_function(
-    #   int error_level,
-    #   const char* file_name,
-    #   int line_number,
-    #   const YR_RULE* rule,
-    #   const char* message,
-    #   void* user_data)
-    callback :add_rule_error_callback, [
-      :int,       # error_level
-      :string,    # file_name
-      :int,       # line_number
-      YrRule.by_ref, # YrRule*
-      :string,    # message
-      :pointer,   # user_data_pointer
-    ], :void
-
-    # void yr_compiler_set_callback(
-    #   YR_COMPILER* compiler,
-    #   YR_COMPILER_CALLBACK_FUNC callback,
-    #   void* user_data)
-    attach_function :yr_compiler_set_callback, [
-      :pointer,                 # compiler_pointer*
-      :add_rule_error_callback, # proc
-      :pointer,                 # user_data_pointer
-    ], :void
-
-    # int yr_compiler_add_string(
-    #   YR_COMPILER* compiler,
-    #   const char* string,
-    #   const char* namespace_)
-    attach_function :yr_compiler_add_string, [
-      :pointer,   # compiler_pointer*
-      :string,    # rule string
-      :string,    # namespace
-    ], :int
-
-    # int yr_compiler_get_rules(
-    #   YR_COMPILER* compiler,
-    #   YR_RULES** rules)
-    attach_function :yr_compiler_get_rules, [
-      :pointer, # compiler_pointer*
-      :pointer, # rules_pointer*
-    ], :int
-
-    # int callback_function(
-    #   int message,
-    #   void* message_data,
-    #   void* user_data)
-    callback :scan_callback, [
-      :pointer,       # YR_SCAN_CONTEXT*
-      :int,           # callback_type
-      YrRule.ptr,     # rule
-      UserData.ptr,   # user_data
-    ], :int
-
-    # int yr_rules_scan_mem(
-    #   YR_RULES* rules,
-    #   const uint8_t* buffer,
-    #   size_t buffer_size,
-    #   int flags,
-    #   YR_CALLBACK_FUNC callback,
-    #   void* user_data,
-    #   int timeout)
-    attach_function :yr_rules_scan_mem, [
-      :pointer,       # rules_pointer*
-      :pointer,       # buffer (aka test subject)
-      :size_t,        # buffer size (String#bytesize)
-      :int,           # flags
-      :scan_callback, # proc
-      :pointer,       # user_data_pointer
-      :int,           # timeout in seconds
-    ], :int
+
+    # Internal: Library search paths for yara_x_capi shared library.
+    #
+    # These paths are tried in order to locate the YARA-X C API library.
+    # The first successful load is used. This supports various deployment
+    # scenarios including system packages, Docker containers, and CI environments.
+    library_paths = [
+      "yara_x_capi",                                          # System library (preferred)
+      "/usr/local/lib/x86_64-linux-gnu/libyara_x_capi.so",    # GitHub Actions/CI
+      "/usr/local/lib/aarch64-linux-gnu/libyara_x_capi.so",   # Local Docker (ARM)
+      "/usr/local/lib/libyara_x_capi.so",                     # Generic fallback
+      "libyara_x_capi"                                        # Final fallback
+    ]
+
+    library_loaded = false
+    library_paths.each do |path|
+      begin
+        ffi_lib path
+        library_loaded = true
+        break
+      rescue LoadError
+        next
+      end
+    end
+
+    raise LoadError, "Could not load yara_x_capi library from any of: #{library_paths.join(', ')}" unless library_loaded
+
+    # Public: Compile YARA rule source into executable rules object.
+    #
+    # This is the primary compilation function that parses YARA rule source code
+    # and creates an optimized rules object for scanning. The rules object must
+    # be freed with yrx_rules_destroy when no longer needed.
+    #
+    # src   - A String containing YARA rule source code
+    # rules - A FFI::MemoryPointer that will receive the rules object pointer
+    #
+    # Examples
+    #
+    #   rules_ptr = FFI::MemoryPointer.new(:pointer)
+    #   result = Yara::FFI.yrx_compile(rule_source, rules_ptr)
+    #
+    # Returns an Integer result code (YRX_SUCCESS on success).
+    # C Signature: enum YRX_RESULT yrx_compile(const char *src, struct YRX_RULES **rules)
+    attach_function :yrx_compile, [:string, :pointer], :int
+
+    # Public: Get the last error message from YARA-X operations.
+    #
+    # When any YARA-X function returns an error code, this function provides
+    # a human-readable description of what went wrong. The returned string
+    # is managed by YARA-X and should not be freed.
+    #
+    # Examples
+    #
+    #   if result != YRX_SUCCESS
+    #     error_msg = Yara::FFI.yrx_last_error
+    #     raise "YARA Error: #{error_msg}"
+    #   end
+    #
+    # Returns a String containing the last error message.
+    # C Signature: const char* yrx_last_error(void)
+    attach_function :yrx_last_error, [], :string
+
+    # Public: Free memory associated with a compiled rules object.
+    #
+    # This function must be called to free the memory allocated by yrx_compile.
+    # After calling this function, the rules pointer becomes invalid and should
+    # not be used.
+    #
+    # rules - A Pointer to the rules object to destroy
+    #
+    # Examples
+    #
+    #   Yara::FFI.yrx_rules_destroy(rules_ptr)
+    #
+    # Returns nothing.
+    # C Signature: void yrx_rules_destroy(struct YRX_RULES *rules)
+    attach_function :yrx_rules_destroy, [:pointer], :void
+
+    # Public: Create a scanner object from compiled rules.
+    #
+    # A scanner is needed to perform actual pattern matching against data.
+    # Multiple scanners can be created from the same rules object to enable
+    # concurrent scanning. The scanner must be freed with yrx_scanner_destroy.
+    #
+    # rules   - A Pointer to compiled rules object
+    # scanner - A FFI::MemoryPointer that will receive the scanner object pointer
+    #
+    # Examples
+    #
+    #   scanner_ptr = FFI::MemoryPointer.new(:pointer)
+    #   result = Yara::FFI.yrx_scanner_create(rules_ptr, scanner_ptr)
+    #
+    # Returns an Integer result code (YRX_SUCCESS on success).
+    # C Signature: enum YRX_RESULT yrx_scanner_create(const struct YRX_RULES *rules, struct YRX_SCANNER **scanner)
+    attach_function :yrx_scanner_create, [:pointer, :pointer], :int
+
+    # Public: Free memory associated with a scanner object.
+    #
+    # This function must be called to free the memory allocated by
+    # yrx_scanner_create. After calling this function, the scanner pointer
+    # becomes invalid and should not be used.
+    #
+    # scanner - A Pointer to the scanner object to destroy
+    #
+    # Examples
+    #
+    #   Yara::FFI.yrx_scanner_destroy(scanner_ptr)
+    #
+    # Returns nothing.
+    # C Signature: void yrx_scanner_destroy(struct YRX_SCANNER *scanner)
+    attach_function :yrx_scanner_destroy, [:pointer], :void
+
+    # Internal: Callback function type for rule matching events.
+    #
+    # This callback is invoked for each rule that matches during scanning.
+    # The callback receives pointers to the matching rule and optional user data.
+    #
+    # rule      - A Pointer to the YRX_RULE structure
+    # user_data - A Pointer to optional user-provided data
+    #
+    # C Signature: typedef void (*YRX_ON_MATCHING_RULE)(const struct YRX_RULE *rule, void *user_data)
+    callback :matching_rule_callback, [:pointer, :pointer], :void
+
+    # Public: Set callback for handling rule matches during scanning.
+    #
+    # This function registers a callback that will be invoked each time a rule
+    # matches during scanning. The callback can extract information about the
+    # matching rule and optionally halt scanning.
+    #
+    # scanner   - A Pointer to the scanner object
+    # callback  - A Proc matching the matching_rule_callback signature
+    # user_data - A Pointer to optional data passed to callback (can be nil)
+    #
+    # Examples
+    #
+    #   callback = proc { |rule_ptr, user_data| puts "Rule matched!" }
+    #   result = Yara::FFI.yrx_scanner_on_matching_rule(scanner_ptr, callback, nil)
+    #
+    # Returns an Integer result code (YRX_SUCCESS on success).
+    # C Signature: enum YRX_RESULT yrx_scanner_on_matching_rule(struct YRX_SCANNER *scanner, YRX_ON_MATCHING_RULE callback, void *user_data)
+    attach_function :yrx_scanner_on_matching_rule, [:pointer, :matching_rule_callback, :pointer], :int
+
+    # Public: Scan data using the configured scanner and rules.
+    #
+    # This function performs pattern matching against the provided data using
+    # all rules in the scanner. Any matching rules trigger the registered
+    # callback function. The data is scanned as binary regardless of content.
+    #
+    # scanner - A Pointer to the scanner object
+    # data    - A Pointer to the data buffer to scan
+    # len     - A size_t indicating the length of data in bytes
+    #
+    # Examples
+    #
+    #   data_ptr = FFI::MemoryPointer.new(:char, data.bytesize)
+    #   data_ptr.put_bytes(0, data)
+    #   result = Yara::FFI.yrx_scanner_scan(scanner_ptr, data_ptr, data.bytesize)
+    #
+    # Returns an Integer result code (YRX_SUCCESS on success).
+    # C Signature: enum YRX_RESULT yrx_scanner_scan(struct YRX_SCANNER *scanner, const uint8_t *data, size_t len)
+    attach_function :yrx_scanner_scan, [:pointer, :pointer, :size_t], :int
+
+    # Public: Extract the identifier (name) from a rule object.
+    #
+    # This function retrieves the rule name from a YRX_RULE pointer, typically
+    # called from within a matching rule callback. The identifier is returned
+    # as a pointer and length rather than a null-terminated string.
+    #
+    # rule  - A Pointer to the YRX_RULE structure
+    # ident - A FFI::MemoryPointer that will receive the identifier pointer
+    # len   - A FFI::MemoryPointer that will receive the identifier length
+    #
+    # Examples
+    #
+    #   ident_ptr = FFI::MemoryPointer.new(:pointer)
+    #   len_ptr = FFI::MemoryPointer.new(:size_t)
+    #   result = Yara::FFI.yrx_rule_identifier(rule_ptr, ident_ptr, len_ptr)
+    #
+    # Returns an Integer result code (YRX_SUCCESS on success).
+    # C Signature: enum YRX_RESULT yrx_rule_identifier(const struct YRX_RULE *rule, const uint8_t **ident, size_t *len)
+    attach_function :yrx_rule_identifier, [:pointer, :pointer, :pointer], :int
+
+    # Internal: Callback function type for metadata iteration.
+    #
+    # This callback is invoked for each metadata entry during rule metadata
+    # iteration. The callback receives pointers to the metadata and user data.
+    #
+    # metadata  - A Pointer to the YRX_METADATA structure
+    # user_data - A Pointer to optional user-provided data
+    #
+    # C Signature: typedef void (*YRX_METADATA_CALLBACK)(const struct YRX_METADATA *metadata, void *user_data)
+    callback :metadata_callback, [:pointer, :pointer], :void
+
+    # Public: Iterate through all metadata entries in a rule.
+    #
+    # This function calls the provided callback for each metadata key-value pair
+    # defined in the rule. Metadata includes information like author, description,
+    # and custom tags defined in the rule's meta section.
+    #
+    # rule      - A Pointer to the YRX_RULE structure
+    # callback  - A Proc matching the metadata_callback signature
+    # user_data - A Pointer to optional data passed to callback (can be nil)
+    #
+    # Examples
+    #
+    #   callback = proc { |metadata_ptr, user_data| puts "Found metadata" }
+    #   result = Yara::FFI.yrx_rule_iter_metadata(rule_ptr, callback, nil)
+    #
+    # Returns an Integer result code (YRX_SUCCESS on success).
+    # C Signature: enum YRX_RESULT yrx_rule_iter_metadata(const struct YRX_RULE *rule, YRX_METADATA_CALLBACK callback, void *user_data)
+    attach_function :yrx_rule_iter_metadata, [:pointer, :metadata_callback, :pointer], :int
+
+    # Internal: Callback function type for pattern iteration.
+    #
+    # This callback is invoked for each pattern (string) during rule pattern
+    # iteration. The callback receives pointers to the pattern and user data.
+    #
+    # pattern   - A Pointer to the YRX_PATTERN structure
+    # user_data - A Pointer to optional user-provided data
+    #
+    # C Signature: typedef void (*YRX_PATTERN_CALLBACK)(const struct YRX_PATTERN *pattern, void *user_data)
+    callback :pattern_callback, [:pointer, :pointer], :void
+
+    # Public: Iterate through all patterns (strings) in a rule.
+    #
+    # This function calls the provided callback for each string pattern defined
+    # in the rule. Patterns are the actual search terms that YARA looks for
+    # during scanning, defined in the rule's strings section.
+    #
+    # rule      - A Pointer to the YRX_RULE structure
+    # callback  - A Proc matching the pattern_callback signature
+    # user_data - A Pointer to optional data passed to callback (can be nil)
+    #
+    # Examples
+    #
+    #   callback = proc { |pattern_ptr, user_data| puts "Found pattern" }
+    #   result = Yara::FFI.yrx_rule_iter_patterns(rule_ptr, callback, nil)
+    #
+    # Returns an Integer result code (YRX_SUCCESS on success).
+    # C Signature: enum YRX_RESULT yrx_rule_iter_patterns(const struct YRX_RULE *rule, YRX_PATTERN_CALLBACK callback, void *user_data)
+    attach_function :yrx_rule_iter_patterns, [:pointer, :pattern_callback, :pointer], :int
+
+    # Public: Extract the identifier (name) from a pattern object.
+    #
+    # This function retrieves the pattern identifier from a YRX_PATTERN pointer,
+    # typically called from within a pattern iteration callback. Pattern
+    # identifiers are the variable names like $string1, $hex_pattern, etc.
+    #
+    # pattern - A Pointer to the YRX_PATTERN structure
+    # ident   - A FFI::MemoryPointer that will receive the identifier pointer
+    # len     - A FFI::MemoryPointer that will receive the identifier length
+    #
+    # Examples
+    #
+    #   ident_ptr = FFI::MemoryPointer.new(:pointer)
+    #   len_ptr = FFI::MemoryPointer.new(:size_t)
+    #   result = Yara::FFI.yrx_pattern_identifier(pattern_ptr, ident_ptr, len_ptr)
+    #
+    # Returns an Integer result code (YRX_SUCCESS on success).
+    # C Signature: enum YRX_RESULT yrx_pattern_identifier(const struct YRX_PATTERN *pattern, const uint8_t **ident, size_t *len)
+    attach_function :yrx_pattern_identifier, [:pointer, :pointer, :pointer], :int
+
+    # Public: YARA-X result codes for operation status.
+    #
+    # These constants represent the possible return values from YARA-X functions.
+    # YRX_SUCCESS (0) indicates successful operation, while other values indicate
+    # various error conditions that can be interpreted using yrx_last_error.
+
+    # Public: Operation completed successfully.
+    YRX_SUCCESS = 0
+
+    # Public: YARA rule syntax error during compilation.
+    YRX_SYNTAX_ERROR = 1
+
+    # Public: Variable definition or reference error.
+    YRX_VARIABLE_ERROR = 2
+
+    # Public: Error during scanning operation.
+    YRX_SCAN_ERROR = 3
+
+    # Public: Scanning operation timed out.
+    YRX_SCAN_TIMEOUT = 4
+
+    # Public: Invalid argument passed to function.
+    YRX_INVALID_ARGUMENT = 5
   end
 end