yara-ffi 3.1.0 → 4.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/copilot-instructions.md +266 -0
- data/.github/workflows/ruby.yml +69 -17
- data/CHANGELOG.md +90 -1
- data/DEVELOPMENT.md +188 -0
- data/Dockerfile +19 -11
- data/Gemfile.lock +38 -23
- data/README.md +56 -36
- data/USAGE.md +747 -0
- data/lib/yara/compiler.rb +161 -0
- data/lib/yara/ffi.rb +500 -111
- data/lib/yara/pattern_match.rb +178 -0
- data/lib/yara/scan_result.rb +573 -71
- data/lib/yara/scan_results.rb +224 -0
- data/lib/yara/scanner.rb +436 -45
- data/lib/yara/version.rb +5 -1
- data/lib/yara.rb +73 -15
- data/yara-ffi.gemspec +4 -4
- metadata +13 -15
- data/lib/yara/user_data.rb +0 -5
- data/lib/yara/yr_meta.rb +0 -10
- data/lib/yara/yr_namespace.rb +0 -5
- data/lib/yara/yr_rule.rb +0 -11
- data/lib/yara/yr_string.rb +0 -15
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: f964b7eb475719ea6ceb456125b6ead06be8d9263a8839267ee96cdbe072b470
|
4
|
+
data.tar.gz: 36cc2f79374b0d00f245329f5d3e18423b6f16740e92f893c07d662f2183ce67
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 5636bffbece91a2b3d48568e7bcac3e75d0f7713231c1a0abfbcbf83a0436b1d950ca5dccb5dae98b82c6ab3686521f61b561a2802f1c26145f887ea64fccf37
|
7
|
+
data.tar.gz: b10394734d3c00d986a5f12a6a2682087a7641e2abb62108728fc699f1e79f395f4bf2c5d94842c440fb15c960690773760907be56b4f14edbe5c86c3589aa3c
|
@@ -0,0 +1,266 @@
|
|
1
|
+
# yara-ffi AI Coding Instructions
|
2
|
+
|
3
|
+
This Ruby gem provides FFI bindings to YARA-X (Rust-based YARA implementation) for malware/pattern detection with advanced pattern matching analysis, rule compilation, serialization, and metadata support.
|
4
|
+
|
5
|
+
## Quick Development Guide
|
6
|
+
|
7
|
+
**Start Here for New Features:**
|
8
|
+
1. Run `script/test` (if Docker image missing, run `script/bootstrap` first)
|
9
|
+
2. Follow **Red-Green-Refactor** cycle with small semantic commits after each cycle
|
10
|
+
3. Scanner lifecycle: `add_rule()` → `compile()` → `scan()` → `close()`
|
11
|
+
4. Always use resource-safe patterns: `Scanner.open { |s| ... }` or manual `close()`
|
12
|
+
5. Interactive testing: `docker run -it --mount type=bind,src="$(pwd)",dst=/app yara-ffi bin/console`
|
13
|
+
6. **Documentation**: See `USAGE.md` for comprehensive examples and patterns
|
14
|
+
|
15
|
+
## Core Components (Read These Files First)
|
16
|
+
|
17
|
+
- `lib/yara/scanner.rb`: Main API - compile-then-scan workflow, resource management, global variables
|
18
|
+
- `lib/yara/compiler.rb`: Advanced rule compilation with globals, error diagnostics, serialization
|
19
|
+
- `lib/yara/scan_result.rb`: Enhanced result parsing with pattern matches, metadata, tags, namespaces
|
20
|
+
- `lib/yara/pattern_match.rb`: Detailed pattern match information with offsets and data extraction
|
21
|
+
- `lib/yara/ffi.rb`: Raw FFI bindings with error codes (`YRX_SUCCESS = 0`)
|
22
|
+
- Tests in `test/`: Comprehensive test coverage for all features
|
23
|
+
- `test/scanner_test.rb`: Basic scanner patterns
|
24
|
+
- `test/scanner_pattern_match_test.rb`: Pattern matching analysis
|
25
|
+
- `test/compiler_test.rb`: Advanced compilation features
|
26
|
+
- `test/serialize_test.rb`: Rule serialization/deserialization
|
27
|
+
- `test/metadata_test.rb`: Metadata extraction
|
28
|
+
- `test/tags_test.rb`: Tag support
|
29
|
+
- `test/namespace_test.rb`: Namespace functionality
|
30
|
+
|
31
|
+
## Key Features & APIs
|
32
|
+
|
33
|
+
### Pattern Matching Analysis (NEW)
|
34
|
+
```ruby
|
35
|
+
# Detailed pattern match information
|
36
|
+
results = scanner.scan(data)
|
37
|
+
result = results.first
|
38
|
+
|
39
|
+
# Access specific pattern matches
|
40
|
+
matches = result.matches_for_pattern(:$suspicious)
|
41
|
+
matches.each do |match|
|
42
|
+
puts "At offset #{match.offset}: #{match.matched_data(data)}"
|
43
|
+
end
|
44
|
+
|
45
|
+
# Pattern match convenience methods
|
46
|
+
result.pattern_matched?(:$api_call) # => true/false
|
47
|
+
result.total_matches # => 5
|
48
|
+
result.all_matches # => [PatternMatch, ...]
|
49
|
+
```
|
50
|
+
|
51
|
+
### Advanced Rule Compilation (NEW)
|
52
|
+
```ruby
|
53
|
+
# Use Compiler for complex scenarios
|
54
|
+
compiler = Yara::Compiler.new
|
55
|
+
compiler.define_global_str("ENV", "production")
|
56
|
+
compiler.define_global_bool("DEBUG", false)
|
57
|
+
compiler.add_source(rule1, "rule1.yar")
|
58
|
+
compiler.add_source(rule2, "rule2.yar")
|
59
|
+
|
60
|
+
# Build and serialize
|
61
|
+
serialized = compiler.build_serialized
|
62
|
+
scanner = Yara::Scanner.from_serialized(serialized)
|
63
|
+
```
|
64
|
+
|
65
|
+
### Global Variables (NEW)
|
66
|
+
```ruby
|
67
|
+
# Set individual globals
|
68
|
+
scanner.set_global_str("ENV", "production")
|
69
|
+
scanner.set_global_int("MAX_SIZE", 1000)
|
70
|
+
scanner.set_global_bool("DEBUG", false)
|
71
|
+
|
72
|
+
# Bulk setting with error handling
|
73
|
+
scanner.set_globals({
|
74
|
+
"ENV" => "production",
|
75
|
+
"RETRIES" => 3
|
76
|
+
}, strict: false)
|
77
|
+
```
|
78
|
+
|
79
|
+
### Metadata & Tags (NEW)
|
80
|
+
```ruby
|
81
|
+
# Access metadata with type safety
|
82
|
+
result.rule_meta[:author] # Raw access
|
83
|
+
result.metadata_string(:author) # Type-safe String
|
84
|
+
result.metadata_int(:severity) # Type-safe Integer
|
85
|
+
|
86
|
+
# Tag support
|
87
|
+
result.tags # => ["malware", "trojan"]
|
88
|
+
result.has_tag?("malware") # => true
|
89
|
+
result.qualified_name # => "namespace.rule_name"
|
90
|
+
```
|
91
|
+
|
92
|
+
## Critical FFI Patterns
|
93
|
+
|
94
|
+
**Memory Management (ALWAYS Required):**
|
95
|
+
```ruby
|
96
|
+
# Preferred - auto-cleanup
|
97
|
+
Scanner.open(rule_string) do |scanner|
|
98
|
+
scanner.compile
|
99
|
+
results = scanner.scan(data)
|
100
|
+
end
|
101
|
+
|
102
|
+
# Manual - MUST call close()
|
103
|
+
scanner = Scanner.new
|
104
|
+
# ... use scanner
|
105
|
+
scanner.close # Memory leak without this!
|
106
|
+
```
|
107
|
+
|
108
|
+
**Error Handling - Check These First:**
|
109
|
+
```ruby
|
110
|
+
result = Yara::FFI.yrx_compile(@rule_source, @rules_pointer)
|
111
|
+
if result != Yara::FFI::YRX_SUCCESS
|
112
|
+
error_msg = Yara::FFI.yrx_last_error
|
113
|
+
raise CompilationError, "Failed: #{error_msg}"
|
114
|
+
end
|
115
|
+
```
|
116
|
+
|
117
|
+
**Library Loading Strategy (Multiple Fallbacks):**
|
118
|
+
```ruby
|
119
|
+
ffi_lib "/usr/local/lib/aarch64-linux-gnu/libyara_x_capi.so" # Specific first
|
120
|
+
ffi_lib "yara_x_capi" # System library fallback
|
121
|
+
```
|
122
|
+
|
123
|
+
## Development Environment
|
124
|
+
|
125
|
+
**Docker-First Development:** All development happens in Docker container with YARA-X pre-built:
|
126
|
+
- `script/test` - runs tests (builds image automatically if needed)
|
127
|
+
- `script/bootstrap` - only run if `script/test` fails due to missing Docker image
|
128
|
+
- Interactive: `docker run -it --mount type=bind,src="$(pwd)",dst=/app yara-ffi bin/console`
|
129
|
+
|
130
|
+
**TDD Workflow:** Follow Red-Green-Refactor with small semantic commits:
|
131
|
+
1. **Red**: Write failing test
|
132
|
+
2. **Green**: Make test pass with minimal code
|
133
|
+
3. **Refactor**: Clean up while keeping tests green
|
134
|
+
4. **Commit**: Small semantic commit describing the feature/fix
|
135
|
+
|
136
|
+
**Testing:** Uses Minitest. Tests in `test/` directory focus on Scanner lifecycle and rule matching.
|
137
|
+
|
138
|
+
## Common YARA Rule Patterns
|
139
|
+
|
140
|
+
**Basic Rule Template:**
|
141
|
+
```ruby
|
142
|
+
rule = <<-RULE
|
143
|
+
rule ExampleRule
|
144
|
+
{
|
145
|
+
meta:
|
146
|
+
description = "Example rule"
|
147
|
+
author = "test"
|
148
|
+
|
149
|
+
strings:
|
150
|
+
$text = "pattern"
|
151
|
+
$regex = /regex pattern/
|
152
|
+
|
153
|
+
condition:
|
154
|
+
$text or $regex
|
155
|
+
}
|
156
|
+
RULE
|
157
|
+
```
|
158
|
+
|
159
|
+
**Multiple Rules Pattern:**
|
160
|
+
```ruby
|
161
|
+
scanner = Scanner.new
|
162
|
+
scanner.add_rule(rule1)
|
163
|
+
scanner.add_rule(rule2)
|
164
|
+
scanner.compile
|
165
|
+
results = scanner.scan(data) # Returns array of ScanResult objects
|
166
|
+
```
|
167
|
+
|
168
|
+
## Code Patterns
|
169
|
+
|
170
|
+
**Resource Management:**
|
171
|
+
```ruby
|
172
|
+
# Preferred block pattern
|
173
|
+
Scanner.open(rule_string) do |scanner|
|
174
|
+
scanner.compile
|
175
|
+
results = scanner.scan(data)
|
176
|
+
end # Auto-cleanup
|
177
|
+
|
178
|
+
# Manual pattern - must call close()
|
179
|
+
scanner = Scanner.new
|
180
|
+
scanner.add_rule(rule)
|
181
|
+
scanner.compile
|
182
|
+
# ... use scanner
|
183
|
+
scanner.close # Required!
|
184
|
+
```
|
185
|
+
|
186
|
+
**Error Handling:** Custom exceptions for different failure modes:
|
187
|
+
- `Scanner::CompilationError` - YARA rule syntax issues
|
188
|
+
- `Scanner::ScanError` - Runtime scanning failures
|
189
|
+
- `Scanner::NotCompiledError` - Scanning before compilation
|
190
|
+
- `Compiler::CompileError` - Compilation errors with structured diagnostics
|
191
|
+
|
192
|
+
**Enhanced Result Processing:** ScanResult now provides:
|
193
|
+
- Structured metadata access via YARA-X API
|
194
|
+
- Detailed pattern match information with offsets/lengths
|
195
|
+
- Tag extraction and querying
|
196
|
+
- Namespace support
|
197
|
+
- Pattern match convenience methods
|
198
|
+
|
199
|
+
## Performance & Advanced Features
|
200
|
+
|
201
|
+
**Rule Serialization for Production:**
|
202
|
+
```ruby
|
203
|
+
# Compile once, use many times
|
204
|
+
compiler = Yara::Compiler.new
|
205
|
+
compiler.add_source(ruleset)
|
206
|
+
serialized = compiler.build_serialized
|
207
|
+
|
208
|
+
# Create multiple scanners from same rules
|
209
|
+
scanners = 10.times.map { Yara::Scanner.from_serialized(serialized) }
|
210
|
+
```
|
211
|
+
|
212
|
+
**Timeout Configuration:**
|
213
|
+
```ruby
|
214
|
+
scanner.set_timeout(10000) # 10 seconds
|
215
|
+
```
|
216
|
+
|
217
|
+
**Error Diagnostics:**
|
218
|
+
```ruby
|
219
|
+
begin
|
220
|
+
compiler.build
|
221
|
+
rescue Yara::Compiler::CompileError
|
222
|
+
errors = compiler.errors_json
|
223
|
+
warnings = compiler.warnings_json
|
224
|
+
end
|
225
|
+
```
|
226
|
+
|
227
|
+
## Adding New FFI Functions
|
228
|
+
|
229
|
+
**Pattern to Follow:**
|
230
|
+
```ruby
|
231
|
+
# In lib/yara/ffi.rb
|
232
|
+
attach_function :yrx_new_function, [:param_types], :return_type
|
233
|
+
|
234
|
+
# In lib/yara/scanner.rb - always check return codes
|
235
|
+
result = Yara::FFI.yrx_new_function(params)
|
236
|
+
if result != Yara::FFI::YRX_SUCCESS
|
237
|
+
error_msg = Yara::FFI.yrx_last_error
|
238
|
+
raise ScanError, "Operation failed: #{error_msg}"
|
239
|
+
end
|
240
|
+
```
|
241
|
+
|
242
|
+
**Available FFI Functions (key ones):**
|
243
|
+
- `yrx_compile(src, rules_ptr)` - Compile rules from string
|
244
|
+
- `yrx_scanner_create(rules, scanner_ptr)` - Create scanner from compiled rules
|
245
|
+
- `yrx_scanner_scan(scanner, data, len)` - Scan data
|
246
|
+
- `yrx_scanner_set_global_*()` - Set global variables on scanner
|
247
|
+
- `yrx_scanner_set_timeout()` - Configure scan timeout
|
248
|
+
- `yrx_compiler_*()` - Advanced compilation functions
|
249
|
+
- `yrx_rules_serialize/deserialize()` - Rule serialization
|
250
|
+
- `yrx_rule_iter_*()` - Iterate rule components (patterns, metadata, tags)
|
251
|
+
- `yrx_pattern_iter_matches()` - Extract pattern match details
|
252
|
+
- `yrx_last_error()` - Get last error message
|
253
|
+
- Cleanup: `yrx_rules_destroy()`, `yrx_scanner_destroy()`, `yrx_compiler_destroy()`
|
254
|
+
|
255
|
+
## Documentation Structure
|
256
|
+
|
257
|
+
- `README.md`: Project overview, installation, minimal usage example
|
258
|
+
- `USAGE.md`: Comprehensive usage guide with quick reference + detailed examples
|
259
|
+
- `DEVELOPMENT.md`: Development setup and contribution workflow
|
260
|
+
- `.github/copilot-instructions.md`: This file - AI coding guidance
|
261
|
+
|
262
|
+
## Dependencies & Constraints
|
263
|
+
|
264
|
+
**Docker Dependencies:** Container includes Rust toolchain + cargo-c for building YARA-X from source.
|
265
|
+
|
266
|
+
When adding features, maintain the resource-managed Scanner pattern and ensure proper C memory cleanup.
|
data/.github/workflows/ruby.yml
CHANGED
@@ -1,10 +1,4 @@
|
|
1
|
-
#
|
2
|
-
# They are provided by a third-party and are governed by
|
3
|
-
# separate terms of service, privacy policy, and support
|
4
|
-
# documentation.
|
5
|
-
# This workflow will download a prebuilt Ruby version, install dependencies and run tests with Rake
|
6
|
-
# For more information see: https://github.com/marketplace/actions/setup-ruby-jruby-and-truffleruby
|
7
|
-
|
1
|
+
# GitHub Actions workflow for Ruby gem testing and validation
|
8
2
|
name: Ruby
|
9
3
|
|
10
4
|
on:
|
@@ -13,28 +7,86 @@ on:
|
|
13
7
|
pull_request:
|
14
8
|
branches: [ main ]
|
15
9
|
|
10
|
+
permissions:
|
11
|
+
contents: read
|
12
|
+
|
16
13
|
jobs:
|
17
14
|
test:
|
18
|
-
|
15
|
+
name: Test Ruby ${{ matrix.ruby-version }}
|
19
16
|
runs-on: ubuntu-latest
|
17
|
+
|
20
18
|
strategy:
|
19
|
+
fail-fast: false
|
21
20
|
matrix:
|
22
|
-
ruby-version: ['
|
21
|
+
ruby-version: ['3.2', '3.3']
|
23
22
|
|
24
23
|
steps:
|
25
|
-
-
|
24
|
+
- name: Checkout code
|
25
|
+
uses: actions/checkout@v4
|
26
|
+
|
26
27
|
- name: Set up Ruby
|
27
|
-
# To automatically get bug fixes and new Ruby versions for ruby/setup-ruby,
|
28
|
-
# change this to (see https://github.com/ruby/setup-ruby#versioning):
|
29
28
|
uses: ruby/setup-ruby@v1
|
30
29
|
with:
|
31
30
|
ruby-version: ${{ matrix.ruby-version }}
|
32
31
|
bundler-cache: true # runs 'bundle install' and caches installed gems automatically
|
33
|
-
|
32
|
+
|
33
|
+
- name: Install system dependencies
|
34
34
|
run: |
|
35
35
|
sudo apt-get update -y
|
36
|
-
sudo apt-get install -y
|
37
|
-
|
38
|
-
|
36
|
+
sudo apt-get install -y curl git unzip build-essential
|
37
|
+
|
38
|
+
- name: Install Rust, cargo-c, and build YARA-X C API library
|
39
|
+
run: |
|
40
|
+
# Install Rust
|
41
|
+
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
|
42
|
+
source $HOME/.cargo/env
|
43
|
+
|
44
|
+
# Set default Rust toolchain
|
45
|
+
rustup default stable
|
46
|
+
|
47
|
+
# Install cargo-c
|
48
|
+
cargo install cargo-c
|
49
|
+
|
50
|
+
# Build and install YARA-X C API library
|
51
|
+
git clone --depth 1 --branch v1.5.0 https://github.com/VirusTotal/yara-x.git /tmp/yara-x
|
52
|
+
cd /tmp/yara-x
|
53
|
+
|
54
|
+
# Ensure rustup default is set before running cargo with sudo
|
55
|
+
source $HOME/.cargo/env
|
56
|
+
rustup default stable
|
57
|
+
sudo env "PATH=$HOME/.cargo/bin:$PATH" "RUSTUP_HOME=$HOME/.rustup" "CARGO_HOME=$HOME/.cargo" $HOME/.cargo/bin/cargo cinstall -p yara-x-capi --release
|
58
|
+
sudo ldconfig
|
39
59
|
- name: Run tests
|
40
|
-
run: bundle exec rake
|
60
|
+
run: bundle exec rake test
|
61
|
+
|
62
|
+
- name: Run RuboCop (if present)
|
63
|
+
run: |
|
64
|
+
if bundle list rubocop > /dev/null 2>&1; then
|
65
|
+
bundle exec rubocop
|
66
|
+
else
|
67
|
+
echo "RuboCop not found, skipping..."
|
68
|
+
fi
|
69
|
+
continue-on-error: true
|
70
|
+
|
71
|
+
# lint:
|
72
|
+
# name: Lint and Security Check
|
73
|
+
# runs-on: ubuntu-latest
|
74
|
+
|
75
|
+
# steps:
|
76
|
+
# - name: Checkout code
|
77
|
+
# uses: actions/checkout@v4
|
78
|
+
|
79
|
+
# - name: Set up Ruby
|
80
|
+
# uses: ruby/setup-ruby@v1
|
81
|
+
# with:
|
82
|
+
# ruby-version: '3.3'
|
83
|
+
# bundler-cache: true
|
84
|
+
|
85
|
+
# - name: Run bundle audit (if present)
|
86
|
+
# run: |
|
87
|
+
# if bundle list bundle-audit > /dev/null 2>&1; then
|
88
|
+
# bundle exec bundle-audit check --update
|
89
|
+
# else
|
90
|
+
# echo "bundle-audit not found, skipping..."
|
91
|
+
# fi
|
92
|
+
# continue-on-error: true
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,94 @@
|
|
1
1
|
## [Unreleased]
|
2
2
|
|
3
|
+
## [4.1.0] - 2025-08-20
|
4
|
+
|
5
|
+
- **NEW**: Added advanced `Yara::Compiler` API for complex rule compilation scenarios
|
6
|
+
- `Compiler.new` - Create a new compiler instance
|
7
|
+
- `Compiler#define_global_*` methods for setting globals before compilation
|
8
|
+
- `Compiler#add_source` for adding rules from multiple sources
|
9
|
+
- `Compiler#build` and `Compiler#build_serialized` for creating compiled rules
|
10
|
+
- `Compiler#errors_json` and `Compiler#warnings_json` for detailed diagnostics
|
11
|
+
- **NEW**: Added rule serialization and deserialization support
|
12
|
+
- `Scanner.from_serialized` - Create scanner from serialized rules
|
13
|
+
- `Scanner.from_rules` - Create scanner from pre-compiled rules
|
14
|
+
- Enables compile-once, use-many-times pattern for production deployments
|
15
|
+
- **NEW**: Enhanced pattern matching analysis with `Yara::PatternMatch`
|
16
|
+
- Detailed pattern match information with offsets and lengths
|
17
|
+
- `PatternMatch#offset`, `PatternMatch#length`, `PatternMatch#matched_data`
|
18
|
+
- `ScanResult#matches_for_pattern` - Get matches for specific patterns
|
19
|
+
- `ScanResult#pattern_matched?` - Check if specific pattern matched
|
20
|
+
- `ScanResult#total_matches` and `ScanResult#all_matches` for match analysis
|
21
|
+
- **NEW**: Added comprehensive metadata and tag support
|
22
|
+
- Type-safe metadata accessors: `metadata_string`, `metadata_int`, `metadata_bool`
|
23
|
+
- `ScanResult#tags` - Access rule tags as array
|
24
|
+
- `ScanResult#has_tag?` - Check for specific tags
|
25
|
+
- `ScanResult#qualified_name` - Get namespaced rule name
|
26
|
+
- **NEW**: Added global variable support for scanners
|
27
|
+
- `Scanner#set_global_str`, `Scanner#set_global_int`, `Scanner#set_global_bool`, `Scanner#set_global_float`
|
28
|
+
- `Scanner#set_globals` - Bulk setting with error handling options
|
29
|
+
- Enable dynamic rule behavior based on runtime variables
|
30
|
+
- **NEW**: Added scanner timeout configuration via `Scanner#set_timeout`
|
31
|
+
- **IMPROVED**: Enhanced documentation with comprehensive usage examples in `USAGE.md`
|
32
|
+
- **IMPROVED**: Updated development documentation and AI coding instructions
|
33
|
+
|
34
|
+
## [4.0.0] - 2025-08-19
|
35
|
+
|
36
|
+
- **BREAKING**: Migrated from legacy libyara FFI bindings to YARA-X C API (`libyara_x_capi.so`) ([#24](https://github.com/jonmagic/yara-ffi/pull/24))
|
37
|
+
- Removed all legacy FFI struct definitions (`YrRule`, `YrMeta`, `YrString`, etc.)
|
38
|
+
- Replaced incremental rule compilation with single-step compilation via `yrx_compile`
|
39
|
+
- Eliminated dependency on `Yara.start` and `Yara.stop` lifecycle methods
|
40
|
+
- **BREAKING**: Changed `Scanner#call` to `Scanner#scan` method ([#24](https://github.com/jonmagic/yara-ffi/pull/24))
|
41
|
+
- **BREAKING**: Require Ruby >= 3.0.0 ([#24](https://github.com/jonmagic/yara-ffi/pull/24))
|
42
|
+
- **BREAKING**: Remove `ScanResult` return for non-matching scans ([#24](https://github.com/jonmagic/yara-ffi/pull/24))
|
43
|
+
- Added `Yara::ScanResults` enumerable collection for managing scan results ([#24](https://github.com/jonmagic/yara-ffi/pull/24))
|
44
|
+
- Added `Scanner.open` for block-based resource management ([#24](https://github.com/jonmagic/yara-ffi/pull/24))
|
45
|
+
- Added streaming scan API support with block yielding ([#24](https://github.com/jonmagic/yara-ffi/pull/24))
|
46
|
+
- Modernized CI workflow with Ruby 3.0-3.3 matrix testing and YARA-X build support ([#24](https://github.com/jonmagic/yara-ffi/pull/24))
|
47
|
+
- Added comprehensive development documentation in `DEVELOPMENT.md` ([#24](https://github.com/jonmagic/yara-ffi/pull/24))
|
48
|
+
- Updated Docker environment to Ruby 3.3 with YARA-X v1.5.0 ([#24](https://github.com/jonmagic/yara-ffi/pull/24))
|
49
|
+
- Improved error handling for compilation and scanning with better exception handling
|
50
|
+
- Preserved backward compatibility in `ScanResult` interface via fallback parsing
|
51
|
+
- Removed obsolete helper files: `user_data.rb`, `yr_meta.rb`, `yr_string.rb`, `yr_namespace.rb`, `yr_rule.rb`
|
52
|
+
|
53
|
+
## [3.1.0] - 2022-04-18
|
54
|
+
|
55
|
+
- Minor documentation fix for `Scanner::call` return value ([#20](https://github.com/jonmagic/yara-ffi/pull/20))
|
56
|
+
- Fix FFI type compatibility issues on ARM64 Linux by converting integer types ([#21](https://github.com/jonmagic/yara-ffi/pull/21))
|
57
|
+
|
58
|
+
## [3.0.0] - 2021-10-21
|
59
|
+
|
60
|
+
- **BREAKING**: Introduced new `Yara::Scanner` API for better memory management and control ([#17](https://github.com/jonmagic/yara-ffi/pull/17))
|
61
|
+
- Added proper memory cleanup with `yr_compiler_destroy` and `yr_rules_destroy` calls
|
62
|
+
- Moved core functionality to `Yara::Scanner` class
|
63
|
+
|
64
|
+
## [2.1.1] - 2021-08-31
|
65
|
+
|
66
|
+
- Fix memory leak by calling destroy methods ([#11](https://github.com/jonmagic/yara-ffi/pull/11))
|
67
|
+
|
68
|
+
## [2.1.0] - 2021-08-30
|
69
|
+
|
70
|
+
- Use struct hash access and `Struct.ptr` where possible ([#14](https://github.com/jonmagic/yara-ffi/pull/14))
|
71
|
+
- Improved struct member access and performance optimizations
|
72
|
+
|
73
|
+
## [2.0.1] - 2021-08-30
|
74
|
+
|
75
|
+
- Bug fixes and improvements
|
76
|
+
|
77
|
+
## [2.0.0] - 2021-08-24
|
78
|
+
|
79
|
+
- **BREAKING**: Changed interface to support rule metas ([#4](https://github.com/jonmagic/yara-ffi/pull/4))
|
80
|
+
- `Yara.test` now returns `Yara::ScanResult` objects instead of rule names
|
81
|
+
- Added support for accessing rule metadata as hash of name => value
|
82
|
+
- Return rule metas in scan results
|
83
|
+
|
84
|
+
## [1.0.0] - 2021-08-16
|
85
|
+
|
86
|
+
- Wire up basic Yara functionality ([#3](https://github.com/jonmagic/yara-ffi/pull/3))
|
87
|
+
- Added `Yara.test(rules_string, string_to_scan)` functionality
|
88
|
+
- Initial FFI bindings to libyara
|
89
|
+
|
3
90
|
## [0.1.0] - 2021-03-11
|
4
91
|
|
5
|
-
- Initial release
|
92
|
+
- Initial release with project structure ([#1](https://github.com/jonmagic/yara-ffi/pull/1), [#2](https://github.com/jonmagic/yara-ffi/pull/2))
|
93
|
+
- Set up GitHub Actions CI
|
94
|
+
- Configured RuboCop
|
data/DEVELOPMENT.md
ADDED
@@ -0,0 +1,188 @@
|
|
1
|
+
# Development Guide
|
2
|
+
|
3
|
+
This guide covers setting up the development environment and working on the yara-ffi gem.
|
4
|
+
|
5
|
+
## Requirements
|
6
|
+
|
7
|
+
- Docker (for containerized development environment)
|
8
|
+
|
9
|
+
## Quick Start
|
10
|
+
|
11
|
+
After checking out the repo, run the bootstrap script to set up the development environment:
|
12
|
+
|
13
|
+
```bash
|
14
|
+
script/bootstrap
|
15
|
+
```
|
16
|
+
|
17
|
+
This will build a Docker image with all the necessary dependencies, including the YARA-X C API library.
|
18
|
+
|
19
|
+
## Development Scripts
|
20
|
+
|
21
|
+
The project includes several convenience scripts in the `script/` directory:
|
22
|
+
|
23
|
+
- `script/bootstrap` - Sets up the development environment (builds Docker image)
|
24
|
+
- `script/test` - Runs the test suite in the Docker container
|
25
|
+
|
26
|
+
## Running Tests
|
27
|
+
|
28
|
+
To run the full test suite:
|
29
|
+
|
30
|
+
```bash
|
31
|
+
script/test
|
32
|
+
```
|
33
|
+
|
34
|
+
This runs `bundle exec rake` inside the Docker container with all dependencies properly configured.
|
35
|
+
|
36
|
+
You can also run tests manually inside the container:
|
37
|
+
|
38
|
+
```bash
|
39
|
+
docker run -it --mount type=bind,src="$(pwd)",dst=/app yara-ffi bundle exec rake
|
40
|
+
```
|
41
|
+
|
42
|
+
Or run specific test files:
|
43
|
+
|
44
|
+
```bash
|
45
|
+
docker run -it --mount type=bind,src="$(pwd)",dst=/app yara-ffi bundle exec ruby -Itest test/scanner_test.rb
|
46
|
+
```
|
47
|
+
|
48
|
+
Alternatively, you can use rake to run specific tests:
|
49
|
+
|
50
|
+
```bash
|
51
|
+
docker run -it --mount type=bind,src="$(pwd)",dst=/app yara-ffi bundle exec rake test TEST=test/scanner_test.rb
|
52
|
+
```
|
53
|
+
|
54
|
+
## Interactive Development
|
55
|
+
|
56
|
+
For an interactive development session, you can start a console in the container:
|
57
|
+
|
58
|
+
```bash
|
59
|
+
docker run -it --mount type=bind,src="$(pwd)",dst=/app yara-ffi bin/console
|
60
|
+
```
|
61
|
+
|
62
|
+
This gives you an IRB session with the gem loaded for experimentation.
|
63
|
+
|
64
|
+
## Development Environment Details
|
65
|
+
|
66
|
+
The development environment uses Docker to provide a consistent setup with:
|
67
|
+
|
68
|
+
- Ruby 3.3 (latest stable)
|
69
|
+
- YARA-X C API library v1.5.0 built from source
|
70
|
+
- All necessary system dependencies
|
71
|
+
- Bundler with locked gem versions
|
72
|
+
|
73
|
+
### Docker Image
|
74
|
+
|
75
|
+
The `Dockerfile` sets up:
|
76
|
+
|
77
|
+
1. Base Ruby 3.3 image
|
78
|
+
2. System dependencies (curl, git, unzip)
|
79
|
+
3. Rust toolchain and cargo-c for building YARA-X
|
80
|
+
4. YARA-X C API library compiled and installed
|
81
|
+
5. Ruby gem dependencies via Bundler
|
82
|
+
|
83
|
+
### Manual Setup (without Docker)
|
84
|
+
|
85
|
+
If you prefer not to use Docker, you'll need to manually install:
|
86
|
+
|
87
|
+
1. Ruby 3.0+
|
88
|
+
2. YARA-X C API library (see [Installation section in README](README.md#installing-yara-x))
|
89
|
+
3. System dependencies for building native gems
|
90
|
+
|
91
|
+
Then run:
|
92
|
+
|
93
|
+
```bash
|
94
|
+
bundle install
|
95
|
+
rake test
|
96
|
+
```
|
97
|
+
|
98
|
+
## Code Structure
|
99
|
+
|
100
|
+
The gem is organized as follows:
|
101
|
+
|
102
|
+
- `lib/yara.rb` - Main entry point and convenience methods
|
103
|
+
- `lib/yara/ffi.rb` - FFI bindings to YARA-X C API
|
104
|
+
- `lib/yara/scanner.rb` - Scanner class for rule compilation and scanning
|
105
|
+
- `lib/yara/scan_result.rb` - Individual scan result wrapper
|
106
|
+
- `lib/yara/scan_results.rb` - Collection of scan results
|
107
|
+
- `lib/yara/version.rb` - Gem version constant
|
108
|
+
|
109
|
+
## Testing
|
110
|
+
|
111
|
+
Tests are located in the `test/` directory:
|
112
|
+
|
113
|
+
- `test/yara_test.rb` - Tests for main module convenience methods
|
114
|
+
- `test/scanner_test.rb` - Tests for Scanner class functionality
|
115
|
+
- `test/test_helper.rb` - Shared test setup and utilities
|
116
|
+
|
117
|
+
The test suite uses Minitest and includes tests for:
|
118
|
+
|
119
|
+
- Rule compilation and validation
|
120
|
+
- Data scanning with various rule types
|
121
|
+
- Memory management and resource cleanup
|
122
|
+
- Error handling and edge cases
|
123
|
+
|
124
|
+
## Release Process
|
125
|
+
|
126
|
+
To release a new version of the gem:
|
127
|
+
|
128
|
+
1. Update the version number in `lib/yara/version.rb`
|
129
|
+
2. Update the `CHANGELOG.md` with release notes
|
130
|
+
3. Commit the changes
|
131
|
+
4. Create and push a git tag:
|
132
|
+
```bash
|
133
|
+
git tag v<version>
|
134
|
+
git push origin v<version>
|
135
|
+
```
|
136
|
+
5. Build and push the gem:
|
137
|
+
```bash
|
138
|
+
gem build yara-ffi.gemspec
|
139
|
+
gem push yara-ffi-<version>.gem
|
140
|
+
```
|
141
|
+
|
142
|
+
## Contributing Guidelines
|
143
|
+
|
144
|
+
1. Fork the repository
|
145
|
+
2. Create a feature branch (`git checkout -b my-new-feature`)
|
146
|
+
3. Make your changes with appropriate tests
|
147
|
+
4. Run the test suite (`script/test`) to ensure all tests pass
|
148
|
+
5. Commit your changes (`git commit -am 'Add some feature'`)
|
149
|
+
6. Push to the branch (`git push origin my-new-feature`)
|
150
|
+
7. Create a Pull Request
|
151
|
+
|
152
|
+
Please ensure your code follows the existing style and includes tests for new functionality.
|
153
|
+
|
154
|
+
## Debugging
|
155
|
+
|
156
|
+
For debugging FFI-related issues:
|
157
|
+
|
158
|
+
1. Enable FFI debugging by setting the `RUBY_FFI_DEBUG` environment variable
|
159
|
+
2. Use `puts` statements or `binding.pry` (if pry is available) for Ruby debugging
|
160
|
+
3. Check YARA-X C API documentation for expected behavior
|
161
|
+
4. Verify memory management - ensure all resources are properly freed
|
162
|
+
|
163
|
+
## Common Issues
|
164
|
+
|
165
|
+
### YARA-X Library Not Found
|
166
|
+
|
167
|
+
If you see errors about missing YARA-X libraries, ensure:
|
168
|
+
|
169
|
+
1. The YARA-X C API library is properly installed
|
170
|
+
2. The library path is in your system's library search path
|
171
|
+
3. You're using the correct version (v1.5.0 is tested)
|
172
|
+
|
173
|
+
### Docker Build Issues
|
174
|
+
|
175
|
+
If Docker builds fail:
|
176
|
+
|
177
|
+
1. Ensure you have sufficient disk space
|
178
|
+
2. Try rebuilding without cache: `docker build --no-cache . -t yara-ffi`
|
179
|
+
3. Check internet connectivity for downloading dependencies
|
180
|
+
|
181
|
+
### Test Failures
|
182
|
+
|
183
|
+
If tests fail:
|
184
|
+
|
185
|
+
1. Ensure all dependencies are properly installed
|
186
|
+
2. Check that YARA-X C API library is available
|
187
|
+
3. Verify Ruby version compatibility (3.0+ required)
|
188
|
+
4. Run tests individually to isolate issues
|