mongory 0.6.3 → 0.7.1

data/SUBMODULE_INTEGRATION.md

@@ -0,0 +1,325 @@
+# Mongory-rb Submodule Integration Guide
+
+This document describes how to integrate `mongory-core` as a Git submodule into `mongory-rb` to enable a high-performance C extension.
+
+## Architecture Overview
+
+```
+mongory-rb/
+├── lib/                    # Ruby code
+│   ├── mongory.rb         # Entry, loads the C extension
+│   └── mongory/           # Other Ruby modules
+├── ext/                   # C extension
+│   └── mongory_ext/
+│       ├── mongory-core/  # Git submodule (sources only; no prior CMake build required)
+│       ├── extconf.rb     # Build configuration (compiles submodule .c sources directly)
+│       └── mongory_ext.c  # Ruby C wrapper
+└── scripts/
+    └── build_with_core.sh # (Optional) build script
+```
+
+## Quick Start
+
+### 1. Clone and initialize
+
+```bash
+# Clone the project
+git clone <your-mongory-rb-repo>
+cd mongory-rb
+
+# Initialize the submodule
+git submodule update --init --recursive
+```
+
+### 2. Install system dependencies
+
+Installing `mongory-rb` (with the C extension) only requires basic build tools and Ruby headers:
+
+**macOS:**
+```bash
+xcode-select --install   # Install Xcode Command Line Tools (includes clang)
+```
+
+**Ubuntu/Debian:**
+```bash
+sudo apt update
+sudo apt install build-essential ruby-dev
+```
+
+**CentOS/RHEL/Fedora:**
+```bash
+sudo yum groupinstall "Development Tools" || sudo dnf groupinstall "Development Tools"
+sudo yum install ruby-devel          || sudo dnf install ruby-devel
+```
+
+> Optional: CMake and cJSON are only needed if you want to run the core tests/benchmarks inside the `mongory-core` submodule (e.g., via CMake/ctest).
+
+### 3. Build the project
+
+Using Rake (recommended):
+
+```bash
+# Automatically initializes the submodule and builds the C extension (no prior CMake build needed)
+bundle exec rake build_all
+
+# If rake-compiler is not available, the fallback :compile will directly run extconf.rb + make
+```
+
+Or use the script (optional):
+
+```bash
+./scripts/build_with_core.sh         # Basic build
+./scripts/build_with_core.sh --debug # Debug build
+./scripts/build_with_core.sh --help  # Show all options
+```
+
+## Detailed Steps
+
+### Submodule management
+
+```bash
+# Initialize submodule
+rake submodule:init
+
+# Update submodule to latest
+rake submodule:update
+
+# Manually update submodule
+git submodule update --remote
+```
+
+### Manual build flow (without rake-compiler)
+
+If you prefer to control the build manually:
+
+```bash
+# 1. Ensure submodule is initialized
+git submodule update --init --recursive
+
+# 2. (Optional) Only needed if you run core tests/benchmarks inside the submodule
+#    Note: This step requires cJSON; typical mongory-rb usage does not.
+# cd ext/mongory_ext/mongory-core
+# ./build.sh --test
+# cd ../../..
+
+# 3. Build the Ruby C extension (compiles submodule sources directly via extconf.rb)
+cd ext/mongory_ext
+ruby extconf.rb
+make
+cd ../..
+
+# 4. Run tests
+bundle exec rspec
+```
+
+### Clean builds
+
+```bash
+# Clean all build artifacts
+rake clean_all
+
+# Or use the build script
+./scripts/build_with_core.sh --clean
+```
+
+## Development Guide
+
+### Modify the C extension
+
+1. Edit `ext/mongory_ext/mongory_ext.c`
+2. Rebuild:
+   ```bash
+   cd ext/mongory_ext
+   make
+   ```
+
+### Update mongory-core
+
+1. Enter the submodule directory:
+   ```bash
+   cd ext/mongory_ext/mongory-core
+   ```
+
+2. Checkout the desired version or branch:
+   ```bash
+   git checkout main
+   git pull origin main
+   ```
+
+3. Return to the main project and commit the submodule update:
+   ```bash
+   cd ../../..
+   git add ext/mongory_ext/mongory-core
+   git commit -m "Update mongory-core submodule"
+   ```
+
+### Debug the C extension
+
+```bash
+# Build in debug mode
+export DEBUG=1
+cd ext/mongory_ext
+ruby extconf.rb
+make
+
+# Or use the build script
+./scripts/build_with_core.sh --debug
+```
+
+Debug mode will:
+- Enable debug symbols (`-g`)
+- Disable optimizations (`-O0`)
+- Define the `DEBUG` macro
+
+### C extension API (current public surface)
+
+```ruby
+require 'mongory'
+
+condition = { "age" => { "$gt" => 18 } }
+matcher   = Mongory::CMatcher.new(condition)
+
+data   = { "name" => "John", "age" => 25 }
+result = matcher.match(data)  # => true/false
+
+# Optional: get explanation (runs explain on the C side, currently returns nil)
+matcher.explain
+```
+
+## Troubleshooting
+
+### Common issues
+
+**1. Submodule is empty**
+```bash
+# Fix
+git submodule update --init --recursive
+```
+
+**2. cJSON library not found (only required if you run mongory-core CMake tests)**
+```bash
+# macOS
+brew install cjson
+
+# Ubuntu/Debian
+sudo apt install libcjson-dev
+
+# Verify installation
+pkg-config --exists libcjson && echo "Found" || echo "Not found"
+```
+
+**3. Build errors**
+```bash
+# Clean and rebuild
+./scripts/build_with_core.sh --clean
+./scripts/build_with_core.sh --debug
+```
+
+**4. C extension fails to load**
+- Check Ruby version compatibility (>= 2.6.0)
+- Ensure all system deps are installed
+- Inspect error messages and compiler warnings
+
+### Logging and debugging
+
+Enable verbose output:
+```bash
+# Verbose during build
+VERBOSE=1 ./scripts/build_with_core.sh
+
+# Debug info when loading Ruby
+RUBY_DEBUG=1 ruby -rmongory -e "puts Mongory::CoreInterface.version"
+```
+
+### Performance check
+
+```ruby
+require 'benchmark'
+require 'mongory'
+
+records = 10000.times.map { |i| { "id" => i, "age" => rand(18..65) } }
+
+
+```
+
+## CI/CD integration
+
+### GitHub Actions example
+
+```yaml
+name: Build and Test
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        ruby-version: ['2.6', '2.7', '3.0', '3.1']
+
+    steps:
+    - uses: actions/checkout@v3
+      with:
+        submodules: recursive
+
+    - name: Set up Ruby
+      uses: ruby/setup-ruby@v1
+      with:
+        ruby-version: ${{ matrix.ruby-version }}
+        bundler-cache: true
+
+    - name: Install system dependencies
+      run: |
+        sudo apt update
+        sudo apt install -y build-essential ruby-dev
+
+    - name: Build with C extension
+      run: bundle exec rake build_all
+
+    - name: Run tests
+      run: bundle exec rspec
+
+    # Only needed if you also run mongory-core C tests/benchmarks (optional):
+    # - name: Install CMake & cJSON (only if running core tests)
+    #   run: sudo apt install -y cmake libcjson-dev
+    # - name: Build mongory-core tests (optional)
+    #   run: |
+    #     cd ext/mongory_ext/mongory-core
+    #     ./build.sh --test
+```
+
+## Contributing Guide
+
+### Development workflow
+
+1. Fork this repository
+2. Create a feature branch: `git checkout -b feature/amazing-feature`
+3. If you modify C code, make sure to update tests
+4. Run the full test suite: `./scripts/build_with_core.sh`
+5. Commit changes: `git commit -m 'Add amazing feature'`
+6. Push the branch: `git push origin feature/amazing-feature`
+7. Open a Pull Request
+
+### Coding standards
+
+- **C code**: C99, follow mongory-core style
+- **Ruby code**: follow project RuboCop config
+- **Docs**: document all public APIs
+
+### Testing requirements
+
+- Add tests for all new features
+- Ensure both C extension and Ruby fallback are covered
+- Run benchmarks to prevent regressions
+
+## References
+
+- [mongory-core doc](https://github.com/mongoryhq/mongory-core)
+- [Ruby C extension doc](https://docs.ruby-lang.org/en/master/extension_rdoc.html)
+- [Git Submodules doc](https://git-scm.com/book/en/v2/Git-Tools-Submodules)
+- [CMake doc](https://cmake.org/documentation/)
+
+## License
+
+This integration follows the MIT license, consistent with mongory-core and mongory-rb.

data/docs/advanced_usage.md

@@ -0,0 +1,40 @@
+# Advanced Usage
+
+## Complex Queries
+
+```ruby
+# Nested conditions
+users.mongory
+  .where(
+    :age.gte => 18,
+    :$or => [
+      { :status => 'active' },
+      { :status => 'pending', :created_at.gte => 1.week.ago }
+    ]
+  )
+
+# Using any_of for nested OR conditions
+users.mongory
+  .where(:age.gte => 18)
+  .any_of(
+    { :status => 'active' },
+    { :status => 'pending', :created_at.gte => 1.week.ago }
+  )
+
+# Array operations
+posts.mongory
+  .where(:tags.elem_match => { :name => 'ruby', :priority.gt => 5 })
+  .where(:comments.every => { :approved => true })
+```
+
+## Integration with ActiveRecord
+
+```ruby
+class User < ActiveRecord::Base
+  def active_friends
+    friends.mongory
+      .where(:status => 'active')
+      .where(:last_seen.gte => 1.day.ago)
+  end
+end
+```

data/docs/clang_bridge.md

@@ -0,0 +1,69 @@
+# Clang Bridge (C Extension)
+
+The Clang bridge connects the Ruby DSL to the `mongory-core` engine via a compact C layer. It exposes two key entry points:
+
+- `Mongory::CMatcher`: a low-level matcher API backed by C.
+- `QueryBuilder#c`: an ergonomic switch that reuses your current Ruby condition and executes it through `CMatcher`.
+
+## Build/Install
+
+```bash
+bundle install
+bundle exec rake compile
+# or, when installing the gem, the extension will compile automatically if toolchain is present
+```
+
+## Quick check
+
+```ruby
+require 'mongory'
+if defined?(Mongory::CMatcher)
+  puts 'C extension available'
+else
+  puts 'C extension not available, using pure Ruby'
+end
+```
+
+## Basic usage
+
+```ruby
+records = [
+  { 'name' => 'Jack', 'age' => 18 },
+  { 'name' => 'Jill', 'age' => 15 },
+  { 'name' => 'Bob',  'age' => 21 }
+]
+
+# Switch existing Ruby query to C path
+query = records.mongory.c # => returns Mongory::CQueryBuilder
+  .where(:age.gte => 18)
+
+query.each.to_a        # enumerate matches via C
+query.fast.to_a        # alias of each
+query.trace.to_a       # print value compare progression
+query.explain          # print core-level matcher tree
+
+# Or use CMatcher directly
+matcher = Mongory::CMatcher.new(:age.gte => 18)
+records.select { |r| matcher.match?(r) }
+```
+
+## Tracing and debugging
+
+```ruby
+matcher = Mongory::CMatcher.new(:$or => [ { :name.regex => /^J/ }, { :age.gt => 20 } ])
+matcher.enable_trace
+records.each { |r| matcher.match?(r) }
+matcher.print_trace    # prints detailed trace
+matcher.disable_trace
+
+# Or trace single record compare progression
+matcher.trace(records.first)
+```
+
+## Notes
+
+- Regexes use Ruby's `Regexp` internally; string patterns are compiled once and cached.
+- Context (`Mongory::Utils::Context`) is shared between Ruby and C during matching, enabling custom converters.
+- If the extension fails to load, `Mongory::CQueryBuilder` is unavailable and `.c` will not be used; the Ruby path continues to work.
+
+

data/docs/field_names.md

@@ -0,0 +1,30 @@
+# Handling Dots in Field Names
+
+Mongory supports field names containing dots, which require escaping:
+
+```ruby
+# Sample data
+records = [
+  { "user.name" => "John", "age" => 25 },  # Field name contains a dot
+  { "user" => { "name" => "Bob" }, "age" => 30 }  # Nested field
+]
+
+# Field name contains a dot
+records.mongory.where("user\\.name" => "John")  # Two backslashes needed with double quotes
+# => [{ "user.name" => "John", "age" => 25 }]
+
+# or
+records.mongory.where('user\.name' => "John")   # One backslash needed with single quotes
+# => [{ "user.name" => "John", "age" => 25 }]
+
+# Nested field (no escaping needed)
+records.mongory.where("user.name" => "Bob")
+# => [{ "user" => { "name" => "Bob" }, "age" => 30 }]
+```
+
+Notes:
+- With double quotes, backslashes need to be escaped (`\\`)
+- With single quotes, backslashes don't need to be escaped (`\`)
+- This behavior is consistent with MongoDB's query syntax
+- The escaped dot pattern (`\.`) matches fields where the dot is part of the field name
+- The unescaped dot pattern (`.`) matches nested fields in the document structure

data/docs/migration.md

@@ -0,0 +1,30 @@
+# Migration Guide
+
+## From Array#select
+```ruby
+# Before
+records.select { |r| r['age'] >= 18 && r['status'] == 'active' }
+
+# After
+records.mongory.where(:age.gte => 18, status: 'active')
+```
+
+## From ActiveRecord
+```ruby
+# Before
+indexed_query.where("age >= ? AND status = ?", 18, 'active')
+
+# After
+indexed_query.mongory.where(:age.gte => 18, status: 'active')
+```
+
+## From MongoDB
+```ruby
+# Before (MongoDB)
+users.where(:age.gte => 18, status: 'active')
+
+# After (Mongory)
+users.mongory.where(:age.gte => 18, status: 'active')
+
+# Just the same.
+```

data/docs/performance.md

@@ -0,0 +1,61 @@
+# Performance & Benchmarks
+
+## C Extension vs Pure Ruby
+
+- C extension provides 3-10x performance improvement for large datasets
+- Automatic fallback to pure Ruby if C extension unavailable
+- Check availability: `defined?(Mongory::CMatcher)` or attempt to require the extension
+- Memory management handled by mongory-core's memory pool
+
+## Memory Usage
+
+- Mongory operates entirely in memory
+- Consider your data size and memory constraints
+- Proc-based implementation reduces memory usage
+- Context system provides better memory management
+
+## Query Optimization
+
+- Complex conditions are evaluated in sequence
+- Use `explain` to analyze query performance
+- Empty conditions are optimized with cached Procs
+- Context system allows fine-grained control over conversion
+
+## Benchmarks
+
+```ruby
+  # Plain Ruby Simple query (100000 records)
+  records.select { |r| r['age'].is_a?(Numeric) && r['age'] >= 18 } # ~9ms
+
+  # Plain Ruby Complex query (100000 records)
+  records.select do |r|
+    next false unless r.key?('age') && r.key?('status')
+
+    r['age'].is_a?(Numeric) && r['age'] >= 18 ||  r['status'] == 'active'
+  end # ~20ms
+
+  # Simple query (100000 records)
+  records.mongory.where(:age.gte => 18) # ~119ms
+
+  # Complex query (100000 records)
+  records.mongory.where(:$or => [{:age.gte => 18}, {:status => 'active'}]) # ~107ms
+
+  # Complex query with fast mode (100000 records)
+  records.mongory.where(:$or => [{:age.gte => 18}, {:status => 'active'}]).fast # ~63ms
+
+  # Simple query with C extension (100000 records)
+  records.mongory.c.where(:age.gte => 18) # ~15ms
+  # Complex query with C extension (100000 records)
+  records.mongory.c.where(:$or => [{:age.gte => 18}, {:status => 'active'}]) # ~23ms, same with plain ruby
+```
+
+Note: Performance varies based on:
+
+- Data size
+- Query complexity
+- Hardware specifications
+- Ruby version
+
+Benchmark in your environment to validate.
+
+