xre 0.5.2-x86_64-darwin → 0.5.4-x86_64-darwin

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '06999ead5c27d2fb25d186e60d7e2864962483298471fc831005c6493bee00dc'
-  data.tar.gz: 303748e91080f002e3c216ed21810f9ecbcc7976888285e068c887ff56b5ee1e
+  metadata.gz: 2820731414ce8e3b3700dfbcb2adf943a9d6e12246dee1e205fb09c45a8b0ca6
+  data.tar.gz: 30de4b46256476eee21cf39b5e2ec036354dd52ee9a061b477c21a3d871b6844
 SHA512:
-  metadata.gz: 7e08ac53642bb3dfcbc7f9c7bf7002aff065de5f7aaac6d95689f508ce169ce19a0fcf167bf02efcd9d3df5d5b58eb28faab4a5c5f84313a02d01e3cc9e8e244
-  data.tar.gz: b1f2057e4ef1a2fd5536a022fc5efab88711e5bb5bde1a86d99abbe559b5ed35484cf88c9f79f893e59f276ee2da770c2275f5ff957be1f45fe95a7bef0288e2
+  metadata.gz: c135babf92993097270aa1cfe8877878f5b14745bd4628f4d89550b30f46239eee2b368f65faeb00667b5b6100e4ef0f6c8b45f811d3c63f718ea458bf174ea3
+  data.tar.gz: 3db4bc888d0e2296cdb09a547c4037754bbe4eccd8476433dbd3dc2919f89def86c0282953d11f4b5eed77a8f40faad1faa006fd607d6033ad4f2c88c8f0eafe

data/README.md

@@ -3,18 +3,57 @@
 This is a rust extension for finding all the matches in a text.
 It's a simple extension that is used in one part of our codebase and this extension makes it substantially faster.
 
-The following text assumes you're in the `xre` directory inside `clearscope` project.
+## Why?
 
-## How to build rust code localy:
+The main reason for this extension is that we have a unique use case where we need to find all captures
+with their respectful offsets in a large text and we need to do this for a huge number of regexes and texts.
+This is a slow operation in ruby since the regular `scan` does not provide the actual captures,
+but only the offsets. This extension provides the captures and offsets in a single call for a
+list of regexes to avoid the overhead of calling the regex engine multiple times.
+
+Moreover, another part of the problem is that we also require the surrounding text of the capture
+with some radius, which is also not possible with the regular `scan` method without resorting to iterating
+the whole text multiple times.
+
+> 🤓
+> Another a bit more technical reason for why the rust extension is faster is that ruby strings are
+> indexed in characters(`O(n)`), but rust strings are indexed in bytes(`O(1)`). And with a careful
+> iteration over the characters on the rust side(see `regex_list.rs#captures_with_context` and `utils.rs#find_char_index`) as we go through the text we can avoid multiple(`n`)
+> `O(n)` operations that ruby would have to do, reducing the algorithmic complexity from
+> `O(n^2)` to `O(n)` for one regex, and from `O(m * n^2)` to `O(m * n)`(where `m` and `n` are the number
+> of regexes and the number of texts respectfuly) for multiple regexes.
+
+## Developing:
+
+The simplest way to develop the gem locally without reinstalling the gem is to just:
+
+1. change the
+   `gem "xre"` in the `Gemfile` to `gem "xre", path: "xre"`
+2. run `bundle install`
+
+and you're good to go. Just change the code(then compile the rust code if you changed it, more on that below),
+go into `rails c` and hack away.
+
+Another simple way is to:
+
+1. change the gem code(then, once again, compile the rust code if you changed it)
+2. go into `rails c` or the console of your choosing
+3. run `require_relative "xre/lib/xre"`
+
+all done.
+
+> 💡Note: _The following part of the readme assumes that you're in the `xre` directory inside `clearscope` project._
+
+### How to build rust code locally:
 
 ```bash
-rake compile
+rake compile # or just rake
 ```
 
 sometimes you might need to clean the build:
 
 ```bash
-rake clean
+rake clean && rake compile
 ```
 
 you might need to install rust toolchain for that:
@@ -23,32 +62,34 @@ you might need to install rust toolchain for that:
 curl https://sh.rustup.rs -sSf | sh
 ```
 
-## How to run tests:
-
-### Ruby tests:
+### How to run tests:
 
-Our ruby tests live in the regular `spec` directory of the project, so inside `clearscope` run:
+**Ruby tests:**
 
 ```bash
-bundle exec rspec spec/xre
+bundle exec rake spec
 ```
 
-### Rust tests:
+**Rust tests:**
 
 ```bash
-cargo test
+cargo test # you will need rust toolchain for that
 ```
 
-## How to publish the gem:
+**Rust linting:**
 
-First, increment the version in the `lib/xre/version.rb` file according to the semantic versioning rules.
+````bash
+# you will need rust toolchain for that
+cargo fmt # will format the code
+cargo clippy # will suggest improvements, not only style ones
+```
 
-Then build the gem and push it to rubygems:
+## Publishing:
 
-```bash
-rake build
-gem push pkg/xre-<version>-gem
-```
+The codebase includes `xre/Rakefile` which in turn defines a `gem:native`
+task that compiles the extension for the `x86_64-linux`, `x86_64-darwin`, and `arm64-darwin`
+and puts the compiled gems into `xre/pkg/` directory, from where one should
+`gem push xre-<version>-x86_64-linux.gem` to publish the gem.
 
-> Note, currently only [`barseek`](https://github.com/Vagab) has access to the rubygems account, so you might need to ask him to publish the gem.
-> As soon as this changes and we get a team rubygems account the README will be updated.
+> Note: You will need a docker installed on your machine for that
+````

data/lib/xre/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Xre
-  VERSION = "0.5.2"
+  VERSION = "0.5.4"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: xre
 version: !ruby/object:Gem::Version
-  version: 0.5.2
+  version: 0.5.4
 platform: x86_64-darwin
 authors:
 - barseek
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2024-04-19 00:00:00.000000000 Z
+date: 2024-04-29 00:00:00.000000000 Z
 dependencies: []
 description: 
 email:
@@ -27,7 +27,6 @@ licenses:
 metadata:
   homepage_uri: https://github.com/vagab/xre
   source_code_uri: https://github.com/vagab/xre
-  rubygems_mfa_required: 'true'
 post_install_message: 
 rdoc_options: []
 require_paths: