ruby-next-core 0.15.2 → 1.0.0.rc.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c392536e2de07a838a8c9818b0651a9d2eb6803471f52d1e3632bada3fb398a4
-  data.tar.gz: 564822b7a5c55ec7fc278bf223206f0b16a75a71a9277b247d862489c7a342cb
+  metadata.gz: 898de1113136ed6652275e5a83ee7e2866bf287ec1533dee71ffefdaf4f8fcae
+  data.tar.gz: 5ccd6bb1f8256c048d9bd18246b0ab372770500111ed265c775ab37498d6814c
 SHA512:
-  metadata.gz: 33d80d8d5bfaaf75cc0cc42b79eb4758ff57c9d72f1ac21eb86770f1f039c79cebc7eabc5b4a20c42e11a9a266b8a435849b2a4ae74547acd6b9a8fb8db03f39
-  data.tar.gz: 1a9b048897ad763e2dd726bc0be7112337499dbb1ddad9e02f13b81a2bb98d28d6f32f2e9abe1e29222a05895433117e082a5a8e42be4101d4d4eb056bcf5c61
+  metadata.gz: 36876dea9517b88f1390bfe72781f77b13f00c5020079476f485f90642b8d8ec66061e2f3693f1cb2c350e9ccdb3a63f1e4c965b4f7010e36cbb226e5a8e7682
+  data.tar.gz: 24d8e22d98e9a3a33a29237b27b02e195bea98d6dda5d1fadb3f5a0ba5a9664bf205b783299f6ed7aeda6879396129a590dbe3aaf7a9e65072b9c4ef9caddfef

data/CHANGELOG.md

@@ -2,6 +2,36 @@
 
 ## master
 
+- Add `it` block parameter support. ([@palkan][])
+
+- Deprecate AST transpiling mode. ([@palkan][])
+
+- Add `Data` backport. ([@palkan][], [@saturnflyer][])
+
+- Add `Time#deconstruct_keys` support. ([@palkan][])
+
+- Add anonymous rest and keyword rest arguments forwarding support (`def foo(*, **); bar(*, **) end`). ([@palkan][])
+
+- Add `MatchData#{deconstruct,deconstruct_keys}` and `#named_captures(symbolize_names: true)` support. ([@palkan][])
+
+- Add stats to `nextify`. ([@palkan][])
+
+- Add text rewriters and `--import-rewriter` option to `nextify`. ([@palkan][])
+
+- Use `.rbnextrc` arguments for runtime transpiling and auto-transpiling gems. ([@palkan][])
+
+- Stop testing Ruby <2.3. ([@palkan][])
+
+- Extract `require-hooks`. ([@palkan][])
+
+- Support passing --overwrite to CLI. ([@prog-supdex][])
+
+Use `nextify original_file --single-version --overwrite` to overwrite the original file without create new one.
+
+## 0.15.3 (2022-10-16)
+
+- Fix handling nested const patterns. ([@palkan][])
+
 ## 0.15.2 (2022-08-02)
 
 - Fix loading transpiled in TruffleRuby. ([@palkan][])
@@ -360,3 +390,5 @@ p a #=> 1
 [backports]: https://github.com/marcandre/backports
 [@sl4vr]: https://github.com/sl4vr
 [@skryukov]: https://github.com/skryukov
+[@prog-supdex]: https://github.com/prog-supdex
+[@saturnflyer]: https://github.com/saturnflyer

data/README.md

@@ -14,11 +14,14 @@ Who might be interested in Ruby Next?
 
 - **Ruby gems maintainers** who want to write code using the latest Ruby version but still support older ones.
 - **Application developers** who want to give new features a try without waiting for the final release (or, more often, for the first patch).
-- **Users of non-MRI implementations** such as [mruby][], [JRuby][], [TruffleRuby][], [Opal][], [RubyMotion][], [Artichoke][], [Prism][].
+- **Users of non-MRI implementations** such as [mruby][], [JRuby][], [TruffleRuby][], [Natalie][], [Opal][], [RubyMotion][], [Artichoke][], [Prism][].
+- **Ruby syntax enthusiasts** who want to experiment with [custom syntax extensions](#custom-syntax-rewriters) 👩‍🔬👨‍🔬.
 
 Ruby Next also aims to help the community to assess new, _experimental_, MRI features by making it easier to play with them.
 That's why Ruby Next implements the `master` features as fast as possible.
 
+See also a companion library (extracted from Ruby Next) that provides **code loading hooks** for your needs—[require-hooks][require-hooks].
+
 Read more about the motivation behind the Ruby Next in this post: [Ruby Next: Make all Rubies quack alike](https://evilmartians.com/chronicles/ruby-next-make-all-rubies-quack-alike).
 
 <table style="border:none;">
@@ -60,14 +63,13 @@ _Please, submit a PR to add your project to the list!_
 ## Table of contents
 
 - [Overview](#overview)
-- [Quick Start](#quick-start)
+- [Quick start](#quick-start)
 - [Polyfills](#using-only-polyfills)
 - [Transpiling](#transpiling)
   - [Modes](#transpiler-modes)
   - [CLI](#cli)
   - [Using in gems](#integrating-into-a-gem-development)
   - [Runtime usage](#runtime-usage)
-  - [Bootsnap integration](#using-with-bootsnap)
   - [`ruby -ruby-next`](#uby-next)
   - [Logging & Debugging](#logging-and-debugging)
 - [RuboCop](#rubocop)
@@ -75,6 +77,7 @@ _Please, submit a PR to add your project to the list!_
 - [Using with Pry](#pry)
 - [Using with EOL Rubies](#using-with-eol-rubies)
 - [Proposed & edge features](#proposed-and-edge-features)
+- [Custom syntax rewriters](#custom-syntax-rewriters)
 - [Known limitations](#known-limitations)
 
 ## Overview
@@ -86,9 +89,9 @@ Core provides **polyfills** for Ruby core classes APIs via Refinements (default
 Language is responsible for **transpiling** edge Ruby syntax into older versions. It could be done
 programmatically or via CLI. It also could be done in runtime.
 
-Currently, Ruby Next supports Ruby versions 2.2+, including JRuby 9.2.8+ and TruffleRuby 20.1+ (with some limitations). Support for EOL versions (<2.5) slightly differs though ([see below](#using-with-eol-rubies)).
+Currently, Ruby Next supports Ruby versions 2.3+, including JRuby 9.2.8+ and TruffleRuby 20.1+ (with some limitations). Support for older versions (<2.5) slightly differs though ([see below](#using-with-eol-rubies)). Versions between 2.0 and 2.3 may work but we no longer test against them.
 
-Please, [open an issue](https://github.com/ruby-next/ruby-next/issues/new/choose) or join the discussion in the existing ones if you would like us to support older Ruby versions.
+Please [open an issue](https://github.com/ruby-next/ruby-next/issues/new/choose) or join the discussion in the existing issues if you would like us to support older Ruby versions.
 
 ## Quick start
 
@@ -100,15 +103,16 @@ $ gem install ruby-next
 
 # Call ruby with -ruby-next flag
 $ ruby -ruby-next -e "
-def greet(val) =
-  case val
+greet = proc do
+  case it
     in hello: hello if hello =~ /human/i
       '🙂'
     in hello: 'martian'
       '👽'
     end
+end
 
-puts greet(hello: 'martian')
+puts greet.call(hello: 'martian')
 "
 
 => 👽
@@ -126,9 +130,9 @@ gem "ruby-next-core"
 spec.add_dependency "ruby-next-core"
 ```
 
-**NOTE:** we use the different _distribution_ gem, `ruby-next-core`, to provide zero-dependency, polyfills-only version.
+**NOTE:** we use a different gem for _distribution_, `ruby-next-core`, to provide a zero-dependency, polyfills-only version.
 
-Then, all you need is to load the Ruby Next:
+Then, all you need is to load Ruby Next:
 
 ```ruby
 require "ruby-next"
@@ -140,9 +144,9 @@ And activate the refinement in every file where you want to use it\*:
 using RubyNext
 ```
 
-Ruby Next only refines core classes if necessary; thus, this line wouldn't have any effect in the edge Ruby.
+Ruby Next only refines core classes if necessary; thus, this line wouldn't have any effect in edge Ruby.
 
-**NOTE:** Even if the runtime already contains a monkey-patch with the backported functionality, we consider the method as _dirty_ and activate the refinement for it. Thus, you always have predictable behaviour. That's why we recommend using refinements for gems development.
+**NOTE:** Even if the runtime already contains a monkey-patch with the backported functionality, we consider the method as _dirty_ and activate the refinement for it. Thus, you always have predictable behaviour. That's why we recommend using refinements for gem development.
 
 Alternatively, you can go with monkey-patches. Just add this line:
 
@@ -152,7 +156,7 @@ require "ruby-next/core_ext"
 
 The following _rule of thumb_ is recommended when choosing between refinements and monkey-patches:
 
-- Use refinements for libraries development (to avoid conflicts with others code)
+- Use refinements for library development (to avoid conflicts with others' code)
 - Using core extensions could be considered for application development (no need to think about `using RubyNext`); this approach could potentially lead to conflicts with dependencies (if these dependencies are not using refinements 🙂)
 - Use core extensions if refinements are not supported by your platform
 
@@ -160,6 +164,12 @@ The following _rule of thumb_ is recommended when choosing between refinements a
 
 [**The list of supported APIs.**][features_core]
 
+### Data backport
+
+Ruby 3.2 has introduced a new core class—[Data](https://bugs.ruby-lang.org/issues/16122). Ruby Next provides a backport functionality which is automatically activated when you `require "ruby-next"` **if and only if the constant is undefined**. If you want to use a custom backport, make sure you loaded it first.
+
+If you want to opt-out from loading Data backport, you must set the `RUBY_NEXT_DISABLE_DATA` env variable to `true`.
+
 ## Transpiling
 
 Ruby Next allows you to transpile\* edge Ruby syntax to older versions.
@@ -187,21 +197,11 @@ gem install ruby-next
 
 ### Transpiler modes
 
-Ruby Next currently provides two different modes of generating transpiled code: _AST_ and _rewrite_.
-
-In the AST mode, we parse the source code into AST, modifies this AST and **generate a new code from AST** (using [unparser][unparser]). Thus, the transpiled code being identical in terms of functionality has different formatting.
-
-In the rewrite mode, we apply changes to the source code itself, thus, keeping the original formatting of the unaffected code (in a similar way to RuboCop's autocorrect feature).
+Since v1.0, Ruby Next only support the _rewrite_ mode, i.e., the code transformations are applied directly to the original source code. This allows us to keep formatting as close as possible to the original code.
 
 The main benefit of the rewrite mode is that it preserves the original code line numbers and layout, which is especially useful in debugging.
 
-By default, we use the rewrite mode. If you found a bug with rewrite mode which is not reproducible in the AST mode, please, let us know.
-
-You can change the transpiler mode:
-
-- From code by setting `RubyNext::Language.mode = :ast` or `RubyNext::Language.mode = :rewrite`.
-- Via environmental variable `RUBY_NEXT_TRANSPILE_MODE=ast`.
-- Via CLI option ([see below](#cli)).
+The legacy AST mode (regenerating source code from the modified abstract syntax tree) is deprecated (though still supported).
 
 ## CLI
 
@@ -219,12 +219,14 @@ Usage: ruby-next nextify DIRECTORY_OR_FILE [options]
     -o, --output=OUTPUT              Specify output directory or file or stdout
         --min-version=VERSION        Specify the minimum Ruby version to support
         --single-version             Only create one version of a file (for the earliest Ruby version)
+        --overwrite                  Overwrites the original file with one version of --single-version (works only with --single-version or --rewrite)
         --edge                       Enable edge (master) Ruby features
         --proposed                   Enable proposed/experimental Ruby features
         --transpile-mode=MODE        Transpiler mode (ast or rewrite). Default: ast
         --[no-]refine                Do not inject `using RubyNext`
         --list-rewriters             List available rewriters
         --rewrite=REWRITERS...       Specify particular Ruby features to rewrite
+        --import-rewriter=PATHS...   Specify the paths to custom rewriters to load
     -h, --help                       Print help
     -V                               Turn on verbose mode
         --dry-run                    Print verbose output without generating files
@@ -302,6 +304,8 @@ nextify: |
   --edge
 ```
 
+**NOTE:** The `nextify` section is also used by auto-transpiling when installing the gem from source and by runtime transpiling.
+
 ## Integrating into a gem development
 
 We recommend _pre-transpiling_ source code to work with older versions before releasing it.
@@ -361,19 +365,21 @@ This feature, _auto-transpiling_, is **disabled** by default (will likely be ena
 
 It is also possible to transpile Ruby source code in run-time via Ruby Next.
 
-All you need is to `require "ruby-next/language/runtime"` as early as possible to hijack `Kernel#require` and friends.
-You can also automatically inject `using RubyNext` to every\* loaded file by also adding `require "ruby-next/core/runtime"`.
+All you need is to `require "ruby-next/language/runtime"` to hijack `Kernel#require` and friends before loading the files you want to transpile. You can also automatically inject `using RubyNext` to every\* loaded file by also adding `require "ruby-next/core/runtime"`.
 
-Since the runtime mode requires Kernel monkey-patching, it should be used carefully. For example, we use it in Ruby Next tests—works perfectly. But think twice before enabling it in production.
+Runtime mode is backed by [require-hooks][require-hooks]—a standalone gem which has been extracted from Ruby Next. Depending on the current runtime, it picks an optimal strategy for hijacking the loading mechanism. Please, refer to its documentation for more details.
 
-Consider using [Bootsnap](#using-with-bootsnap) integration, 'cause its monkey-patching has been bullet-proofed 😉.
+\* Ruby Next doesn't hijack every required file but only the configured directories: `./app/`, `./lib/`, `./spec/`, `./test/` (relative to the `pwd`). It also excludes the `./vendor/bundle` directory by default.
 
-\* Ruby Next doesn't hijack every required file but _watches_ only the configured directories: `./app/`, `./lib/`, `./spec/`, `./test/` (relative to the `pwd`). You can configure the watch dirs:
+You can customize target files via the `include_patterns` and `exclude_patterns` configuration options:
 
 ```ruby
-RubyNext::Language.watch_dirs << "path/to/other/dir"
+RubyNext::Language.include_patterns << "path/to/other/dir/*.rb"
+RubyNext::Language.exclude_patterns << "path/to/other/dir/subdir/*"
 ```
 
+**NOTE:** Directories MUST be configured before requiring `ruby-next/language/runtime`.
+
 ### Eval & similar
 
 By default, we do not hijack `Kernel.eval` and similar methods due to some limitations (e.g., there is no easy and efficient way to access the caller's scope, or _binding_, and some evaluations relies on local variables).
@@ -384,20 +390,6 @@ If you want to support transpiling in `eval`-like methods, opt-in explicitly by
 using RubyNext::Language::Eval
 ```
 
-## Using with Bootsnap
-
-[Bootsnap][] is a great tool to speed-up your application load and it's included into the default Rails Gemfile. It patches Ruby mechanism of loading source files to make it possible to cache the intermediate representation (_iseq_).
-
-Ruby Next provides a specific integration which allows to add a transpiling step to this process, thus making the transpiler overhead as small as possible, because the cached and **already transpiled** version is used if no changes were made.
-
-To enable this integration, add the following line after the `require "bootsnap/setup"`:
-
-```ruby
-require "ruby-next/language/bootsnap"
-```
-
-**NOTE:** There is no way to invalidate the cache when you upgrade Ruby Next (e.g., due to the bug fixes), so you should do this manually.
-
 ## `uby-next`
 
 _This is [not a typo](https://github.com/ruby-next/ruby-next/pull/8), that’s the way `ruby -ruby-next` works: it’s equal to `ruby -r uby-next`, and [`uby-next.rb`](https://github.com/ruby-next/ruby-next/blob/master/lib/uby-next.rb) is a special file that activates the runtime mode._
@@ -490,7 +482,7 @@ pry -ruby-next/pry
 
 ## Using with EOL Rubies
 
-We currently provide support for Ruby 2.2, 2.3 and 2.4.
+We currently provide support for Ruby 2.3+.
 
 **NOTE:** By "support" here we mean using `ruby-next` CLI and runtime transpiling. Transpiled code may run on Ruby 2.0+.
 
@@ -557,13 +549,85 @@ require "ruby-next/language/runtime"
 
 ### Supported edge features
 
-It's too early, Ruby 3.1 has just been released. See its features in the [supported features list](./SUPPORTED_FEATURES.md).
+- Implicit `it` block parameter ([#19890](https://bugs.ruby-lang.org/issues/18980)).
 
 ### Supported proposed features
 
 - _Method reference_ operator (`.:`) ([#13581](https://bugs.ruby-lang.org/issues/13581)).
 - Binding non-local variables in pattern matching (`42 => @v`) ([#18408](https://bugs.ruby-lang.org/issues/18408)).
 
+## Custom syntax rewriters
+
+Wonder what would happen if Ruby get a null coalescing operator (`??=`) or some other syntactic feature you want to try out? Ruby Next is here to help you!
+
+Ruby Next allows you to write your own syntax rewriters. Full-featured rewriters (used by Ruby Next itself) operate on AST and usually require parser modifications. However, we also support text-based rewriters which can be used to experiment with new syntax much quicker without dealing with grammars, parsers and syntax trees.
+
+To implement a text-based rewriter, you need to create a new class inherited from `RubyNext::Language::Rewriters::Text` and implementing either `#rewrite` or `#safe_rewrite` method. For example, the method reference operator (`.:`) could be implemented as follows:
+
+```ruby
+class MethodReferenceRewriter < RubyNext::Language::Rewriters::Text
+  # Rewriter configuration includes its name, a syntax probe and a minimum supported Ruby version.
+  # The latter two are used to determine whether the rewriter should be activated for the current file in runtime or when running `ruby-next nextify`.
+  NAME = "method-reference"
+  SYNTAX_PROBE = "Language.:transform"
+  MIN_SUPPORTED_VERSION = Gem::Version.new(RubyNext::NEXT_VERSION)
+
+  def safe_rewrite(source)
+    source.gsub(/\.:([\w_]+)/) do |match|
+      context.track! self
+
+      ".method(:#{$1})"
+    end
+  end
+end
+
+# Add the rewriter to the list of rewriters
+RubyNext::Language.rewriters << MethodReferenceRewriter
+```
+
+The `context` object is responsible for tracking if the rewriter was used for the current file. You must call the `context.track!` method to mark the file as _dirty_ (i.e., it should be transpiled). The input parameter (`source`) is the Ruby source code of the file being transpiled and the output must be the transpiled source code.
+
+The `#safe_rewrite` method operates on the normalized source code (i.e., without comments and string literals). It's useful when you want to avoid transpiling inside strings or comments. If you want to transpile the original contents, you can use the `#rewrite` method instead.
+
+Under the hood, `#safe_rewrite` uses [Paco][] to parse the source and separate string literals from the rest of the code. You can also leverage [Paco][] in your text rewriters, if you want more control on the parsing process. For better experience, we provide a DSL to define a custom parser and the `#parse` method to use it. Here is an example of implementing the `.:` operator using a Paco parser:
+
+```ruby
+class MethodReferenceRewriter < RubyNext::Language::Rewriters::Text
+  NAME = "method-reference"
+  SYNTAX_PROBE = "Language.:transform"
+
+  parser do
+    def default
+      many(
+        alt(
+          method_ref,
+          any_char
+        )
+      )
+    end
+
+    def method_ref
+      seq(
+        string(".:").result(""),
+        method_name
+      # IMPORTANT: Use `#track!` method to mark the file as dirty
+      ).fmap { track! }.fmap { ".method(:#{_1})" }
+    end
+
+    def method_name = regexp(/[\w_]+/)
+  end
+
+  def safe_rewrite(source)
+    parse(source).join
+  end
+end
+
+# Add the rewriter to the list of rewriters
+RubyNext::Language.rewriters << MethodReferenceRewriter
+```
+
+When using the `ruby-next nextify` command, you can load custom rewriters via the `--import-rewriter` option.
+
 ## Known limitations
 
 Ruby Next aims to be _reasonably compatible_ with MRI. That means, some edge cases could be uncovered. Below is the list of known limitations.
@@ -596,6 +660,10 @@ Bug reports and pull requests are welcome on GitHub at [https://github.com/ruby-
 
 See also the [development guide](./DEVELOPMENT.md).
 
+## Acknowledgments
+
+- Thanks to [Jim Gay](https://github.com/saturnflyer) for the original Data polyfill implementation ([polyfill-data](https://github.com/saturnflyer/polyfill-data))
+
 ## License
 
 The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
@@ -613,6 +681,8 @@ The gem is available as open source under the terms of the [MIT License](https:/
 [parser]: https://github.com/whitequark/parser
 [unparser]: https://github.com/mbj/unparser
 [next_parser]: https://github.com/ruby-next/parser
-[Bootsnap]: https://github.com/Shopify/bootsnap
 [rubocop]: https://github.com/rubocop-hq/rubocop
 [backports]: https://github.com/marcandre/backports
+[require-hooks]: https://github.com/ruby-next/require-hooks
+[Natalie]: https://natalie-lang.org
+[Paco]: https://github.com/ruby-next/paco

data/bin/mspec

@@ -0,0 +1,11 @@
+#!/usr/bin/env ruby
+
+unless File.directory?('mspec/lib')
+  $stdout.puts "Cloning mspec..."
+  system("git clone https://github.com/ruby/mspec.git mspec")
+end
+
+$:.unshift File.expand_path(File.join(__dir__, '..', 'mspec', 'lib'))
+
+require 'mspec/commands/mspec'
+MSpecMain.main(false)