ed-precompiled_prism 1.5.2-arm64-darwin

data/docs/local_variable_depth.md

@@ -0,0 +1,229 @@
+# Local variable depth
+
+One feature of Prism is that it resolves local variables as it parses. It's necessary to do this because of ambiguities in the grammar. For example, consider the following code:
+
+```ruby
+foo / bar#/
+```
+
+If `foo` is a local variable, this is a call to `/` with `bar` as an argument, followed by a comment. If it's not a local variable, this is a method call to `foo` with a regular expression argument.
+
+"Depth" refers to the number of visible scopes that Prism has to go up to find the declaration of a local variable.
+Note that this follows the same scoping rules as Ruby, so a local variable is only visible in the scope it is declared in and in blocks nested in that scope.
+The rules for calculating the depth are very important to understand because they may differ from individual Ruby implementations since they are not specified by the language.
+
+Prism uses the minimum number of scopes, i.e., it only creates scopes when necessary semantically, in other words when there must be distinct scopes (which can be observed through `binding.local_variables`).
+That are no "transparent/invisible" scopes in Prism.
+Some Ruby implementations use those for some language constructs and need to adjust by maintaining a depth offset.
+
+Below are the places where a local variable can be written/targeted, along with how the depth is calculated at that point.
+
+## General
+
+In the course of general Ruby code when reading a local variable, the depth is equal to the number of scopes to go up to find the declaration of that variable. For example:
+
+```ruby
+foo = 1
+bar = 2
+baz = 3
+
+foo # depth 0
+tap { bar } # depth 1
+tap { tap { baz } } # depth 2
+```
+
+This also includes writing to a local variable, which could be writing to a local variable that is already declared. For example:
+
+```ruby
+foo = 1
+bar = 2
+
+foo = 3 # depth 0
+tap { bar = 4 } # depth 1
+```
+
+This includes multiple assignment, where the same principle applies. For example:
+
+```ruby
+foo = 1
+bar = 2
+
+foo, bar = 3, 4 # depth 0
+tap { foo, bar = 5, 6 } # depth 1
+```
+
+## `for` loops
+
+`for` loops in Ruby break down to calls to `.each` with a block.
+However in that case local variable reads and writes within the block will be in the same scope as the scope surrounding the `for` and not in a deeper/separate scope (surprising, but this is Ruby semantics).
+For example:
+
+```ruby
+foo = 1
+
+for e in baz
+  foo # depth 0
+  bar = 2 # depth 0
+end
+
+p bar # depth 0, prints 2
+```
+
+The local variable(s) used for the index of the `for` are also at the same depth (as variables inside and outside the `for`):
+
+```ruby
+for e in [1, 2] # depth 0
+  e # depth 0
+end
+
+p e # depth 0, prints 2
+```
+
+## Pattern matching captures
+
+You can target a local variable in a pattern matching expression using capture syntax. Using this syntax, you can target local variables in the current scope or in visible parent scopes. For example:
+
+```ruby
+42 => bar # depth 0
+```
+
+The example above writes to a local variable in the current scope. If the variable is already declared in a higher visible scope, it will be written to that scope instead. For example:
+
+```ruby
+foo = 1
+tap { 42 => foo } # depth 1
+```
+
+## Named capture groups
+
+You can target local variables through named capture groups in regular expressions if they are used on the left-hand side of a `=~` operator. For example:
+
+```ruby
+/(?<foo>\d+)/ =~ "42" # depth 0
+```
+
+This will write to a `foo` local variable. If the variable is already declared in a higher visible scope, it will be written to that scope instead. For example:
+
+```ruby
+foo = 1
+tap { /(?<foo>\d+)/ =~ "42" } # depth 1
+```
+
+## "interpolated once" regular expressions
+
+Regular expressions that interpolate local variables (unrelated to capture group local variables) and have the `o` flag will only interpolate the local variables once for the runtime of the program.
+In CRuby, this is implemented by compiling the regular expression within a nested instruction sequence, which means CRuby thinks the depth is one more than prism does. For example:
+
+```
+$ ruby --dump=insns -e 'foo = 1; /#{foo}/o'
+== disasm: #<ISeq:<main>@-e:1 (1,0)-(1,18)> (catch: false)
+local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
+[ 1] foo@0
+0000 putobject_INT2FIX_1_                                             (   1)[Li]
+0001 setlocal_WC_0                          foo@0
+0003 once                                   block in <main>, <is:0>
+0006 leave
+
+== disasm: #<ISeq:block in <main>@-e:1 (1,9)-(1,18)> (catch: false)
+0000 putobject                              ""                        (   1)
+0002 getlocal_WC_1                          foo@0
+0004 dup
+0005 objtostring                            <calldata!mid:to_s, argc:0, FCALL|ARGS_SIMPLE>
+0007 anytostring
+0008 toregexp                               0, 2
+0011 leave
+```
+
+In this case CRuby fetches the local variable with `getlocal_WC_1` as the second instruction to the "once" instruction sequence. When compiling CRuby, prism therefore will adjust the depth to account for this difference.
+
+## `rescue` clauses
+
+In CRuby, `rescue` clauses are implemented as their own instruction sequence, and therefore CRuby thinks the depth is one more than prism does. For example:
+
+```
+$ ruby --dump=insns -e 'begin; foo = 1; rescue; foo; end'
+== disasm: #<ISeq:<main>@-e:1 (1,0)-(1,32)> (catch: true)
+== catch table
+| catch type: rescue st: 0000 ed: 0004 sp: 0000 cont: 0005
+| == disasm: #<ISeq:rescue in <main>@-e:1 (1,16)-(1,28)> (catch: true)
+| local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
+| [ 1] $!@0
+| 0000 getlocal_WC_0                          $!@0                      (   1)
+| 0002 putobject                              StandardError
+| 0004 checkmatch                             3
+| 0006 branchunless                           11
+| 0008 getlocal_WC_1                          foo@0[Li]
+| 0010 leave
+| 0011 getlocal_WC_0                          $!@0
+| 0013 throw                                  0
+| catch type: retry  st: 0004 ed: 0005 sp: 0000 cont: 0000
+|------------------------------------------------------------------------
+local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
+[ 1] foo@0
+0000 putobject_INT2FIX_1_                                             (   1)[Li]
+0001 dup
+0002 setlocal_WC_0                          foo@0
+0004 nop
+0005 leave
+```
+
+In the catch table, CRuby is reading the `foo` local variable using `getlocal_WC_1` as the fifth instruction to the "rescue" instruction sequence. When compiling CRuby, prism therefore will adjust the depth to account for this difference.
+
+Note that this includes the error reference, which can target local variables, as in:
+
+```
+$ ruby --dump=insns -e 'foo = 1; begin; rescue => foo; end'              
+== disasm: #<ISeq:<main>@-e:1 (1,0)-(1,34)> (catch: true)
+== catch table
+| catch type: rescue st: 0003 ed: 0004 sp: 0000 cont: 0005
+| == disasm: #<ISeq:rescue in <main>@-e:1 (1,16)-(1,30)> (catch: true)
+| local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
+| [ 1] $!@0
+| 0000 getlocal_WC_0                          $!@0                      (   1)
+| 0002 putobject                              StandardError
+| 0004 checkmatch                             3
+| 0006 branchunless                           14
+| 0008 getlocal_WC_0                          $!@0
+| 0010 setlocal_WC_1                          foo@0
+| 0012 putnil
+| 0013 leave
+| 0014 getlocal_WC_0                          $!@0
+| 0016 throw                                  0
+| catch type: retry  st: 0004 ed: 0005 sp: 0000 cont: 0003
+|------------------------------------------------------------------------
+local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
+[ 1] foo@0
+0000 putobject_INT2FIX_1_                                             (   1)[Li]
+0001 setlocal_WC_0                          foo@0
+0003 putnil
+0004 nop
+0005 leave
+```
+
+Note that CRuby is writing to the `foo` local variable using the `setlocal_WC_1` instruction as the sixth instruction to the "rescue" instruction sequence. When compiling CRuby, prism therefore will adjust the depth to account for this difference.
+
+## Post execution blocks
+
+The `END {}` syntax allows executing code when the program exits. In CRuby, this is implemented as two nested instruction sequences. CRuby therefore thinks the depth is two more than prism does. For example:
+
+```
+$ ruby --dump=insns -e 'foo = 1; END { foo }'            
+== disasm: #<ISeq:<main>@-e:1 (1,0)-(1,20)> (catch: false)
+local table (size: 1, argc: 0 [opts: 0, rest: -1, post: 0, block: -1, kw: -1@-1, kwrest: -1])
+[ 1] foo@0
+0000 putobject_INT2FIX_1_                                             (   1)[Li]
+0001 setlocal_WC_0                          foo@0
+0003 once                                   block in <main>, <is:0>
+0006 leave
+
+== disasm: #<ISeq:block in <main>@-e:0 (0,0)-(-1,-1)> (catch: false)
+0000 putspecialobject                       1                         (   1)
+0002 send                                   <calldata!mid:core#set_postexe, argc:0, FCALL>, block in <main>
+0005 leave
+
+== disasm: #<ISeq:block in <main>@-e:1 (1,9)-(1,20)> (catch: false)
+0000 getlocal                               foo@0, 2                  (   1)[LiBc]
+0003 leave                                  [Br]
+```
+
+In the instruction sequence corresponding to the code that gets executed inside the `END` block, CRuby is reading the `foo` local variable using `getlocal` as the second instruction to the `"block in <main>"` instruction sequence. When compiling CRuby, prism therefore will adjust the depth to account for this difference.

data/docs/mapping.md

@@ -0,0 +1,117 @@
+# Mapping
+
+When considering the previous CRuby parser versus prism, this document should be helpful to understand how various concepts are mapped.
+
+## Nodes
+
+The following table shows how the various CRuby nodes are mapped to prism nodes.
+
+| CRuby | prism |
+| --- | --- |
+| `NODE_SCOPE` | |
+| `NODE_BLOCK` | |
+| `NODE_IF` | `PM_IF_NODE` |
+| `NODE_UNLESS` | `PM_UNLESS_NODE` |
+| `NODE_CASE` | `PM_CASE_NODE` |
+| `NODE_CASE2` | `PM_CASE_NODE` (with a null predicate) |
+| `NODE_CASE3` | |
+| `NODE_WHEN` | `PM_WHEN_NODE` |
+| `NODE_IN` | `PM_IN_NODE` |
+| `NODE_WHILE` | `PM_WHILE_NODE` |
+| `NODE_UNTIL` | `PM_UNTIL_NODE` |
+| `NODE_ITER` | `PM_CALL_NODE` (with a non-null block) |
+| `NODE_FOR` | `PM_FOR_NODE` |
+| `NODE_FOR_MASGN` | `PM_FOR_NODE` (with a multi-write node as the index) |
+| `NODE_BREAK` | `PM_BREAK_NODE` |
+| `NODE_NEXT` | `PM_NEXT_NODE` |
+| `NODE_REDO` | `PM_REDO_NODE` |
+| `NODE_RETRY` | `PM_RETRY_NODE` |
+| `NODE_BEGIN` | `PM_BEGIN_NODE` |
+| `NODE_RESCUE` | `PM_RESCUE_NODE` |
+| `NODE_RESBODY` | |
+| `NODE_ENSURE` | `PM_ENSURE_NODE` |
+| `NODE_AND` | `PM_AND_NODE` |
+| `NODE_OR` | `PM_OR_NODE` |
+| `NODE_MASGN` | `PM_MULTI_WRITE_NODE` |
+| `NODE_LASGN` | `PM_LOCAL_VARIABLE_WRITE_NODE` |
+| `NODE_DASGN` | `PM_LOCAL_VARIABLE_WRITE_NODE` |
+| `NODE_GASGN` | `PM_GLOBAL_VARIABLE_WRITE_NODE` |
+| `NODE_IASGN` | `PM_INSTANCE_VARIABLE_WRITE_NODE` |
+| `NODE_CDECL` | `PM_CONSTANT_PATH_WRITE_NODE` |
+| `NODE_CVASGN` | `PM_CLASS_VARIABLE_WRITE_NODE` |
+| `NODE_OP_ASGN1` | |
+| `NODE_OP_ASGN2` | |
+| `NODE_OP_ASGN_AND` | `PM_OPERATOR_AND_ASSIGNMENT_NODE` |
+| `NODE_OP_ASGN_OR` | `PM_OPERATOR_OR_ASSIGNMENT_NODE` |
+| `NODE_OP_CDECL` | |
+| `NODE_CALL` | `PM_CALL_NODE` |
+| `NODE_OPCALL` | `PM_CALL_NODE` (with an operator as the method) |
+| `NODE_FCALL` | `PM_CALL_NODE` (with a null receiver and parentheses) |
+| `NODE_VCALL` | `PM_CALL_NODE` (with a null receiver and parentheses or arguments) |
+| `NODE_QCALL` | `PM_CALL_NODE` (with a &. operator) |
+| `NODE_SUPER` | `PM_SUPER_NODE` |
+| `NODE_ZSUPER` | `PM_FORWARDING_SUPER_NODE` |
+| `NODE_LIST` | `PM_ARRAY_NODE` |
+| `NODE_ZLIST` | `PM_ARRAY_NODE` (with no child elements) |
+| `NODE_VALUES` | `PM_ARGUMENTS_NODE` |
+| `NODE_HASH` | `PM_HASH_NODE` |
+| `NODE_RETURN` | `PM_RETURN_NODE` |
+| `NODE_YIELD` | `PM_YIELD_NODE` |
+| `NODE_LVAR` | `PM_LOCAL_VARIABLE_READ_NODE` |
+| `NODE_DVAR` | `PM_LOCAL_VARIABLE_READ_NODE` |
+| `NODE_GVAR` | `PM_GLOBAL_VARIABLE_READ_NODE` |
+| `NODE_IVAR` | `PM_INSTANCE_VARIABLE_READ_NODE` |
+| `NODE_CONST` | `PM_CONSTANT_PATH_READ_NODE` |
+| `NODE_CVAR` | `PM_CLASS_VARIABLE_READ_NODE` |
+| `NODE_NTH_REF` | `PM_NUMBERED_REFERENCE_READ_NODE` |
+| `NODE_BACK_REF` | `PM_BACK_REFERENCE_READ_NODE` |
+| `NODE_MATCH` | |
+| `NODE_MATCH2` | `PM_CALL_NODE` (with regular expression as receiver) |
+| `NODE_MATCH3` | `PM_CALL_NODE` (with regular expression as only argument) |
+| `NODE_LIT` | |
+| `NODE_STR` | `PM_STRING_NODE` |
+| `NODE_DSTR` | `PM_INTERPOLATED_STRING_NODE` |
+| `NODE_XSTR` | `PM_X_STRING_NODE` |
+| `NODE_DXSTR` | `PM_INTERPOLATED_X_STRING_NODE` |
+| `NODE_EVSTR` | `PM_STRING_INTERPOLATED_NODE` |
+| `NODE_DREGX` | `PM_INTERPOLATED_REGULAR_EXPRESSION_NODE` |
+| `NODE_ONCE` | |
+| `NODE_ARGS` | `PM_PARAMETERS_NODE` |
+| `NODE_ARGS_AUX` | |
+| `NODE_OPT_ARG` | `PM_OPTIONAL_PARAMETER_NODE` |
+| `NODE_KW_ARG` | `PM_KEYWORD_PARAMETER_NODE` |
+| `NODE_POSTARG` | `PM_REQUIRED_PARAMETER_NODE` |
+| `NODE_ARGSCAT` | |
+| `NODE_ARGSPUSH` | |
+| `NODE_SPLAT` | `PM_SPLAT_NODE` |
+| `NODE_BLOCK_PASS` | `PM_BLOCK_ARGUMENT_NODE` |
+| `NODE_DEFN` | `PM_DEF_NODE` (with a null receiver) |
+| `NODE_DEFS` | `PM_DEF_NODE` (with a non-null receiver) |
+| `NODE_ALIAS` | `PM_ALIAS_NODE` |
+| `NODE_VALIAS` | `PM_ALIAS_NODE` (with a global variable first argument) |
+| `NODE_UNDEF` | `PM_UNDEF_NODE` |
+| `NODE_CLASS` | `PM_CLASS_NODE` |
+| `NODE_MODULE` | `PM_MODULE_NODE` |
+| `NODE_SCLASS` | `PM_S_CLASS_NODE` |
+| `NODE_COLON2` | `PM_CONSTANT_PATH_NODE` |
+| `NODE_COLON3` | `PM_CONSTANT_PATH_NODE` (with a null receiver) |
+| `NODE_DOT2` | `PM_RANGE_NODE` (with a .. operator) |
+| `NODE_DOT3` | `PM_RANGE_NODE` (with a ... operator) |
+| `NODE_FLIP2` | `PM_RANGE_NODE` (with a .. operator) |
+| `NODE_FLIP3` | `PM_RANGE_NODE` (with a ... operator) |
+| `NODE_SELF` | `PM_SELF_NODE` |
+| `NODE_NIL` | `PM_NIL_NODE` |
+| `NODE_TRUE` | `PM_TRUE_NODE` |
+| `NODE_FALSE` | `PM_FALSE_NODE` |
+| `NODE_ERRINFO` | |
+| `NODE_DEFINED` | `PM_DEFINED_NODE` |
+| `NODE_POSTEXE` | `PM_POST_EXECUTION_NODE` |
+| `NODE_DSYM` | `PM_INTERPOLATED_SYMBOL_NODE` |
+| `NODE_ATTRASGN` | `PM_CALL_NODE` (with a message that ends with =) |
+| `NODE_LAMBDA` | `PM_LAMBDA_NODE` |
+| `NODE_ARYPTN` | `PM_ARRAY_PATTERN_NODE` |
+| `NODE_HSHPTN` | `PM_HASH_PATTERN_NODE` |
+| `NODE_FNDPTN` | `PM_FIND_PATTERN_NODE` |
+| `NODE_ERROR` | `PM_MISSING_NODE` |
+| `NODE_LAST` | |
+```

data/docs/parser_translation.md

@@ -0,0 +1,24 @@
+# parser translation
+
+Prism ships with the ability to translate its syntax tree into the syntax tree used by the [whitequark/parser](https://github.com/whitequark/parser) gem. This allows you to use tools built on top of the `parser` gem with the `prism` parser.
+
+## Usage
+
+The `parser` gem provides multiple parsers to support different versions of the Ruby grammar. This includes all of the Ruby versions going back to 1.8, as well as third-party parsers like MacRuby and RubyMotion. The `prism` gem provides another parser that uses the `prism` parser to build the syntax tree.
+
+You can use the `prism` parser like you would any other. After requiring `prism`, you should be able to call any of the regular `Parser::Base` APIs that you would normally use.
+
+```ruby
+require "prism"
+
+# Same as `Parser::Ruby34`
+Prism::Translation::Parser34.parse_file("path/to/file.rb")
+
+# Same as `Parser::CurrentRuby`
+Prism::Translation::ParserCurrent.parse("puts 'Hello World!'")
+```
+
+All the parsers are autoloaded, so you don't have to worry about requiring them yourself.
+
+If you also need to parse Ruby versions below 3.3 (for which the `prism` translation layer does not have explicit support), check out
+[this guide](https://github.com/whitequark/parser/blob/master/doc/PRISM_TRANSLATION.md) from the `parser` gem on how to use both in conjunction.

data/docs/parsing_rules.md

@@ -0,0 +1,22 @@
+# Rules
+
+This document contains information related to the rules of the parser for Ruby source code.
+
+As an example, in the documentation of many of the fields of nodes, it's mentioned that a field follows the lexing rules for `identifier` or `constant`. This document describes what those rules are.
+
+## Constants
+
+Constants in Ruby begin with an upper-case letter. This is followed by any number of underscores, alphanumeric, or non-ASCII characters. The definition of "alphanumeric" and "upper-case letter" are encoding-dependent.
+
+## Non-void expression
+
+Most expressions in CRuby are non-void. This means the expression they represent resolves to a value. For example, `1 + 2` is a non-void expression, because it resolves to a method call. Even things like `class Foo; end` is a non-void expression, because it returns the last evaluated expression in the body of the class (or `nil`).
+
+Certain nodes, however, are void expressions, and cannot be combined to form larger expressions.
+* `BEGIN {}`, `END {}`, `alias foo bar`, and `undef foo` can only be at a statement position.
+* The "jumps": `return`, `break`, `next`, `redo`, `retry` are void expressions.
+* `value => pattern` is also considered a void expression.
+
+## Identifiers
+
+Identifiers in Ruby begin with an underscore or lower-case letter. This is followed by any number of underscores, alphanumeric, or non-ASCII characters. The definition of "alphanumeric" and "lower-case letter" are encoding-dependent.

data/docs/releasing.md

@@ -0,0 +1,98 @@
+# Releasing
+
+To release a new version of Prism, perform the following steps:
+
+## Preparation
+
+* Update the `CHANGELOG.md` file.
+  * Add a new section for the new version at the top of the file.
+  * Fill in the relevant changes — it may be easiest to click the link for the `Unreleased` heading to find the commits.
+  * Update the links at the bottom of the file.
+* Update the version numbers in the various files that reference them:
+
+```sh
+export PRISM_MAJOR="x"
+export PRISM_MINOR="y"
+export PRISM_PATCH="z"
+export PRISM_VERSION="$PRISM_MAJOR.$PRISM_MINOR.$PRISM_PATCH"
+ruby -pi -e 'gsub(/spec\.version = ".+?"/, %Q{spec.version = "#{ENV["PRISM_VERSION"]}"})' prism.gemspec
+ruby -pi -e 'gsub(/EXPECTED_PRISM_VERSION ".+?"/, %Q{EXPECTED_PRISM_VERSION "#{ENV["PRISM_VERSION"]}"})' ext/prism/extension.h
+ruby -pi -e 'gsub(/PRISM_VERSION_MAJOR \d+/, %Q{PRISM_VERSION_MAJOR #{ENV["PRISM_MAJOR"]}})' include/prism/version.h
+ruby -pi -e 'gsub(/PRISM_VERSION_MINOR \d+/, %Q{PRISM_VERSION_MINOR #{ENV["PRISM_MINOR"]}})' include/prism/version.h
+ruby -pi -e 'gsub(/PRISM_VERSION_PATCH \d+/, %Q{PRISM_VERSION_PATCH #{ENV["PRISM_PATCH"]}})' include/prism/version.h
+ruby -pi -e 'gsub(/PRISM_VERSION ".+?"/, %Q{PRISM_VERSION "#{ENV["PRISM_VERSION"]}"})' include/prism/version.h
+ruby -pi -e 'gsub(/"version": ".+?"/, %Q{"version": "#{ENV["PRISM_VERSION"]}"})' javascript/package.json
+ruby -pi -e 'gsub(/lossy\(\), ".+?"/, %Q{lossy(), "#{ENV["PRISM_VERSION"]}"})' rust/ruby-prism-sys/tests/utils_tests.rs
+ruby -pi -e 'gsub(/\d+, "prism major/, %Q{#{ENV["PRISM_MAJOR"]}, "prism major})' templates/java/org/prism/Loader.java.erb
+ruby -pi -e 'gsub(/\d+, "prism minor/, %Q{#{ENV["PRISM_MINOR"]}, "prism minor})' templates/java/org/prism/Loader.java.erb
+ruby -pi -e 'gsub(/\d+, "prism patch/, %Q{#{ENV["PRISM_PATCH"]}, "prism patch})' templates/java/org/prism/Loader.java.erb
+ruby -pi -e 'gsub(/MAJOR_VERSION = \d+/, %Q{MAJOR_VERSION = #{ENV["PRISM_MAJOR"]}})' templates/javascript/src/deserialize.js.erb
+ruby -pi -e 'gsub(/MINOR_VERSION = \d+/, %Q{MINOR_VERSION = #{ENV["PRISM_MINOR"]}})' templates/javascript/src/deserialize.js.erb
+ruby -pi -e 'gsub(/PATCH_VERSION = \d+/, %Q{PATCH_VERSION = #{ENV["PRISM_PATCH"]}})' templates/javascript/src/deserialize.js.erb
+ruby -pi -e 'gsub(/MAJOR_VERSION = \d+/, %Q{MAJOR_VERSION = #{ENV["PRISM_MAJOR"]}})' templates/lib/prism/serialize.rb.erb
+ruby -pi -e 'gsub(/MINOR_VERSION = \d+/, %Q{MINOR_VERSION = #{ENV["PRISM_MINOR"]}})' templates/lib/prism/serialize.rb.erb
+ruby -pi -e 'gsub(/PATCH_VERSION = \d+/, %Q{PATCH_VERSION = #{ENV["PRISM_PATCH"]}})' templates/lib/prism/serialize.rb.erb
+ruby -pi -e 'gsub(/^version = ".+?"/, %Q{version = "#{ENV["PRISM_VERSION"]}"})' rust/ruby-prism-sys/Cargo.toml
+ruby -pi -e 'gsub(/^version = ".+?"/, %Q{version = "#{ENV["PRISM_VERSION"]}"})' rust/ruby-prism/Cargo.toml
+ruby -pi -e 'gsub(/^ruby-prism-sys = \{ version = ".+?"/, %Q{ruby-prism-sys = \{ version = "#{ENV["PRISM_VERSION"]}"})' rust/ruby-prism/Cargo.toml
+```
+
+* Update the `Gemfile.lock` file:
+
+```sh
+chruby ruby-3.5.0-dev
+bundle install
+```
+
+* Update the version-specific lockfiles:
+
+```sh
+for VERSION in "2.7" "3.0" "3.1" "3.2" "3.3" "3.4"; do docker run -it --rm -v "$PWD":/usr/src/app -w /usr/src/app -e BUNDLE_GEMFILE="gemfiles/$VERSION/Gemfile" "ruby:$VERSION" bundle update; done
+docker run -it --rm -v "$PWD":/usr/src/app -w /usr/src/app -e BUNDLE_GEMFILE="gemfiles/3.5/Gemfile" ruby:3.5.0-preview1 bundle update
+BUNDLE_GEMFILE=gemfiles/truffleruby/Gemfile chruby-exec truffleruby -- bundle update
+```
+
+* Update the cargo lockfiles:
+
+```sh
+bundle exec rake cargo:build
+```
+
+* Commit all of the updated files:
+
+```sh
+git commit -am "Bump to v$PRISM_VERSION"
+```
+
+* Push up the changes:
+
+```sh
+git push
+```
+
+## Publishing
+
+* Update the GitHub release page with a copy of the latest entry in the `CHANGELOG.md` file.
+* Publish the gem to [rubygems.org](rubygems.org). Note that you must have access to the `prism` gem to do this.
+
+```sh
+bundle exec rake release
+```
+
+* Generate the `wasm` artifact (or download it from GitHub actions and put it in `javascript/src/prism.wasm`).
+
+```sh
+make wasm
+```
+
+* Publish the JavaScript package to [npmjs.com](npmjs.com). Note that you must have access to the `@ruby/prism` package to do this.
+
+```sh
+npm publish
+```
+
+* Publish the rust crate to [crates.io](crates.io). Note that you must have access to the `ruby-prism-sys` and `ruby-prism` crates to do this.
+
+```sh
+bundle exec rake cargo:publish:real
+```

data/docs/relocation.md

@@ -0,0 +1,34 @@
+# Relocation
+
+Prism parses deterministically for the same input. This provides a nice property that is exposed through the `#node_id` API on nodes. Effectively this means that for the same input, these values will remain consistent every time the source is parsed. This means we can reparse the source same with a `#node_id` value and find the exact same node again.
+
+The `Relocation` module provides an API around this property. It allows you to "save" nodes and locations using a minimal amount of memory (just the node_id and a field identifier) and then reify them later. This minimizes the amount of memory you need to allocate to store this information because it does not keep around a pointer to the source string.
+
+## Getting started
+
+To get started with the `Relocation` module, you would first instantiate a `Repository` object. You do this through a DSL that chains method calls for configuration. For example, if for every entry in the repository you want to store the start and end lines, the start and end code unit columns for in UTF-16, and the leading comments, you would:
+
+```ruby
+repository = Prism::Relocation.filepath("path/to/file").lines.code_unit_columns(Encoding::UTF_16).leading_comments
+```
+
+Now that you have the repository, you can pass it into any of the `save*` APIs on nodes or locations to create entries in the repository that will be lazily reified.
+
+```ruby
+# assume that node is a Prism::ClassNode object
+entry = node.constant_path.save(repository)
+```
+
+Now that you have the entry object, you do not need to keep around a reference to the repository, it will be cleaned up on its own when the last entry is reified. Now, whenever you need to, you may call the associated field methods on the entry object, as in:
+
+```ruby
+entry.start_line
+entry.end_line
+
+entry.start_code_units_column
+entry.end_code_units_column
+
+entry.leading_comments
+```
+
+Note that if you had configured other fields to be saved, you would be able to access them as well. The first time one of these fields is accessed, the repository will reify every entry it knows about and then clean itself up. In this way, you can effectively treat them as if you had kept around lightweight versions of `Prism::Node` or `Prism::Location` objects.

data/docs/ripper_translation.md

@@ -0,0 +1,72 @@
+# Ripper translation
+
+Prism provides the ability to mirror the `Ripper` standard library. You can do this by:
+
+```ruby
+require "prism/translation/ripper/shim"
+```
+
+This provides the APIs like:
+
+```ruby
+Ripper.lex
+Ripper.parse
+Ripper.sexp_raw
+Ripper.sexp
+
+Ripper::SexpBuilder
+Ripper::SexpBuilderPP
+```
+
+Briefly, `Ripper` is a streaming parser that allows you to construct your own syntax tree. As an example:
+
+```ruby
+class ArithmeticRipper < Prism::Translation::Ripper
+  def on_binary(left, operator, right)
+    left.public_send(operator, right)
+  end
+
+  def on_int(value)
+    value.to_i
+  end
+
+  def on_program(stmts)
+    stmts
+  end
+
+  def on_stmts_new
+    []
+  end
+
+  def on_stmts_add(stmts, stmt)
+    stmts << stmt
+    stmts
+  end
+end
+
+ArithmeticRipper.new("1 + 2 - 3").parse # => [0]
+```
+
+The exact names of the `on_*` methods are listed in the `Ripper` source.
+
+## Background
+
+It is helpful to understand the differences between the `Ripper` library and the `Prism` library. Both libraries perform parsing and provide you with APIs to manipulate and understand the resulting syntax tree. However, there are a few key differences.
+
+### Design
+
+`Ripper` is a streaming parser. This means as it is parsing Ruby code, it dispatches events back to the consumer. This allows quite a bit of flexibility. You can use it to build your own syntax tree or to find specific patterns in the code. `Prism` on the other hand returns to you the completed syntax tree _before_ it allows you to manipulate it. This means the tree that you get back is the only representation that can be generated by the parser _at parse time_ (but of course can be manipulated later).
+
+### Fields
+
+We use the term "field" to mean a piece of information on a syntax tree node. `Ripper` provides the minimal number of fields to accurately represent the syntax tree for the purposes of compilation/interpretation. For example, in the callbacks for nodes that are based on keywords (`class`, `module`, `for`, `while`, etc.) you are not given the keyword itself, you need to attach it on your own. In other cases, tokens are not necessarily dispatched at all, meaning you need to find them yourself. `Prism` provides the opposite: the maximum number of fields on nodes is provided. As a tradeoff, this requires more memory, but this is chosen to make it easier on consumers.
+
+### Maintainability
+
+The `Ripper` interface is not guaranteed in any way, and tends to change between patch versions of CRuby. This is largely due to the fact that `Ripper` is a by-product of the generated parser, as opposed to its own parser. As an example, in the expression `foo::bar = baz`, there are three different represents possible for the call operator, including:
+
+* `:"::"` - Ruby 1.9 to Ruby 3.1.4
+* `73` - Ruby 3.1.5 to Ruby 3.1.6
+* `[:@op, "::", [lineno, column]]` - Ruby 3.2.0 and later
+
+The `Prism` interface is guaranteed going forward to be the consistent, and the official Ruby syntax tree interface. This means you can rely on this interface without having to worry about individual changes between Ruby versions. It also is a gem, which means it is versioned based on the gem version, as opposed to being versioned based on the Ruby version. Finally, you can use `Prism` to parse multiple versions of Ruby, whereas `Ripper` is tied to the Ruby version it is running on.

data/docs/ruby_api.md

@@ -0,0 +1,44 @@
+# Ruby API
+
+The `prism` gem provides a Ruby API for accessing the syntax tree.
+
+For the most part, the API for accessing the tree mirrors that found in the [Syntax Tree](https://github.com/ruby-syntax-tree/syntax_tree) project. This means:
+
+* Walking the tree involves creating a visitor and passing it to the `#accept` method on any node in the tree
+* Nodes in the tree respond to named methods for accessing their children as well as `#child_nodes`
+* Nodes respond to the pattern matching interfaces `#deconstruct` and `#deconstruct_keys`
+
+Every entry in `config.yml` will generate a Ruby class as well as the code that builds the nodes themselves.
+Creating a syntax tree involves calling one of the class methods on the `Prism` module.
+The full API is documented below.
+
+## API
+
+* `Prism.dump(source)` - parse the syntax tree corresponding to the given source string, and serialize it to a string
+* `Prism.dump_file(filepath)` - parse the syntax tree corresponding to the given source file and serialize it to a string
+* `Prism.lex(source)` - parse the tokens corresponding to the given source string and return them as an array within a parse result
+* `Prism.lex_file(filepath)` - parse the tokens corresponding to the given source file and return them as an array within a parse result
+* `Prism.parse(source)` - parse the syntax tree corresponding to the given source string and return it within a parse result
+* `Prism.parse_file(filepath)` - parse the syntax tree corresponding to the given source file and return it within a parse result
+* `Prism.parse_stream(io)` - parse the syntax tree corresponding to the source that is read out of the given IO object using the `#gets` method and return it within a parse result
+* `Prism.parse_lex(source)` - parse the syntax tree corresponding to the given source string and return it within a parse result, along with the tokens
+* `Prism.parse_lex_file(filepath)` - parse the syntax tree corresponding to the given source file and return it within a parse result, along with the tokens
+* `Prism.load(source, serialized, freeze = false)` - load the serialized syntax tree using the source as a reference into a syntax tree
+* `Prism.parse_comments(source)` - parse the comments corresponding to the given source string and return them
+* `Prism.parse_file_comments(source)` - parse the comments corresponding to the given source file and return them
+* `Prism.parse_success?(source)` - parse the syntax tree corresponding to the given source string and return true if it was parsed without errors
+* `Prism.parse_file_success?(filepath)` - parse the syntax tree corresponding to the given source file and return true if it was parsed without errors
+
+## Nodes
+
+Once you have nodes in hand coming out of a parse result, there are a number of common APIs that are available on each instance. They are:
+
+* `#accept(visitor)` - a method that will immediately call `visit_*` to specialize for the node type
+* `#child_nodes` - a positional array of the child nodes of the node, with `nil` values for any missing children
+* `#compact_child_nodes` - a positional array of the child nodes of the node with no `nil` values
+* `#copy(**keys)` - a method that allows creating a shallow copy of the node with the given keys overridden
+* `#deconstruct`/`#deconstruct_keys(keys)` - the pattern matching interface for nodes
+* `#inspect` - a string representation that looks like the syntax tree of the node
+* `#location` - a `Location` object that describes the location of the node in the source file
+* `#to_dot` - convert the node's syntax tree into graphviz dot notation
+* `#type` - a symbol that represents the type of the node, useful for quick comparisons

data/docs/ruby_parser_translation.md

@@ -0,0 +1,19 @@
+# ruby_parser translation
+
+Prism ships with the ability to translate its syntax tree into the syntax tree used by the [seattlerb/ruby_parser](https://github.com/seattlerb/ruby_parser) gem. This allows you to use tools built on top of the `ruby_parser` gem with the `prism` parser.
+
+## Usage
+
+You can call the `parse` and `parse_file` methods on the `Prism::Translation::RubyParser` module:
+
+```ruby
+filepath = "path/to/file.rb"
+Prism::Translation::RubyParser.parse_file(filepath)
+```
+
+This will return to you `Sexp` objects that mirror the result of calling `RubyParser` methods, as in:
+
+```ruby
+filepath = "path/to/file.rb"
+RubyParser.new.parse(File.read(filepath), filepath)
+```