prism 1.2.0 → 1.4.0

data/docs/build_system.md

@@ -16,9 +16,9 @@ The main solution for the second point seems a Makefile, otherwise many of the u
 ## General Design
 
 1. Templates are generated by `templates/template.rb`
-4. The `Makefile` compiles both `libprism.a` and `libprism.{so,dylib,dll}` from the `src/**/*.c` and `include/**/*.h` files
-5. The `Rakefile` `:compile` task ensures the above prerequisites are done, then calls `make`,
-  and uses `Rake::ExtensionTask` to compile the C extension (using its `extconf.rb`), which uses `libprism.a`
+2. The `Makefile` compiles both `libprism.a` and `libprism.{so,dylib,dll}` from the `src/**/*.c` and `include/**/*.h` files
+3. The `Rakefile` `:compile` task ensures the above prerequisites are done, then calls `make`,
+   and uses `Rake::ExtensionTask` to compile the C extension (using its `extconf.rb`)
 
 This way there is minimal duplication, and each layer builds on the previous one and has its own responsibilities.
 
@@ -35,14 +35,11 @@ loaded per process (i.e., at most one version of the prism *gem* loaded in a pro
 ### Building the prism gem by `gem install/bundle install`
 
 The gem contains the pre-generated templates.
-When installing the gem, `extconf.rb` is used and that:
-* runs `make build/libprism.a`
-* compiles the C extension with mkmf
 
-When installing the gem on JRuby and TruffleRuby, no C extension is built, so instead of the last step,
-there is Ruby code using FFI which uses `libprism.{so,dylib,dll}`
-to implement the same methods as the C extension, but using serialization instead of many native calls/accesses
-(JRuby does not support C extensions, serialization is faster on TruffleRuby than the C extension).
+When installing the gem on CRuby, `extconf.rb` is used and that compiles the C extension with mkmf, including both the extension files and the sources of prism itself.
+
+When installing the gem on JRuby and TruffleRuby, no C extension is built, so instead the `extconf.rb` runs `make build/libprism.{so,dylib,dll}`.
+There is Ruby code using FFI which uses `libprism.{so,dylib,dll}` to implement the same methods as the C extension, but using serialization instead of many native calls/accesses (JRuby does not support C extensions, serialization is faster on TruffleRuby than the C extension).
 
 ### Building the prism gem from git, e.g. `gem "prism", github: "ruby/prism"`
 
@@ -66,7 +63,7 @@ The script generates the templates when importing.
 
 Then when `mx build` builds TruffleRuby and the `prism` mx project inside, it runs `make`.
 
-Then the `prism bindings` mx project is built, which contains the [bindings](https://github.com/oracle/truffleruby/blob/master/src/main/c/prism_bindings/src/prism_bindings.c)
+Then the `prism bindings` mx project is built, which contains the [bindings](https://github.com/oracle/truffleruby/blob/vm-24.1.1/src/main/c/yarp_bindings/src/yarp_bindings.c)
 and links to `libprism.a` (to avoid exporting symbols, so no conflict when installing the prism gem).
 
 ### Building prism as part of JRuby

data/docs/releasing.md

@@ -40,7 +40,7 @@ ruby -pi -e 'gsub(/^ruby-prism-sys = \{ version = ".+?"/, %Q{ruby-prism-sys = \{
 * Update the `Gemfile.lock` file:
 
 ```sh
-chruby ruby-3.4.0-dev
+chruby ruby-3.5.0-dev
 bundle install
 ```
 

data/docs/relocation.md

@@ -0,0 +1,34 @@
+# Relocation
+
+Prism parses deterministically for the same input. This provides a nice property that is exposed through the `#node_id` API on nodes. Effectively this means that for the same input, these values will remain consistent every time the source is parsed. This means we can reparse the source same with a `#node_id` value and find the exact same node again.
+
+The `Relocation` module provides an API around this property. It allows you to "save" nodes and locations using a minimal amount of memory (just the node_id and a field identifier) and then reify them later. This minimizes the amount of memory you need to allocate to store this information because it does not keep around a pointer to the source string.
+
+## Getting started
+
+To get started with the `Relocation` module, you would first instantiate a `Repository` object. You do this through a DSL that chains method calls for configuration. For example, if for every entry in the repository you want to store the start and end lines, the start and end code unit columns for in UTF-16, and the leading comments, you would:
+
+```ruby
+repository = Prism::Relocation.filepath("path/to/file").lines.code_unit_columns(Encoding::UTF_16).leading_comments
+```
+
+Now that you have the repository, you can pass it into any of the `save*` APIs on nodes or locations to create entries in the repository that will be lazily reified.
+
+```ruby
+# assume that node is a Prism::ClassNode object
+entry = node.constant_path.save(repository)
+```
+
+Now that you have the entry object, you do not need to keep around a reference to the repository, it will be cleaned up on its own when the last entry is reified. Now, whenever you need to, you may call the associated field methods on the entry object, as in:
+
+```ruby
+entry.start_line
+entry.end_line
+
+entry.start_code_units_column
+entry.end_code_units_column
+
+entry.leading_comments
+```
+
+Note that if you had configured other fields to be saved, you would be able to access them as well. The first time one of these fields is accessed, the repository will reify every entry it knows about and then clean itself up. In this way, you can effectively treat them as if you had kept around lightweight versions of `Prism::Node` or `Prism::Location` objects.

data/docs/ruby_api.md

@@ -23,7 +23,7 @@ The full API is documented below.
 * `Prism.parse_stream(io)` - parse the syntax tree corresponding to the source that is read out of the given IO object using the `#gets` method and return it within a parse result
 * `Prism.parse_lex(source)` - parse the syntax tree corresponding to the given source string and return it within a parse result, along with the tokens
 * `Prism.parse_lex_file(filepath)` - parse the syntax tree corresponding to the given source file and return it within a parse result, along with the tokens
-* `Prism.load(source, serialized)` - load the serialized syntax tree using the source as a reference into a syntax tree
+* `Prism.load(source, serialized, freeze = false)` - load the serialized syntax tree using the source as a reference into a syntax tree
 * `Prism.parse_comments(source)` - parse the comments corresponding to the given source string and return them
 * `Prism.parse_file_comments(source)` - parse the comments corresponding to the given source file and return them
 * `Prism.parse_success?(source)` - parse the syntax tree corresponding to the given source string and return true if it was parsed without errors