yarp 0.12.0 → 0.13.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9afbc3b5f4c070c404f0421f1814d5a1200cb849ef9bf38b2ae23f50ba738fdf
-  data.tar.gz: bed121fb1ac414cf5918a2bcf7df2a9b1318df2f46dff9863cc5b4427ce79409
+  metadata.gz: da8f3b5f2cdae92e669cfe7865566c2014b57746644fd550e23dfe27a7f4a5cd
+  data.tar.gz: 994f5db733b1261c76920a6426820dc941a72ba2619f71de0b7ac731da68d57d
 SHA512:
-  metadata.gz: 3fd5831ab86ca1ca299e86ca5f2ca184164fa56de353cfa0d9b51d2ece522c2f5bebfadd1896222e4b7e1e984604be414ee8900b1a5bbc0b8eb3b6ee7bb738dc
-  data.tar.gz: 6fa02c77777391a1c4b5dc47445d905fee835213c7c02687cac432a4b566bbae8ead26467609066e83bf030ce19e09f46cf553fc0acdd8f7ef977074568b7061
+  metadata.gz: bc2bc26648b224d5195a1649f906a7d66d6772626b20481b17ccff76a224946b614919aa2d84afb1756ed6bae4e80e460715e25e0a1f0af41b5f5c02d217c99d
+  data.tar.gz: db5a41e5abd08dbd4184e1fb6284c4c414f9377894ef4cca689f15b10c388427d2d9e465067159abce7cb8e545bc32c3996983feb5f1ffbd340c84d48796c04b

data/CHANGELOG.md

@@ -6,6 +6,26 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 ## [Unreleased]
 
+## [0.13.0] - 2023-09-29
+
+### Added
+
+- `BEGIN {}` blocks are only allowed at the top-level, and will now provide a syntax error if they are not.
+- Numbered parameters are not allowed in block parameters, and will now provide a syntax error if they are.
+- Many more Ruby modules and classes are now documented. Also, many have been moved into their own files and autoloaded so that initial boot time of the gem is much faster.
+- `PM_TOKEN_METHOD_NAME` is introduced, used to indicate an identifier that if definitely a method name because it has an `!` or `?` at the end.
+- In the C API, arrays, assocs, and hashes now can have the `PM_NODE_FLAG_STATIC_LITERAL` flag attached if they can be compiled statically. This is used in CRuby, for example, to determine if a `duphash`/`duparray` instruction can be used as opposed to a `newhash`/`newarray`.
+- `Node#type` is introduced, which returns a symbol representing the type of the node. This is useful for case comparisons when you have to compare against multiple types.
+
+### Changed
+
+- **BREAKING**: Everything has been renamed to `prism` instead of `yarp`. The `yp_`/`YP_` prefix in the C API has been changed to `pm_`/`PM_`. For the most part, everything should be find/replaceable.
+- **BREAKING**: `BlockArgumentNode` nodes now go into the `block` field on `CallNode` nodes, in addition to the `BlockNode` nodes that used to be there. Hopefully this makes it more consistent to compile/deal with in general, but it does mean it can be a surprising breaking change.
+- Escaped whitespace in `%w` lists is now properly unescaped.
+- `Node#pretty_print` now respects pretty print indentation.
+- `Dispatcher` was previously firing `_leave` events in the incorrect order. This has now been fixed.
+- **BREAKING**: `Visitor` has now been split into `Visitor` and `Compiler`. The visitor visits nodes but doesn't return anything from the visit methods. It is suitable for taking action based on the tree, but not manipulating the tree itself. The `Compiler` visits nodes and returns the computed value up the tree. It is suitable for compiling the tree into another format. As such, `MutationVisitor` has been renamed to `MutationCompiler`.
+
 ## [0.12.0] - 2023-09-15
 
 ### Added
@@ -141,11 +161,12 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 - 🎉 Initial release! 🎉
 
-[unreleased]: https://github.com/ruby/yarp/compare/v0.12.0...HEAD
-[0.12.0]: https://github.com/ruby/yarp/compare/v0.11.0...v0.12.0
-[0.11.0]: https://github.com/ruby/yarp/compare/v0.10.0...v0.11.0
-[0.10.0]: https://github.com/ruby/yarp/compare/v0.9.0...v0.10.0
-[0.9.0]: https://github.com/ruby/yarp/compare/v0.8.0...v0.9.0
-[0.8.0]: https://github.com/ruby/yarp/compare/v0.7.0...v0.8.0
-[0.7.0]: https://github.com/ruby/yarp/compare/v0.6.0...v0.7.0
-[0.6.0]: https://github.com/ruby/yarp/compare/d60531...v0.6.0
+[unreleased]: https://github.com/ruby/prism/compare/v0.13.0...HEAD
+[0.13.0]: https://github.com/ruby/prism/compare/v0.12.0...v0.13.0
+[0.12.0]: https://github.com/ruby/prism/compare/v0.11.0...v0.12.0
+[0.11.0]: https://github.com/ruby/prism/compare/v0.10.0...v0.11.0
+[0.10.0]: https://github.com/ruby/prism/compare/v0.9.0...v0.10.0
+[0.9.0]: https://github.com/ruby/prism/compare/v0.8.0...v0.9.0
+[0.8.0]: https://github.com/ruby/prism/compare/v0.7.0...v0.8.0
+[0.7.0]: https://github.com/ruby/prism/compare/v0.6.0...v0.7.0
+[0.6.0]: https://github.com/ruby/prism/compare/d60531...v0.6.0

data/CONTRIBUTING.md

@@ -1,6 +1,6 @@
 # Contributing
 
-Thank you for your interest in contributing to YARP! Below are a couple of ways that you can help out.
+Thank you for your interest in contributing to prism! Below are a couple of ways that you can help out.
 
 ## Discussions
 
@@ -29,7 +29,7 @@ or explicitly running the `compile` task:
 ``` sh
 bundle exec rake compile test
 # or to just compile the C extension ...
-bundle exec rake compile:yarp test
+bundle exec rake compile:prism test
 ```
 
 To test the rust bindings (with caveats about setting up your Rust environment properly first):

data/Makefile

@@ -35,7 +35,7 @@ build/librubyparser.a: $(STATIC_OBJECTS)
 build/shared/%.o: src/%.c Makefile $(HEADERS)
 	$(ECHO) "compiling $@"
 	$(Q) mkdir -p $(@D)
-	$(Q) $(CC) $(DEBUG_FLAGS) -DYP_EXPORT_SYMBOLS $(CPPFLAGS) $(CFLAGS) -c -o $@ $<
+	$(Q) $(CC) $(DEBUG_FLAGS) -DPRISM_EXPORT_SYMBOLS $(CPPFLAGS) $(CFLAGS) -c -o $@ $<
 
 build/static/%.o: src/%.c Makefile $(HEADERS)
 	$(ECHO) "compiling $@"
@@ -55,20 +55,20 @@ build/fuzz.heisenbug.%: $(SOURCES) fuzz/%.c fuzz/heisenbug.c
 
 fuzz-debug:
 	$(ECHO) "entering debug shell"
-	$(Q) docker run -it --rm -e HISTFILE=/yarp/fuzz/output/.bash_history -v $(shell pwd):/yarp -v $(FUZZ_OUTPUT_DIR):/fuzz_output yarp/fuzz
+	$(Q) docker run -it --rm -e HISTFILE=/prism/fuzz/output/.bash_history -v $(shell pwd):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz
 
 fuzz-docker-build: fuzz/docker/Dockerfile
 	$(ECHO) "building docker image"
-	$(Q) docker build -t yarp/fuzz fuzz/docker/
+	$(Q) docker build -t prism/fuzz fuzz/docker/
 
 fuzz-run-%: FORCE fuzz-docker-build
 	$(ECHO) "generating templates"
 	$(Q) bundle exec rake templates
 	$(ECHO) "running $* fuzzer"
-	$(Q) docker run --rm -v $(shell pwd):/yarp yarp/fuzz /bin/bash -c "FUZZ_FLAGS=\"$(FUZZ_FLAGS)\" make build/fuzz.$*"
+	$(Q) docker run --rm -v $(shell pwd):/prism prism/fuzz /bin/bash -c "FUZZ_FLAGS=\"$(FUZZ_FLAGS)\" make build/fuzz.$*"
 	$(ECHO) "starting AFL++ run"
 	$(Q) mkdir -p $(FUZZ_OUTPUT_DIR)/$*
-	$(Q) docker run -it --rm -v $(shell pwd):/yarp -v $(FUZZ_OUTPUT_DIR):/fuzz_output yarp/fuzz /bin/bash -c "./fuzz/$*.sh /fuzz_output/$*"
+	$(Q) docker run -it --rm -v $(shell pwd):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz /bin/bash -c "./fuzz/$*.sh /fuzz_output/$*"
 FORCE:
 
 fuzz-clean:

data/README.md

@@ -1,4 +1,4 @@
-# Yet Another Ruby Parser
+# Prism Ruby parser
 
 This is a parser for the Ruby programming language. It is designed to be portable, error tolerant, and maintainable. It is written in C99 and has no dependencies. It is currently being integrated into [CRuby](https://github.com/ruby/ruby), [JRuby](https://github.com/jruby/jruby), [TruffleRuby](https://github.com/oracle/truffleruby), [Sorbet](https://github.com/sorbet/sorbet), and [Syntax Tree](https://github.com/ruby-syntax-tree/syntax_tree).
 
@@ -16,29 +16,29 @@ The repository contains the infrastructure for both a shared library (librubypar
 ├── config.yml            specification for tokens and nodes in the tree
 ├── docs                  documentation about the project
 ├── ext
-│   └── yarp
+│   └── prism
 │       ├── extconf.rb    configuration to generate the Makefile for the native extension
 │       └── extension.c   the native extension that interacts with librubyparser
 ├── fuzz                  files related to fuzz testing
 ├── include
-│   ├── yarp              header files for the shared library
-│   └── yarp.h            main header file for the shared library
+│   ├── prism             header files for the shared library
+│   └── prism.h           main header file for the shared library
 ├── java                  Java bindings for the shared library
 ├── lib
-│   ├── yarp              Ruby library files
-│   └── yarp.rb           main entrypoint for the Ruby library
+│   ├── prism             Ruby library files
+│   └── prism.rb          main entrypoint for the Ruby library
 ├── rakelib               various Rake tasks for the project
 ├── rust
-│   ├── yarp              Rustified crate for the shared library
-│   └── yarp-sys          FFI binding for Rust
+│   ├── prism             Rustified crate for the shared library
+│   └── prism-sys         FFI binding for Rust
 ├── src
 │   ├── enc               various encoding files
 │   ├── util              various utility files
-│   └── yarp.c            main entrypoint for the shared library
+│   └── prism.c           main entrypoint for the shared library
 ├── templates             contains ERB templates generated by templates/template.rb
 │   └── template.rb       generates code from the nodes and tokens configured by config.yml
 └── test
-    └── yarp
+    └── prism
         ├── fixtures      Ruby code used for testing
         └── snapshots     snapshots of generated syntax trees corresponding to fixtures
 ```
@@ -48,7 +48,7 @@ The repository contains the infrastructure for both a shared library (librubypar
 To compile the shared library, you will need:
 
 * A C99 compiler
-* autotools autoconf, automake, libtool)
+* autotools (autoconf, automake, libtool)
 * make
 * Ruby 3.3.0-preview1 or later
 
@@ -87,4 +87,3 @@ See the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information. We additio
 * [Ruby API](docs/ruby_api.md)
 * [Serialization](docs/serialization.md)
 * [Testing](docs/testing.md)
-

data/config.yml

@@ -232,6 +232,8 @@ tokens:
     comment: "<<"
   - name: LESS_LESS_EQUAL
     comment: "<<="
+  - name: METHOD_NAME
+    comment: "a method name"
   - name: MINUS
     comment: "-"
   - name: MINUS_EQUAL
@@ -668,7 +670,6 @@ nodes:
         type: location?
       - name: block
         type: node?
-        kind: BlockNode
       - name: flags
         type: flags
         kind: CallNodeFlags
@@ -1771,6 +1772,7 @@ nodes:
         type: location
       - name: content_loc
         type: location
+        semantic_field: true # https://github.com/ruby/prism/issues/1452
       - name: closing_loc
         type: location
       - name: unescaped
@@ -2091,6 +2093,7 @@ nodes:
         type: location
       - name: content_loc
         type: location
+        semantic_field: true # https://github.com/ruby/prism/issues/1452
       - name: closing_loc
         type: location
       - name: unescaped
@@ -2284,8 +2287,10 @@ nodes:
         kind: StringFlags
       - name: opening_loc
         type: location?
+        semantic_field: true # https://github.com/ruby/prism/issues/1452
       - name: content_loc
         type: location
+        semantic_field: true # https://github.com/ruby/prism/issues/1452
       - name: closing_loc
         type: location?
       - name: unescaped
@@ -2315,7 +2320,6 @@ nodes:
         type: location?
       - name: block
         type: node?
-        kind: BlockNode
     comment: |
       Represents the use of the `super` keyword with parentheses or arguments.
 

data/docs/build_system.md

@@ -1,17 +1,17 @@
 # Build System
 
-There are many ways to build YARP, which means the build system is a bit more complicated than usual.
+There are many ways to build prism, which means the build system is a bit more complicated than usual.
 
 ## Requirements
 
-* It must work to build YARP for all 6 uses-cases below.
-* It must be possible to build YARP without needing ruby/rake/etc.
-  Because once YARP is the single parser in TruffleRuby, JRuby or CRuby there won't be another Ruby parser around to parse such Ruby code.
+* It must work to build prism for all 6 uses-cases below.
+* It must be possible to build prism without needing ruby/rake/etc.
+  Because once prism is the single parser in TruffleRuby, JRuby or CRuby there won't be another Ruby parser around to parse such Ruby code.
   Most/every Ruby implementations want to avoid depending on another Ruby during the build process as that is very brittle.
-* It is desirable to compile YARP with the same or very similar compiler flags for all use-cases (e.g. optimization level, warning flags, etc).
-  Otherwise, there is the risk YARP does not work correctly with those different compiler flags.
+* It is desirable to compile prism with the same or very similar compiler flags for all use-cases (e.g. optimization level, warning flags, etc).
+  Otherwise, there is the risk prism does not work correctly with those different compiler flags.
 
-The main solution for the second point seems a Makefile, otherwise many of the usages would have to duplicate the logic to build YARP.
+The main solution for the second point seems a Makefile, otherwise many of the usages would have to duplicate the logic to build prism.
 
 ## General Design
 
@@ -24,15 +24,15 @@ This way there is minimal duplication, and each layer builds on the previous one
 
 The static library exports no symbols, to avoid any conflict.
 The shared library exports some symbols, and this is fine since there should only be one librubyparser shared library
-loaded per process (i.e., at most one version of the yarp *gem* loaded in a process, only the gem uses the shared library).
+loaded per process (i.e., at most one version of the prism *gem* loaded in a process, only the gem uses the shared library).
 
-## The various ways to build YARP
+## The various ways to build prism
 
-### Building from ruby/yarp repository with `bundle exec rake`
+### Building from ruby/prism repository with `bundle exec rake`
 
 `rake` calls `make` and then uses `Rake::ExtensionTask` to compile the C extension (see above).
 
-### Building the yarp gem by `gem install/bundle install`
+### Building the prism gem by `gem install/bundle install`
 
 The gem contains the pre-generated templates.
 When installing the gem, `extconf.rb` is used and that:
@@ -44,31 +44,31 @@ there is Ruby code using FFI which uses `librubyparser.{so,dylib,dll}`
 to implement the same methods as the C extension, but using serialization instead of many native calls/accesses
 (JRuby does not support C extensions, serialization is faster on TruffleRuby than the C extension).
 
-### Building the yarp gem from git, e.g. `gem 'yarp', github: 'ruby/yarp'`
+### Building the prism gem from git, e.g. `gem "prism", github: "ruby/prism"`
 
 The same as above, except the `extconf.rb` additionally runs first:
 * `templates/template.rb` to generate the templates
 
 Because of course those files are not part of the git repository.
 
-### Building YARP as part of CRuby
+### Building prism as part of CRuby
 
-[This script](https://github.com/ruby/ruby/blob/32e828bb4a6c65a392b2300f3bdf93008c7b6f25/tool/sync_default_gems.rb#L399-L426) imports YARP sources in CRuby.
+[This script](https://github.com/ruby/ruby/blob/32e828bb4a6c65a392b2300f3bdf93008c7b6f25/tool/sync_default_gems.rb#L399-L426) imports prism sources in CRuby.
 
 The script generates the templates when importing.
 
-YARP's `Makefile` is not used at all in CRuby. Instead, CRuby's `Makefile` is used.
+prism's `Makefile` is not used at all in CRuby. Instead, CRuby's `Makefile` is used.
 
-### Building YARP as part of TruffleRuby
+### Building prism as part of TruffleRuby
 
-[This script](https://github.com/oracle/truffleruby/blob/master/tool/import-yarp.sh) imports YARP sources in TruffleRuby.
+[This script](https://github.com/oracle/truffleruby/blob/master/tool/import-prism.sh) imports prism sources in TruffleRuby.
 The script generates the templates when importing.
 
-Then when `mx build` builds TruffleRuby and the `yarp` mx project inside, it runs `make`.
+Then when `mx build` builds TruffleRuby and the `prism` mx project inside, it runs `make`.
 
-Then the `yarp bindings` mx project is built, which contains the [bindings](https://github.com/oracle/truffleruby/blob/master/src/main/c/yarp_bindings/src/yarp_bindings.c)
-and links to `librubyparser.a` (to avoid exporting symbols, so no conflict when installing the yarp gem).
+Then the `prism bindings` mx project is built, which contains the [bindings](https://github.com/oracle/truffleruby/blob/master/src/main/c/prism_bindings/src/prism_bindings.c)
+and links to `librubyparser.a` (to avoid exporting symbols, so no conflict when installing the prism gem).
 
-### Building YARP as part of JRuby
+### Building prism as part of JRuby
 
 TODO, probably similar to TruffleRuby.

data/docs/building.md

@@ -1,13 +1,13 @@
 # Building
 
-The following describes how to build YARP from source.
+The following describes how to build prism from source.
 This comes directly from the [Makefile](../Makefile).
 
 ## Common
 
 All of the source files match `src/**/*.c` and all of the headers match `include/**/*.h`.
 
-The following flags should be used to compile YARP:
+The following flags should be used to compile prism:
 
 * `-std=c99` - Use the C99 standard
 * `-Wall -Wconversion -Wextra -Wpedantic -Wundef` - Enable the warnings we care about
@@ -16,7 +16,7 @@ The following flags should be used to compile YARP:
 
 ## Shared
 
-If you want to build YARP as a shared library and link against it, you should compile with:
+If you want to build prism as a shared library and link against it, you should compile with:
 
 * `-fPIC -shared` - Compile as a shared library
-* `-DYP_EXPORT_SYMBOLS` - Export the symbols (by default nothing is exported)
+* `-DPRISM_EXPORT_SYMBOLS` - Export the symbols (by default nothing is exported)

data/docs/configuration.md

@@ -1,15 +1,19 @@
 # Configuration
 
-A lot of code in YARP's repository is templated from a single configuration file, [config.yml](../config.yml). This file is used to generate the following files:
-
-* `ext/yarp/api_node.c` - for defining how to build Ruby objects for the nodes out of C structs
-* `include/yarp/ast.h` - for defining the C structs that represent the nodes
-* `java/org/yarp/AbstractNodeVisitor.java` - for defining the visitor interface for the nodes in Java
-* `java/org/yarp/Loader.java` - for defining how to deserialize the nodes in Java
-* `java/org/yarp/Nodes.java` - for defining the nodes in Java
-* `lib/yarp/mutation_visitor.rb` - for defining the mutation visitor for the nodes in Ruby
-* `lib/yarp/node.rb` - for defining the nodes in Ruby
-* `lib/yarp/serialize.rb` - for defining how to deserialize the nodes in Ruby
+A lot of code in prism's repository is templated from a single configuration file, [config.yml](../config.yml). This file is used to generate the following files:
+
+* `ext/prism/api_node.c` - for defining how to build Ruby objects for the nodes out of C structs
+* `include/prism/ast.h` - for defining the C structs that represent the nodes
+* `java/org/prism/AbstractNodeVisitor.java` - for defining the visitor interface for the nodes in Java
+* `java/org/prism/Loader.java` - for defining how to deserialize the nodes in Java
+* `java/org/prism/Nodes.java` - for defining the nodes in Java
+* `lib/prism/compiler.rb` - for defining the compiler for the nodes in Ruby
+* `lib/prism/dispatcher.rb` - for defining the dispatch visitors for the nodes in Ruby
+* `lib/prism/dsl.rb` - for defining the DSL for the nodes in Ruby
+* `lib/prism/mutation_compiler.rb` - for defining the mutation compiler for the nodes in Ruby
+* `lib/prism/node.rb` - for defining the nodes in Ruby
+* `lib/prism/serialize.rb` - for defining how to deserialize the nodes in Ruby
+* `lib/prism/visitor.rb` - for defining the visitor interface for the nodes in Ruby
 * `src/node.c` - for defining how to free the nodes in C and calculate the size in memory in C
 * `src/prettyprint.c` - for defining how to prettyprint the nodes in C
 * `src/serialize.c` - for defining how to serialize the nodes in C
@@ -25,7 +29,7 @@ This is a list of tokens to be used by the lexer. It is shared here so that it c
 
 Each token is expected to have a `name` key and a `comment` key (both as strings). Optionally they can have a `value` key (an integer) which is used to represent the value in the enum.
 
-In C these tokens will be templated out with the prefix `YP_TOKEN_`. For example, if you have a `name` key with the value `PERCENT`, you can access this in C through `YP_TOKEN_PERCENT`.
+In C these tokens will be templated out with the prefix `PM_TOKEN_`. For example, if you have a `name` key with the value `PERCENT`, you can access this in C through `PM_TOKEN_PERCENT`.
 
 ## `flags`
 
@@ -33,7 +37,7 @@ Sometimes we need to communicate more information in the tree than can be repres
 
 Each flag is expected to have a `name` key (a string) and a `values` key (an array). Each value in the `values` key should be an object that contains both a `name` key (a string) that represents the name of the flag and a `comment` key (a string) that represents the comment for the flag.
 
-In C these flags will get templated out with a `YP_` prefix, then a snake-case version of the flag name, then the flag itself. For example, if you have a flag with the name `RegularExpressionFlags` and a value with the name `IGNORE_CASE`, you can access this in C through `YP_REGULAR_EXPRESSION_FLAGS_IGNORE_CASE`.
+In C these flags will get templated out with a `PM_` prefix, then a snake-case version of the flag name, then the flag itself. For example, if you have a flag with the name `RegularExpressionFlags` and a value with the name `IGNORE_CASE`, you can access this in C through `PM_REGULAR_EXPRESSION_FLAGS_IGNORE_CASE`.
 
 ## `nodes`
 
@@ -43,14 +47,14 @@ Optionally, every node can define a `child_nodes` key that is an array. This arr
 
 The available values for `type` are:
 
-* `node` - A child node that is a node itself. This is a `yp_node_t *` in C.
-* `node?` - A child node that is optionally present. This is also a `yp_node_t *` in C, but can be `NULL`.
-* `node[]` - A child node that is an array of nodes. This is a `yp_node_list_t` in C.
-* `string` - A child node that is a string. For example, this is used as the name of the method in a call node, since it cannot directly reference the source string (as in `@-` or `foo=`). This is a `yp_string_t` in C.
-* `constant` - A variable-length integer that represents an index in the constant pool. This is a `yp_constant_id_t` in C.
-* `constant[]` - A child node that is an array of constants. This is a `yp_constant_id_list_t` in C.
-* `location` - A child node that is a location. This is a `yp_location_t` in C.
-* `location?` - A child node that is a location that is optionally present. This is a `yp_location_t` in C, but if the value is not present then the `start` and `end` fields will be `NULL`.
+* `node` - A child node that is a node itself. This is a `pm_node_t *` in C.
+* `node?` - A child node that is optionally present. This is also a `pm_node_t *` in C, but can be `NULL`.
+* `node[]` - A child node that is an array of nodes. This is a `pm_node_list_t` in C.
+* `string` - A child node that is a string. For example, this is used as the name of the method in a call node, since it cannot directly reference the source string (as in `@-` or `foo=`). This is a `pm_string_t` in C.
+* `constant` - A variable-length integer that represents an index in the constant pool. This is a `pm_constant_id_t` in C.
+* `constant[]` - A child node that is an array of constants. This is a `pm_constant_id_list_t` in C.
+* `location` - A child node that is a location. This is a `pm_location_t` in C.
+* `location?` - A child node that is a location that is optionally present. This is a `pm_location_t` in C, but if the value is not present then the `start` and `end` fields will be `NULL`.
 * `uint32` - A child node that is a 32-bit unsigned integer. This is a `uint32_t` in C.
 
-If the type is `node` or `node?` then the value also accepts an optional `kind` key (a string). This key is expected to match to the name of another node type within `config.yml`. This changes a couple of places where code is templated out to use the more specific struct name instead of the generic `yp_node_t`. For example, with `kind: StatementsNode` the `yp_node_t *` in C becomes a `yp_statements_node_t *`.
+If the type is `node` or `node?` then the value also accepts an optional `kind` key (a string). This key is expected to match to the name of another node type within `config.yml`. This changes a couple of places where code is templated out to use the more specific struct name instead of the generic `pm_node_t`. For example, with `kind: StatementsNode` the `pm_node_t *` in C becomes a `pm_statements_node_t *`.

data/docs/design.md

@@ -12,7 +12,7 @@ The design of the parser is based around these main goals.
 
 The first piece to understand about the parser is the design of its syntax tree. This is documented in `config.yml`. Every token and node is defined in that file, along with comments about where they are found in what kinds of syntax. This file is used to template out a lot of different files, all found in the `templates` directory. The `templates/template.rb` script performs the templating and outputs all files matching the directory structure found in the templates directory.
 
-The templated files contain all of the code required to allocate and initialize nodes, pretty print nodes, and serialize nodes. This means for the most part, you will only need to then hook up the parser to call the templated functions to create the nodes in the correct position. That means editing the parser itself, which is housed in `yarp.c`.
+The templated files contain all of the code required to allocate and initialize nodes, pretty print nodes, and serialize nodes. This means for the most part, you will only need to then hook up the parser to call the templated functions to create the nodes in the correct position. That means editing the parser itself, which is housed in `prism.c`.
 
 ## Pratt parsing
 
@@ -24,7 +24,7 @@ In order to provide the best possible error tolerance, the parser is hand-writte
 * https://matklad.github.io/2020/04/13/simple-but-powerful-pratt-parsing.html
 * https://chidiwilliams.com/post/on-recursive-descent-and-pratt-parsing/
 
-You can find most of the functions that correspond to constructs in the Pratt parsing algorithm in `yarp.c`. As a couple of examples:
+You can find most of the functions that correspond to constructs in the Pratt parsing algorithm in `prism.c`. As a couple of examples:
 
 * `parse` corresponds to the `parse_expression` function
 * `nud` (null denotation) corresponds to the `parse_expression_prefix` function

data/docs/encoding.md

@@ -10,7 +10,7 @@ If the file is not encoded in UTF-8, the user must specify the encoding in a "ma
 # encoding: iso-8859-9
 ```
 
-The key of the comment can be either "encoding" or "coding". The value of the comment must be a string that is a valid encoding name. The encodings that YARP supports by default are:
+The key of the comment can be either "encoding" or "coding". The value of the comment must be a string that is a valid encoding name. The encodings that prism supports by default are:
 
 * `ascii`
 * `ascii-8bit`
@@ -44,11 +44,11 @@ The key of the comment can be either "encoding" or "coding". The value of the co
 * `windows-1251`
 * `windows-1252`
 
-For each of these encodings, YARP provides a function for checking if the subsequent bytes form an alphabetic or alphanumeric character.
+For each of these encodings, prism provides a function for checking if the subsequent bytes form an alphabetic or alphanumeric character.
 
 ## Support for other encodings
 
-If an encoding is encountered that is not supported by YARP, YARP will call a user-provided callback function with the name of the encoding if one is provided. That function can be registered with `yp_parser_register_encoding_decode_callback`. The user-provided callback function can then provide a pointer to an encoding struct that contains the requisite functions that YARP will use those to parse identifiers going forward.
+If an encoding is encountered that is not supported by prism, prism will call a user-provided callback function with the name of the encoding if one is provided. That function can be registered with `pm_parser_register_encoding_decode_callback`. The user-provided callback function can then provide a pointer to an encoding struct that contains the requisite functions that prism will use those to parse identifiers going forward.
 
 If the user-provided callback function returns `NULL` (the value also provided by the default implementation in case a callback was not registered), an error will be added to the parser's error list and parsing will continue on using the default UTF-8 encoding.
 
@@ -84,34 +84,34 @@ typedef struct {
 
     // Return true if the encoding is a multibyte encoding.
     bool multibyte;
-} yp_encoding_t;
+} pm_encoding_t;
 
-// When an encoding is encountered that isn't understood by YARP, we provide
+// When an encoding is encountered that isn't understood by prism, we provide
 // the ability here to call out to a user-defined function to get an encoding
 // struct. If the function returns something that isn't NULL, we set that to
 // our encoding and use it to parse identifiers.
-typedef yp_encoding_t *(*yp_encoding_decode_callback_t)(yp_parser_t *parser, const uint8_t *name, size_t width);
+typedef pm_encoding_t *(*pm_encoding_decode_callback_t)(pm_parser_t *parser, const uint8_t *name, size_t width);
 
-// Register a callback that will be called when YARP encounters a magic comment
+// Register a callback that will be called when prism encounters a magic comment
 // with an encoding referenced that it doesn't understand. The callback should
 // return NULL if it also doesn't understand the encoding or it should return a
-// pointer to a yp_encoding_t struct that contains the functions necessary to
+// pointer to a pm_encoding_t struct that contains the functions necessary to
 // parse identifiers.
-YP_EXPORTED_FUNCTION void
-yp_parser_register_encoding_decode_callback(yp_parser_t *parser, yp_encoding_decode_callback_t callback);
+PRISM_EXPORTED_FUNCTION void
+pm_parser_register_encoding_decode_callback(pm_parser_t *parser, pm_encoding_decode_callback_t callback);
 ```
 
 ## Getting notified when the encoding changes
 
-You may want to get notified when the encoding changes based on the result of parsing an encoding comment. We use this internally for our `lex` function in order to provide the correct encodings for the tokens that are returned. For that you can register a callback with `yp_parser_register_encoding_changed_callback`. The callback will be called with a pointer to the parser. The encoding can be accessed through `parser->encoding`.
+You may want to get notified when the encoding changes based on the result of parsing an encoding comment. We use this internally for our `lex` function in order to provide the correct encodings for the tokens that are returned. For that you can register a callback with `pm_parser_register_encoding_changed_callback`. The callback will be called with a pointer to the parser. The encoding can be accessed through `parser->encoding`.
 
 ```c
-// When the encoding that is being used to parse the source is changed by YARP,
+// When the encoding that is being used to parse the source is changed by prism,
 // we provide the ability here to call out to a user-defined function.
-typedef void (*yp_encoding_changed_callback_t)(yp_parser_t *parser);
+typedef void (*pm_encoding_changed_callback_t)(pm_parser_t *parser);
 
-// Register a callback that will be called whenever YARP changes the encoding it
-// is using to parse based on the magic comment.
-YP_EXPORTED_FUNCTION void
-yp_parser_register_encoding_changed_callback(yp_parser_t *parser, yp_encoding_changed_callback_t callback);
+// Register a callback that will be called whenever prism changes the encoding
+// it is using to parse based on the magic comment.
+PRISM_EXPORTED_FUNCTION void
+pm_parser_register_encoding_changed_callback(pm_parser_t *parser, pm_encoding_changed_callback_t callback);
 ```

data/docs/fuzzing.md

@@ -28,9 +28,9 @@ fuzz
 
 There are currently three fuzzing targets
 
-- `yp_parse_serialize` (parse)
-- `yp_regexp_named_capture_group_names` (regexp)
-- `yp_unescape_manipulate_string` (unescape)
+- `pm_parse_serialize` (parse)
+- `pm_regexp_named_capture_group_names` (regexp)
+- `pm_unescape_manipulate_string` (unescape)
 
 Respectively, fuzzing can be performed with
 
@@ -48,7 +48,7 @@ make fuzz-debug
 
 # Out-of-bounds reads
 
-Currently, encoding functionality implementing the `yp_encoding_t` interface can read outside of inputs. For the time being, ASAN instrumentation is disabled for functions from src/enc. See `fuzz/asan.ignore`.
+Currently, encoding functionality implementing the `pm_encoding_t` interface can read outside of inputs. For the time being, ASAN instrumentation is disabled for functions from src/enc. See `fuzz/asan.ignore`.
 
 To disable ASAN read instrumentation globally, use the `FUZZ_FLAGS` environment variable e.g.
 

data/docs/heredocs.md

@@ -4,7 +4,7 @@ Heredocs are one of the most complicated pieces of this parser. There are many d
 
 ## 1. Lexing the identifier
 
-When a heredoc identifier is encountered in the regular process of lexing, we push the `YP_LEX_HEREDOC` mode onto the stack with the following metadata:
+When a heredoc identifier is encountered in the regular process of lexing, we push the `PM_LEX_HEREDOC` mode onto the stack with the following metadata:
 
 * `ident_start`: A pointer to the start of the identifier for the heredoc. We need this to match against the end of the heredoc.
 * `ident_length`: The length of the identifier for the heredoc. We also need this to match.
@@ -16,7 +16,7 @@ Note that if the `parser.heredoc_end` field is already set, then it means we hav
 
 ## 2. Lexing the body
 
-The next time the lexer is asked for a token, it will be in the `YP_LEX_HEREDOC` mode. In this mode we are lexing the body of the heredoc. It will start by checking if the `next_start` field is set. If it is, then this is the first token within the body of the heredoc so we'll start lexing from there. Otherwise we'll start lexing from the end of the previous token.
+The next time the lexer is asked for a token, it will be in the `PM_LEX_HEREDOC` mode. In this mode we are lexing the body of the heredoc. It will start by checking if the `next_start` field is set. If it is, then this is the first token within the body of the heredoc so we'll start lexing from there. Otherwise we'll start lexing from the end of the previous token.
 
 Lexing these fields is extremely similar to lexing an interpolated string. The only difference is that we also do an additional check at the beginning of each line to check if we have hit the terminator.
 
@@ -33,4 +33,4 @@ Once the heredoc has been lexed, the lexer will resume lexing from the `next_sta
 
 ## Compatibility with Ripper
 
-The order in which tokens are emitted is different from that of Ripper. Ripper emits each token in the file in the order in which it appears. YARP instead will emit the tokens that makes the most sense for the lexer, using the process described above. Therefore to line things up, `YARP.lex_compat` will shuffle the tokens around to match Ripper's output.
+The order in which tokens are emitted is different from that of Ripper. Ripper emits each token in the file in the order in which it appears. prism instead will emit the tokens that makes the most sense for the lexer, using the process described above. Therefore to line things up, `Prism.lex_compat` will shuffle the tokens around to match Ripper's output.