yarp 0.11.0 → 0.13.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9d096d8bdda05ec360a58c8c6d98d1aeb6680b95a983a7d10d916d4a941be1c8
-  data.tar.gz: 7833c692ae2547f6628d15972b539484e06804179277d8f73a33cf448687f48b
+  metadata.gz: da8f3b5f2cdae92e669cfe7865566c2014b57746644fd550e23dfe27a7f4a5cd
+  data.tar.gz: 994f5db733b1261c76920a6426820dc941a72ba2619f71de0b7ac731da68d57d
 SHA512:
-  metadata.gz: 6c42795d64e15210922249c5b81a7bd8a4f04f6589f8266aaf73341afbd72bde016aeafa879eddb12dd0ae75b6e2b91bba7c28599495f31030031117d12be9cf
-  data.tar.gz: 6534173b9aa41bf180358750198c16749279723575045fffb819040509f6ca2e4f264653ad7857af152f64f7bdb16e95cbdedaab474bb62d86f9f920c5e44d83
+  metadata.gz: bc2bc26648b224d5195a1649f906a7d66d6772626b20481b17ccff76a224946b614919aa2d84afb1756ed6bae4e80e460715e25e0a1f0af41b5f5c02d217c99d
+  data.tar.gz: db5a41e5abd08dbd4184e1fb6284c4c414f9377894ef4cca689f15b10c388427d2d9e465067159abce7cb8e545bc32c3996983feb5f1ffbd340c84d48796c04b

data/CHANGELOG.md

@@ -6,6 +6,50 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 ## [Unreleased]
 
+## [0.13.0] - 2023-09-29
+
+### Added
+
+- `BEGIN {}` blocks are only allowed at the top-level, and will now provide a syntax error if they are not.
+- Numbered parameters are not allowed in block parameters, and will now provide a syntax error if they are.
+- Many more Ruby modules and classes are now documented. Also, many have been moved into their own files and autoloaded so that initial boot time of the gem is much faster.
+- `PM_TOKEN_METHOD_NAME` is introduced, used to indicate an identifier that if definitely a method name because it has an `!` or `?` at the end.
+- In the C API, arrays, assocs, and hashes now can have the `PM_NODE_FLAG_STATIC_LITERAL` flag attached if they can be compiled statically. This is used in CRuby, for example, to determine if a `duphash`/`duparray` instruction can be used as opposed to a `newhash`/`newarray`.
+- `Node#type` is introduced, which returns a symbol representing the type of the node. This is useful for case comparisons when you have to compare against multiple types.
+
+### Changed
+
+- **BREAKING**: Everything has been renamed to `prism` instead of `yarp`. The `yp_`/`YP_` prefix in the C API has been changed to `pm_`/`PM_`. For the most part, everything should be find/replaceable.
+- **BREAKING**: `BlockArgumentNode` nodes now go into the `block` field on `CallNode` nodes, in addition to the `BlockNode` nodes that used to be there. Hopefully this makes it more consistent to compile/deal with in general, but it does mean it can be a surprising breaking change.
+- Escaped whitespace in `%w` lists is now properly unescaped.
+- `Node#pretty_print` now respects pretty print indentation.
+- `Dispatcher` was previously firing `_leave` events in the incorrect order. This has now been fixed.
+- **BREAKING**: `Visitor` has now been split into `Visitor` and `Compiler`. The visitor visits nodes but doesn't return anything from the visit methods. It is suitable for taking action based on the tree, but not manipulating the tree itself. The `Compiler` visits nodes and returns the computed value up the tree. It is suitable for compiling the tree into another format. As such, `MutationVisitor` has been renamed to `MutationCompiler`.
+
+## [0.12.0] - 2023-09-15
+
+### Added
+
+- `RegularExpressionNode#options` and `InterpolatedRegularExpressionNode#options` are now provided. These return integers that match up to the `Regexp#options` API.
+- Greatly improved `Node#inspect` and `Node#pretty_print` APIs.
+- `MatchLastLineNode` and `InterpolatedMatchLastLineNode` are introduced to represent using a regular expression as the predicate of an `if` or `unless` statement.
+- `IntegerNode` now has a base flag on it.
+- Heredocs that were previously `InterpolatedStringNode` and `InterpolatedXStringNode` nodes without any actual interpolation are now `StringNode` and `XStringNode`, respectively.
+- `StringNode` now has a `frozen?` flag on it, which respects the `frozen_string_literal` magic comment.
+- Numbered parameters are now supported, and are properly represented using `LocalVariableReadNode` nodes.
+- `ImplicitNode` is introduced, which wraps implicit calls, local variable reads, or constant reads in omitted hash values.
+- `YARP::Dispatcher` is introduced, which provides a way for multiple objects to listen for certain events on the AST while it is being walked. This is effectively a way to implement a more efficient visitor pattern when you have many different uses for the AST.
+
+### Changed
+
+- **BREAKING**: Flags fields are now marked as private, to ensure we can change their implementation under the hood. Actually querying should be through the accessor methods.
+- **BREAKING**: `AliasNode` is now split into `AliasMethodNode` and `AliasGlobalVariableNode`.
+- Method definitions on local variables is now correctly handled.
+- Unary minus precedence has been fixed.
+- Concatenating character literals with string literals is now fixed.
+- Many more invalid syntaxes are now properly rejected.
+- **BREAKING**: Comments now no longer include their trailing newline.
+
 ## [0.11.0] - 2023-09-08
 
 ### Added
@@ -117,10 +161,12 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 - 🎉 Initial release! 🎉
 
-[unreleased]: https://github.com/ruby/yarp/compare/v0.11.0...HEAD
-[0.11.0]: https://github.com/ruby/yarp/compare/v0.10.0...v0.11.0
-[0.10.0]: https://github.com/ruby/yarp/compare/v0.9.0...v0.10.0
-[0.9.0]: https://github.com/ruby/yarp/compare/v0.8.0...v0.9.0
-[0.8.0]: https://github.com/ruby/yarp/compare/v0.7.0...v0.8.0
-[0.7.0]: https://github.com/ruby/yarp/compare/v0.6.0...v0.7.0
-[0.6.0]: https://github.com/ruby/yarp/compare/d60531...v0.6.0
+[unreleased]: https://github.com/ruby/prism/compare/v0.13.0...HEAD
+[0.13.0]: https://github.com/ruby/prism/compare/v0.12.0...v0.13.0
+[0.12.0]: https://github.com/ruby/prism/compare/v0.11.0...v0.12.0
+[0.11.0]: https://github.com/ruby/prism/compare/v0.10.0...v0.11.0
+[0.10.0]: https://github.com/ruby/prism/compare/v0.9.0...v0.10.0
+[0.9.0]: https://github.com/ruby/prism/compare/v0.8.0...v0.9.0
+[0.8.0]: https://github.com/ruby/prism/compare/v0.7.0...v0.8.0
+[0.7.0]: https://github.com/ruby/prism/compare/v0.6.0...v0.7.0
+[0.6.0]: https://github.com/ruby/prism/compare/d60531...v0.6.0

data/CONTRIBUTING.md

@@ -1,6 +1,6 @@
 # Contributing
 
-Thank you for your interest in contributing to YARP! Below are a couple of ways that you can help out.
+Thank you for your interest in contributing to prism! Below are a couple of ways that you can help out.
 
 ## Discussions
 
@@ -29,7 +29,7 @@ or explicitly running the `compile` task:
 ``` sh
 bundle exec rake compile test
 # or to just compile the C extension ...
-bundle exec rake compile:yarp test
+bundle exec rake compile:prism test
 ```
 
 To test the rust bindings (with caveats about setting up your Rust environment properly first):

data/Makefile

@@ -35,7 +35,7 @@ build/librubyparser.a: $(STATIC_OBJECTS)
 build/shared/%.o: src/%.c Makefile $(HEADERS)
 	$(ECHO) "compiling $@"
 	$(Q) mkdir -p $(@D)
-	$(Q) $(CC) $(DEBUG_FLAGS) -DYP_EXPORT_SYMBOLS $(CPPFLAGS) $(CFLAGS) -c -o $@ $<
+	$(Q) $(CC) $(DEBUG_FLAGS) -DPRISM_EXPORT_SYMBOLS $(CPPFLAGS) $(CFLAGS) -c -o $@ $<
 
 build/static/%.o: src/%.c Makefile $(HEADERS)
 	$(ECHO) "compiling $@"
@@ -55,20 +55,20 @@ build/fuzz.heisenbug.%: $(SOURCES) fuzz/%.c fuzz/heisenbug.c
 
 fuzz-debug:
 	$(ECHO) "entering debug shell"
-	$(Q) docker run -it --rm -e HISTFILE=/yarp/fuzz/output/.bash_history -v $(shell pwd):/yarp -v $(FUZZ_OUTPUT_DIR):/fuzz_output yarp/fuzz
+	$(Q) docker run -it --rm -e HISTFILE=/prism/fuzz/output/.bash_history -v $(shell pwd):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz
 
 fuzz-docker-build: fuzz/docker/Dockerfile
 	$(ECHO) "building docker image"
-	$(Q) docker build -t yarp/fuzz fuzz/docker/
+	$(Q) docker build -t prism/fuzz fuzz/docker/
 
 fuzz-run-%: FORCE fuzz-docker-build
 	$(ECHO) "generating templates"
 	$(Q) bundle exec rake templates
 	$(ECHO) "running $* fuzzer"
-	$(Q) docker run --rm -v $(shell pwd):/yarp yarp/fuzz /bin/bash -c "FUZZ_FLAGS=\"$(FUZZ_FLAGS)\" make build/fuzz.$*"
+	$(Q) docker run --rm -v $(shell pwd):/prism prism/fuzz /bin/bash -c "FUZZ_FLAGS=\"$(FUZZ_FLAGS)\" make build/fuzz.$*"
 	$(ECHO) "starting AFL++ run"
 	$(Q) mkdir -p $(FUZZ_OUTPUT_DIR)/$*
-	$(Q) docker run -it --rm -v $(shell pwd):/yarp -v $(FUZZ_OUTPUT_DIR):/fuzz_output yarp/fuzz /bin/bash -c "./fuzz/$*.sh /fuzz_output/$*"
+	$(Q) docker run -it --rm -v $(shell pwd):/prism -v $(FUZZ_OUTPUT_DIR):/fuzz_output prism/fuzz /bin/bash -c "./fuzz/$*.sh /fuzz_output/$*"
 FORCE:
 
 fuzz-clean:

data/README.md

@@ -1,4 +1,4 @@
-# Yet Another Ruby Parser
+# Prism Ruby parser
 
 This is a parser for the Ruby programming language. It is designed to be portable, error tolerant, and maintainable. It is written in C99 and has no dependencies. It is currently being integrated into [CRuby](https://github.com/ruby/ruby), [JRuby](https://github.com/jruby/jruby), [TruffleRuby](https://github.com/oracle/truffleruby), [Sorbet](https://github.com/sorbet/sorbet), and [Syntax Tree](https://github.com/ruby-syntax-tree/syntax_tree).
 
@@ -16,29 +16,29 @@ The repository contains the infrastructure for both a shared library (librubypar
 ├── config.yml            specification for tokens and nodes in the tree
 ├── docs                  documentation about the project
 ├── ext
-│   └── yarp
+│   └── prism
 │       ├── extconf.rb    configuration to generate the Makefile for the native extension
 │       └── extension.c   the native extension that interacts with librubyparser
 ├── fuzz                  files related to fuzz testing
 ├── include
-│   ├── yarp              header files for the shared library
-│   └── yarp.h            main header file for the shared library
+│   ├── prism             header files for the shared library
+│   └── prism.h           main header file for the shared library
 ├── java                  Java bindings for the shared library
 ├── lib
-│   ├── yarp              Ruby library files
-│   └── yarp.rb           main entrypoint for the Ruby library
+│   ├── prism             Ruby library files
+│   └── prism.rb          main entrypoint for the Ruby library
 ├── rakelib               various Rake tasks for the project
 ├── rust
-│   ├── yarp              Rustified crate for the shared library
-│   └── yarp-sys          FFI binding for Rust
+│   ├── prism             Rustified crate for the shared library
+│   └── prism-sys         FFI binding for Rust
 ├── src
 │   ├── enc               various encoding files
 │   ├── util              various utility files
-│   └── yarp.c            main entrypoint for the shared library
+│   └── prism.c           main entrypoint for the shared library
 ├── templates             contains ERB templates generated by templates/template.rb
 │   └── template.rb       generates code from the nodes and tokens configured by config.yml
 └── test
-    └── yarp
+    └── prism
         ├── fixtures      Ruby code used for testing
         └── snapshots     snapshots of generated syntax trees corresponding to fixtures
 ```
@@ -48,7 +48,7 @@ The repository contains the infrastructure for both a shared library (librubypar
 To compile the shared library, you will need:
 
 * A C99 compiler
-* autotools autoconf, automake, libtool)
+* autotools (autoconf, automake, libtool)
 * make
 * Ruby 3.3.0-preview1 or later
 
@@ -87,4 +87,3 @@ See the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information. We additio
 * [Ruby API](docs/ruby_api.md)
 * [Serialization](docs/serialization.md)
 * [Testing](docs/testing.md)
-

data/config.yml

@@ -232,6 +232,8 @@ tokens:
     comment: "<<"
   - name: LESS_LESS_EQUAL
     comment: "<<="
+  - name: METHOD_NAME
+    comment: "a method name"
   - name: MINUS
     comment: "-"
   - name: MINUS_EQUAL
@@ -333,6 +335,16 @@ flags:
         comment: "&. operator"
       - name: VARIABLE_CALL
         comment: "a call that could have been a local variable"
+  - name: IntegerBaseFlags
+    values:
+      - name: BINARY
+        comment: "0b prefix"
+      - name: OCTAL
+        comment: "0o or 0 prefix"
+      - name: DECIMAL
+        comment: "0d or no prefix"
+      - name: HEXADECIMAL
+        comment: "0x prefix"
   - name: LoopFlags
     values:
       - name: BEGIN_MODIFIER
@@ -345,10 +357,10 @@ flags:
     values:
       - name: IGNORE_CASE
         comment: "i - ignores the case of characters when matching"
-      - name: MULTI_LINE
-        comment: "m - allows $ to match the end of lines within strings"
       - name: EXTENDED
         comment: "x - ignores whitespace and allows comments in regular expressions"
+      - name: MULTI_LINE
+        comment: "m - allows $ to match the end of lines within strings"
       - name: EUC_JP
         comment: "e - forces the EUC-JP encoding"
       - name: ASCII_8BIT
@@ -359,8 +371,12 @@ flags:
         comment: "u - forces the UTF-8 encoding"
       - name: ONCE
         comment: "o - only interpolates values into the regular expression once"
+  - name: StringFlags
+    values:
+      - name: FROZEN
+        comment: "frozen by virtue of a frozen_string_literal comment"
 nodes:
-  - name: AliasNode
+  - name: AliasGlobalVariableNode
     fields:
       - name: new_name
         type: node
@@ -369,7 +385,20 @@ nodes:
       - name: keyword_loc
         type: location
     comment: |
-      Represents the use of the `alias` keyword.
+      Represents the use of the `alias` keyword to alias a global variable.
+
+          alias $foo $bar
+          ^^^^^^^^^^^^^^^
+  - name: AliasMethodNode
+    fields:
+      - name: new_name
+        type: node
+      - name: old_name
+        type: node
+      - name: keyword_loc
+        type: location
+    comment: |
+      Represents the use of the `alias` keyword to alias a method.
 
           alias foo bar
           ^^^^^^^^^^^^^
@@ -641,7 +670,6 @@ nodes:
         type: location?
       - name: block
         type: node?
-        kind: BlockNode
       - name: flags
         type: flags
         kind: CallNodeFlags
@@ -1386,6 +1414,19 @@ nodes:
 
           1.0i
           ^^^^
+  - name: ImplicitNode
+    fields:
+      - name: value
+        type: node
+    comment: |
+      Represents a node that is implicitly being added to the tree but doesn't
+      correspond directly to a node in the source.
+
+          { foo: }
+            ^^^^
+
+          { Foo: }
+            ^^^^
   - name: InNode
     fields:
       - name: pattern
@@ -1483,11 +1524,34 @@ nodes:
           @foo = 1
           ^^^^^^^^
   - name: IntegerNode
+    fields:
+      - name: flags
+        type: flags
+        kind: IntegerBaseFlags
     comment: |
       Represents an integer number literal.
 
           1
           ^
+  - name: InterpolatedMatchLastLineNode
+    fields:
+      - name: opening_loc
+        type: location
+      - name: parts
+        type: node[]
+      - name: closing_loc
+        type: location
+      - name: flags
+        type: flags
+        kind: RegularExpressionFlags
+    newline: parts
+    comment: |
+      Represents a regular expression literal that contains interpolation that
+      is being used in the predicate of a conditional to implicitly match
+      against the last line read by an IO object.
+
+          if /foo #{bar} baz/ then end
+             ^^^^^^^^^^^^^^^^
   - name: InterpolatedRegularExpressionNode
     fields:
       - name: opening_loc
@@ -1702,6 +1766,27 @@ nodes:
 
           foo = 1
           ^^^^^^^
+  - name: MatchLastLineNode
+    fields:
+      - name: opening_loc
+        type: location
+      - name: content_loc
+        type: location
+        semantic_field: true # https://github.com/ruby/prism/issues/1452
+      - name: closing_loc
+        type: location
+      - name: unescaped
+        type: string
+      - name: flags
+        type: flags
+        kind: RegularExpressionFlags
+    comment: |
+      Represents a regular expression literal used in the predicate of a
+      conditional to implicitly match against the last line read by an IO
+      object.
+
+          if /foo/i then end
+             ^^^^^^
   - name: MatchPredicateNode
     fields:
       - name: value
@@ -1728,6 +1813,19 @@ nodes:
 
           foo => bar
           ^^^^^^^^^^
+  - name: MatchWriteNode
+    fields:
+      - name: call
+        type: node
+        kind: CallNode
+      - name: locals
+        type: constant[]
+    comment: |
+      Represents writing local variables using a regular expression match with
+      named capture groups.
+
+          /(?<foo>bar)/ =~ baz
+          ^^^^^^^^^^^^^^^^^^^^
   - name: MissingNode
     comment: |
       Represents a node that is missing from the source and results in a syntax
@@ -1855,11 +1953,11 @@ nodes:
         type: node[]
       - name: optionals
         type: node[]
-      - name: posts
-        type: node[]
       - name: rest
         type: node?
         kind: RestParameterNode
+      - name: posts
+        type: node[]
       - name: keywords
         type: node[]
       - name: keyword_rest
@@ -1995,6 +2093,7 @@ nodes:
         type: location
       - name: content_loc
         type: location
+        semantic_field: true # https://github.com/ruby/prism/issues/1452
       - name: closing_loc
         type: location
       - name: unescaped
@@ -2183,10 +2282,15 @@ nodes:
           ^^^^^^^^^^^
   - name: StringNode
     fields:
+      - name: flags
+        type: flags
+        kind: StringFlags
       - name: opening_loc
         type: location?
+        semantic_field: true # https://github.com/ruby/prism/issues/1452
       - name: content_loc
         type: location
+        semantic_field: true # https://github.com/ruby/prism/issues/1452
       - name: closing_loc
         type: location?
       - name: unescaped
@@ -2216,7 +2320,6 @@ nodes:
         type: location?
       - name: block
         type: node?
-        kind: BlockNode
     comment: |
       Represents the use of the `super` keyword with parentheses or arguments.
 

data/docs/build_system.md

@@ -1,17 +1,17 @@
 # Build System
 
-There are many ways to build YARP, which means the build system is a bit more complicated than usual.
+There are many ways to build prism, which means the build system is a bit more complicated than usual.
 
 ## Requirements
 
-* It must work to build YARP for all 6 uses-cases below.
-* It must be possible to build YARP without needing ruby/rake/etc.
-  Because once YARP is the single parser in TruffleRuby, JRuby or CRuby there won't be another Ruby parser around to parse such Ruby code.
+* It must work to build prism for all 6 uses-cases below.
+* It must be possible to build prism without needing ruby/rake/etc.
+  Because once prism is the single parser in TruffleRuby, JRuby or CRuby there won't be another Ruby parser around to parse such Ruby code.
   Most/every Ruby implementations want to avoid depending on another Ruby during the build process as that is very brittle.
-* It is desirable to compile YARP with the same or very similar compiler flags for all use-cases (e.g. optimization level, warning flags, etc).
-  Otherwise, there is the risk YARP does not work correctly with those different compiler flags.
+* It is desirable to compile prism with the same or very similar compiler flags for all use-cases (e.g. optimization level, warning flags, etc).
+  Otherwise, there is the risk prism does not work correctly with those different compiler flags.
 
-The main solution for the second point seems a Makefile, otherwise many of the usages would have to duplicate the logic to build YARP.
+The main solution for the second point seems a Makefile, otherwise many of the usages would have to duplicate the logic to build prism.
 
 ## General Design
 
@@ -24,15 +24,15 @@ This way there is minimal duplication, and each layer builds on the previous one
 
 The static library exports no symbols, to avoid any conflict.
 The shared library exports some symbols, and this is fine since there should only be one librubyparser shared library
-loaded per process (i.e., at most one version of the yarp *gem* loaded in a process, only the gem uses the shared library).
+loaded per process (i.e., at most one version of the prism *gem* loaded in a process, only the gem uses the shared library).
 
-## The various ways to build YARP
+## The various ways to build prism
 
-### Building from ruby/yarp repository with `bundle exec rake`
+### Building from ruby/prism repository with `bundle exec rake`
 
 `rake` calls `make` and then uses `Rake::ExtensionTask` to compile the C extension (see above).
 
-### Building the yarp gem by `gem install/bundle install`
+### Building the prism gem by `gem install/bundle install`
 
 The gem contains the pre-generated templates.
 When installing the gem, `extconf.rb` is used and that:
@@ -44,31 +44,31 @@ there is Ruby code using FFI which uses `librubyparser.{so,dylib,dll}`
 to implement the same methods as the C extension, but using serialization instead of many native calls/accesses
 (JRuby does not support C extensions, serialization is faster on TruffleRuby than the C extension).
 
-### Building the yarp gem from git, e.g. `gem 'yarp', github: 'ruby/yarp'`
+### Building the prism gem from git, e.g. `gem "prism", github: "ruby/prism"`
 
 The same as above, except the `extconf.rb` additionally runs first:
 * `templates/template.rb` to generate the templates
 
 Because of course those files are not part of the git repository.
 
-### Building YARP as part of CRuby
+### Building prism as part of CRuby
 
-[This script](https://github.com/ruby/ruby/blob/32e828bb4a6c65a392b2300f3bdf93008c7b6f25/tool/sync_default_gems.rb#L399-L426) imports YARP sources in CRuby.
+[This script](https://github.com/ruby/ruby/blob/32e828bb4a6c65a392b2300f3bdf93008c7b6f25/tool/sync_default_gems.rb#L399-L426) imports prism sources in CRuby.
 
 The script generates the templates when importing.
 
-YARP's `Makefile` is not used at all in CRuby. Instead, CRuby's `Makefile` is used.
+prism's `Makefile` is not used at all in CRuby. Instead, CRuby's `Makefile` is used.
 
-### Building YARP as part of TruffleRuby
+### Building prism as part of TruffleRuby
 
-[This script](https://github.com/oracle/truffleruby/blob/master/tool/import-yarp.sh) imports YARP sources in TruffleRuby.
+[This script](https://github.com/oracle/truffleruby/blob/master/tool/import-prism.sh) imports prism sources in TruffleRuby.
 The script generates the templates when importing.
 
-Then when `mx build` builds TruffleRuby and the `yarp` mx project inside, it runs `make`.
+Then when `mx build` builds TruffleRuby and the `prism` mx project inside, it runs `make`.
 
-Then the `yarp bindings` mx project is built, which contains the [bindings](https://github.com/oracle/truffleruby/blob/master/src/main/c/yarp_bindings/src/yarp_bindings.c)
-and links to `librubyparser.a` (to avoid exporting symbols, so no conflict when installing the yarp gem).
+Then the `prism bindings` mx project is built, which contains the [bindings](https://github.com/oracle/truffleruby/blob/master/src/main/c/prism_bindings/src/prism_bindings.c)
+and links to `librubyparser.a` (to avoid exporting symbols, so no conflict when installing the prism gem).
 
-### Building YARP as part of JRuby
+### Building prism as part of JRuby
 
 TODO, probably similar to TruffleRuby.

data/docs/building.md

@@ -1,13 +1,13 @@
 # Building
 
-The following describes how to build YARP from source.
+The following describes how to build prism from source.
 This comes directly from the [Makefile](../Makefile).
 
 ## Common
 
 All of the source files match `src/**/*.c` and all of the headers match `include/**/*.h`.
 
-The following flags should be used to compile YARP:
+The following flags should be used to compile prism:
 
 * `-std=c99` - Use the C99 standard
 * `-Wall -Wconversion -Wextra -Wpedantic -Wundef` - Enable the warnings we care about
@@ -16,7 +16,7 @@ The following flags should be used to compile YARP:
 
 ## Shared
 
-If you want to build YARP as a shared library and link against it, you should compile with:
+If you want to build prism as a shared library and link against it, you should compile with:
 
 * `-fPIC -shared` - Compile as a shared library
-* `-DYP_EXPORT_SYMBOLS` - Export the symbols (by default nothing is exported)
+* `-DPRISM_EXPORT_SYMBOLS` - Export the symbols (by default nothing is exported)

data/docs/configuration.md

@@ -1,15 +1,19 @@
 # Configuration
 
-A lot of code in YARP's repository is templated from a single configuration file, [config.yml](../config.yml). This file is used to generate the following files:
-
-* `ext/yarp/api_node.c` - for defining how to build Ruby objects for the nodes out of C structs
-* `include/yarp/ast.h` - for defining the C structs that represent the nodes
-* `java/org/yarp/AbstractNodeVisitor.java` - for defining the visitor interface for the nodes in Java
-* `java/org/yarp/Loader.java` - for defining how to deserialize the nodes in Java
-* `java/org/yarp/Nodes.java` - for defining the nodes in Java
-* `lib/yarp/mutation_visitor.rb` - for defining the mutation visitor for the nodes in Ruby
-* `lib/yarp/node.rb` - for defining the nodes in Ruby
-* `lib/yarp/serialize.rb` - for defining how to deserialize the nodes in Ruby
+A lot of code in prism's repository is templated from a single configuration file, [config.yml](../config.yml). This file is used to generate the following files:
+
+* `ext/prism/api_node.c` - for defining how to build Ruby objects for the nodes out of C structs
+* `include/prism/ast.h` - for defining the C structs that represent the nodes
+* `java/org/prism/AbstractNodeVisitor.java` - for defining the visitor interface for the nodes in Java
+* `java/org/prism/Loader.java` - for defining how to deserialize the nodes in Java
+* `java/org/prism/Nodes.java` - for defining the nodes in Java
+* `lib/prism/compiler.rb` - for defining the compiler for the nodes in Ruby
+* `lib/prism/dispatcher.rb` - for defining the dispatch visitors for the nodes in Ruby
+* `lib/prism/dsl.rb` - for defining the DSL for the nodes in Ruby
+* `lib/prism/mutation_compiler.rb` - for defining the mutation compiler for the nodes in Ruby
+* `lib/prism/node.rb` - for defining the nodes in Ruby
+* `lib/prism/serialize.rb` - for defining how to deserialize the nodes in Ruby
+* `lib/prism/visitor.rb` - for defining the visitor interface for the nodes in Ruby
 * `src/node.c` - for defining how to free the nodes in C and calculate the size in memory in C
 * `src/prettyprint.c` - for defining how to prettyprint the nodes in C
 * `src/serialize.c` - for defining how to serialize the nodes in C
@@ -25,7 +29,7 @@ This is a list of tokens to be used by the lexer. It is shared here so that it c
 
 Each token is expected to have a `name` key and a `comment` key (both as strings). Optionally they can have a `value` key (an integer) which is used to represent the value in the enum.
 
-In C these tokens will be templated out with the prefix `YP_TOKEN_`. For example, if you have a `name` key with the value `PERCENT`, you can access this in C through `YP_TOKEN_PERCENT`.
+In C these tokens will be templated out with the prefix `PM_TOKEN_`. For example, if you have a `name` key with the value `PERCENT`, you can access this in C through `PM_TOKEN_PERCENT`.
 
 ## `flags`
 
@@ -33,7 +37,7 @@ Sometimes we need to communicate more information in the tree than can be repres
 
 Each flag is expected to have a `name` key (a string) and a `values` key (an array). Each value in the `values` key should be an object that contains both a `name` key (a string) that represents the name of the flag and a `comment` key (a string) that represents the comment for the flag.
 
-In C these flags will get templated out with a `YP_` prefix, then a snake-case version of the flag name, then the flag itself. For example, if you have a flag with the name `RegularExpressionFlags` and a value with the name `IGNORE_CASE`, you can access this in C through `YP_REGULAR_EXPRESSION_FLAGS_IGNORE_CASE`.
+In C these flags will get templated out with a `PM_` prefix, then a snake-case version of the flag name, then the flag itself. For example, if you have a flag with the name `RegularExpressionFlags` and a value with the name `IGNORE_CASE`, you can access this in C through `PM_REGULAR_EXPRESSION_FLAGS_IGNORE_CASE`.
 
 ## `nodes`
 
@@ -43,14 +47,14 @@ Optionally, every node can define a `child_nodes` key that is an array. This arr
 
 The available values for `type` are:
 
-* `node` - A child node that is a node itself. This is a `yp_node_t *` in C.
-* `node?` - A child node that is optionally present. This is also a `yp_node_t *` in C, but can be `NULL`.
-* `node[]` - A child node that is an array of nodes. This is a `yp_node_list_t` in C.
-* `string` - A child node that is a string. For example, this is used as the name of the method in a call node, since it cannot directly reference the source string (as in `@-` or `foo=`). This is a `yp_string_t` in C.
-* `constant` - A variable-length integer that represents an index in the constant pool. This is a `yp_constant_id_t` in C.
-* `constant[]` - A child node that is an array of constants. This is a `yp_constant_id_list_t` in C.
-* `location` - A child node that is a location. This is a `yp_location_t` in C.
-* `location?` - A child node that is a location that is optionally present. This is a `yp_location_t` in C, but if the value is not present then the `start` and `end` fields will be `NULL`.
+* `node` - A child node that is a node itself. This is a `pm_node_t *` in C.
+* `node?` - A child node that is optionally present. This is also a `pm_node_t *` in C, but can be `NULL`.
+* `node[]` - A child node that is an array of nodes. This is a `pm_node_list_t` in C.
+* `string` - A child node that is a string. For example, this is used as the name of the method in a call node, since it cannot directly reference the source string (as in `@-` or `foo=`). This is a `pm_string_t` in C.
+* `constant` - A variable-length integer that represents an index in the constant pool. This is a `pm_constant_id_t` in C.
+* `constant[]` - A child node that is an array of constants. This is a `pm_constant_id_list_t` in C.
+* `location` - A child node that is a location. This is a `pm_location_t` in C.
+* `location?` - A child node that is a location that is optionally present. This is a `pm_location_t` in C, but if the value is not present then the `start` and `end` fields will be `NULL`.
 * `uint32` - A child node that is a 32-bit unsigned integer. This is a `uint32_t` in C.
 
-If the type is `node` or `node?` then the value also accepts an optional `kind` key (a string). This key is expected to match to the name of another node type within `config.yml`. This changes a couple of places where code is templated out to use the more specific struct name instead of the generic `yp_node_t`. For example, with `kind: StatementsNode` the `yp_node_t *` in C becomes a `yp_statements_node_t *`.
+If the type is `node` or `node?` then the value also accepts an optional `kind` key (a string). This key is expected to match to the name of another node type within `config.yml`. This changes a couple of places where code is templated out to use the more specific struct name instead of the generic `pm_node_t`. For example, with `kind: StatementsNode` the `pm_node_t *` in C becomes a `pm_statements_node_t *`.

data/docs/design.md

@@ -12,7 +12,7 @@ The design of the parser is based around these main goals.
 
 The first piece to understand about the parser is the design of its syntax tree. This is documented in `config.yml`. Every token and node is defined in that file, along with comments about where they are found in what kinds of syntax. This file is used to template out a lot of different files, all found in the `templates` directory. The `templates/template.rb` script performs the templating and outputs all files matching the directory structure found in the templates directory.
 
-The templated files contain all of the code required to allocate and initialize nodes, pretty print nodes, and serialize nodes. This means for the most part, you will only need to then hook up the parser to call the templated functions to create the nodes in the correct position. That means editing the parser itself, which is housed in `yarp.c`.
+The templated files contain all of the code required to allocate and initialize nodes, pretty print nodes, and serialize nodes. This means for the most part, you will only need to then hook up the parser to call the templated functions to create the nodes in the correct position. That means editing the parser itself, which is housed in `prism.c`.
 
 ## Pratt parsing
 
@@ -24,7 +24,7 @@ In order to provide the best possible error tolerance, the parser is hand-writte
 * https://matklad.github.io/2020/04/13/simple-but-powerful-pratt-parsing.html
 * https://chidiwilliams.com/post/on-recursive-descent-and-pratt-parsing/
 
-You can find most of the functions that correspond to constructs in the Pratt parsing algorithm in `yarp.c`. As a couple of examples:
+You can find most of the functions that correspond to constructs in the Pratt parsing algorithm in `prism.c`. As a couple of examples:
 
 * `parse` corresponds to the `parse_expression` function
 * `nud` (null denotation) corresponds to the `parse_expression_prefix` function