prism 0.19.0 → 0.24.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (69) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +102 -1
  3. data/Makefile +5 -0
  4. data/README.md +9 -6
  5. data/config.yml +236 -38
  6. data/docs/build_system.md +19 -2
  7. data/docs/cruby_compilation.md +27 -0
  8. data/docs/parser_translation.md +34 -0
  9. data/docs/parsing_rules.md +19 -0
  10. data/docs/releasing.md +84 -16
  11. data/docs/ruby_api.md +1 -1
  12. data/docs/ruby_parser_translation.md +19 -0
  13. data/docs/serialization.md +19 -5
  14. data/ext/prism/api_node.c +1989 -1525
  15. data/ext/prism/extension.c +130 -30
  16. data/ext/prism/extension.h +2 -2
  17. data/include/prism/ast.h +1700 -505
  18. data/include/prism/defines.h +8 -0
  19. data/include/prism/diagnostic.h +49 -7
  20. data/include/prism/encoding.h +17 -0
  21. data/include/prism/options.h +40 -14
  22. data/include/prism/parser.h +34 -18
  23. data/include/prism/util/pm_buffer.h +9 -0
  24. data/include/prism/util/pm_constant_pool.h +18 -0
  25. data/include/prism/util/pm_newline_list.h +4 -14
  26. data/include/prism/util/pm_strpbrk.h +4 -1
  27. data/include/prism/version.h +2 -2
  28. data/include/prism.h +19 -2
  29. data/lib/prism/debug.rb +11 -5
  30. data/lib/prism/desugar_compiler.rb +225 -80
  31. data/lib/prism/dot_visitor.rb +36 -14
  32. data/lib/prism/dsl.rb +302 -299
  33. data/lib/prism/ffi.rb +107 -76
  34. data/lib/prism/lex_compat.rb +17 -1
  35. data/lib/prism/node.rb +4580 -2607
  36. data/lib/prism/node_ext.rb +27 -4
  37. data/lib/prism/parse_result.rb +75 -29
  38. data/lib/prism/serialize.rb +633 -305
  39. data/lib/prism/translation/parser/compiler.rb +1838 -0
  40. data/lib/prism/translation/parser/lexer.rb +335 -0
  41. data/lib/prism/translation/parser/rubocop.rb +45 -0
  42. data/lib/prism/translation/parser.rb +190 -0
  43. data/lib/prism/translation/parser33.rb +12 -0
  44. data/lib/prism/translation/parser34.rb +12 -0
  45. data/lib/prism/translation/ripper.rb +696 -0
  46. data/lib/prism/translation/ruby_parser.rb +1521 -0
  47. data/lib/prism/translation.rb +11 -0
  48. data/lib/prism.rb +1 -1
  49. data/prism.gemspec +18 -7
  50. data/rbi/prism.rbi +150 -88
  51. data/rbi/prism_static.rbi +15 -3
  52. data/sig/prism.rbs +996 -961
  53. data/sig/prism_static.rbs +123 -46
  54. data/src/diagnostic.c +264 -219
  55. data/src/encoding.c +21 -26
  56. data/src/node.c +2 -6
  57. data/src/options.c +29 -5
  58. data/src/prettyprint.c +176 -44
  59. data/src/prism.c +1499 -564
  60. data/src/serialize.c +35 -21
  61. data/src/token_type.c +353 -4
  62. data/src/util/pm_buffer.c +11 -0
  63. data/src/util/pm_constant_pool.c +37 -11
  64. data/src/util/pm_newline_list.c +6 -15
  65. data/src/util/pm_string.c +0 -7
  66. data/src/util/pm_strpbrk.c +122 -14
  67. metadata +16 -5
  68. data/docs/building.md +0 -29
  69. data/lib/prism/ripper_compat.rb +0 -207
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 72e48ffbc7a58cb622a709f860470c4cf2178113a2ba6a2c9cd4f838229ccff8
4
- data.tar.gz: 709ecc0c274a141a868661f94d28aad3fda3032e1b253b2f699a15ff71200c37
3
+ metadata.gz: 827194fd8b477f9f021677ca386e35fd8300daa86a09fc11fe66a85ee54da97d
4
+ data.tar.gz: 2a44a3cf5e31260a949965aaf2aa5fba0cd8b1ac910ae694d22fd8e42f4d6b9b
5
5
  SHA512:
6
- metadata.gz: 42a21574ff5ca1ce62c836839a8c54ed7d6a3f943cfbbd13d860aa30f3d1cf2b6fd5add3082d7622b30e3403ecbf98a404f3c9ef831744e60d40deabeae44ba2
7
- data.tar.gz: 8ef39e9818ba2f13808a0e1131514709e6c7cbbbf2a0c784d047637d56b0024d35951a852c91d8f29c72477b5e923a6cc2687335cf44ccc761ab2e8e5eb5e521
6
+ metadata.gz: 41dab6cd85f9f57bf2e2256e7b7fd7ac44c46883ad6d79c0056bff3b34500b2ecb28a1febeb1deed5a49b862c4f4382856ad2195639f86e770c006d1b6a41f8e
7
+ data.tar.gz: 0474d64c9bbd3385f65b00bf83e6b2d3c9543785c85b42d50bce59327e6af7a8fc13d54e5b485b6c930997dabe0fd81c43835243fc96fb28beb3523395ddde71
data/CHANGELOG.md CHANGED
@@ -6,6 +6,102 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.24.0] - 2024-02-15
10
+
11
+ ### Added
12
+
13
+ - More support for `Prism::Translation::Ripper` is added.
14
+ - Support multiple versions for `Prism::Translation::Parser`.
15
+ - Improved memory usage in the FFI backend.
16
+ - Very large speed and memory improvements for creating the Ruby AST.
17
+
18
+ ### Changed
19
+
20
+ - Fix location for empty symbol in hash key.
21
+ - Fix parsing a rescue modifier on the value of an assignment when the LHS is a method call with arguments and no parentheses.
22
+
23
+ ## [0.23.0] - 2024-02-14
24
+
25
+ ### Added
26
+
27
+ - More support for `Prism::RipperCompat` is added.
28
+ - A significantly faster offset cache for `Prism::Translation::Parser` is added for files with multibyte characters.
29
+ - `Prism::Translation::RubyParser` is added.
30
+ - `Prism::ConstantPathTarget#full_name` is added.
31
+ - `version: "3.4.0"` is added as an option that is an alias for `version: "latest"`.
32
+ - Four new APIs are added to `Prism::Location`:
33
+ - `Prism::Location#start_code_units_offset`
34
+ - `Prism::Location#end_code_units_offset`
35
+ - `Prism::Location#start_code_units_column`
36
+ - `Prism::Location#end_code_units_column`
37
+ - Invalid multibyte characters are now validated within strings, lists, and heredocs.
38
+
39
+ ### Changed
40
+
41
+ - When defining `def !@`, the `name_loc` was previously only pointing to `!`, but now includes the `@`. The `name` is the same.
42
+ - `Prism::RipperCompat` has been moved to `Prism::Translation::Ripper`.
43
+ - Many of the error messages that prism produces have been changed to match the error messages that CRuby produces.
44
+
45
+ ## [0.22.0] - 2024-02-07
46
+
47
+ ### Added
48
+
49
+ - More support for `Prism::RipperCompat` is added.
50
+ - Support for Ruby 2.7 has been added, and the minimum Ruby requirement has been lowered to 2.7.
51
+
52
+ ### Changed
53
+
54
+ - The error for an invalid source encoding has a new `:argument` level to indicate it raises an argument error.
55
+ - `BeginNode` nodes that are used when a class, singleton class, module, method definition, or block have an inline `rescue`/`ensure`/`else` now have their opening locations set to the beginning of the respective keyword.
56
+ - Improved error messages for invalid characters.
57
+ - `Prism.parse_file` and similar APIs will raise more appropriate errors when the file does not exist or cannot be mapped.
58
+ - Correctly handle the `recover` parameter for `Prism::Translation::Parser`.
59
+
60
+ ## [0.21.0] - 2024-02-05
61
+
62
+ ### Added
63
+
64
+ - Add the `pm_constant_pool_find` API for finding a constant.
65
+
66
+ ### Changed
67
+
68
+ - Fixes for `Prism::Translation::Parser`.
69
+ - Ensure all errors flow through `parser.diagnostics.process`.
70
+ - Fix the find pattern node.
71
+ - Fix block forwarding with `NumberedParametersNode`.
72
+ - Ensure we can parse strings with invalid bytes for the encoding.
73
+ - Fix hash pairs in pattern matching.
74
+ - Properly reject operator writes on operator calls, e.g., `a.+ -= b`.
75
+ - Fix multi-byte escapes.
76
+ - Handle missing body in `begin` within the receiver of a method call.
77
+
78
+ ## [0.20.0] - 2024-02-01
79
+
80
+ ### Added
81
+
82
+ - String literal hash keys are now marked as frozen as static literal.
83
+ - `IndexTargetNode` now always has the `ATTRIBUTE_WRITE` flag.
84
+ - `Call*Node` nodes now have an `IGNORE_VISIBILITY` flag.
85
+ - We now support the `it` default parameter.
86
+ - Errors and warnings now have levels associated with them.
87
+ - Symbols now have correct encoding flags.
88
+ - We have now merged `parser-prism` in, which provides translation to the `whitequark/parser` AST.
89
+ - We now emit errors for invalid method definition receivers.
90
+
91
+ ### Changed
92
+
93
+ - We now emit errors on invalid pinned local variables.
94
+ - When passed scopes, it is now assumed that the innermost scope is the current binding.
95
+ - We now provide better error recovery for non terminated heredocs.
96
+ - Fix for `RationalNode#value` for non-decimal integers.
97
+ - Unary symbols `!@` and `~@` now unescape to `!` and `~`, respectively.
98
+ - `frozen_string_literal: false` now works properly.
99
+
100
+ ### Removed
101
+
102
+ - We've removed the `locals_body_index` field.
103
+ - We've removed the `verbose` option on the various parse APIs. Warnings are now always emitted with their associated level so that consumers can decide how to handle them.
104
+
9
105
  ## [0.19.0] - 2023-12-14
10
106
 
11
107
  ### Added
@@ -296,7 +392,12 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
296
392
 
297
393
  - 🎉 Initial release! 🎉
298
394
 
299
- [unreleased]: https://github.com/ruby/prism/compare/v0.19.0...HEAD
395
+ [unreleased]: https://github.com/ruby/prism/compare/v0.24.0...HEAD
396
+ [0.24.0]: https://github.com/ruby/prism/compare/v0.23.0...v0.24.0
397
+ [0.23.0]: https://github.com/ruby/prism/compare/v0.22.0...v0.23.0
398
+ [0.22.0]: https://github.com/ruby/prism/compare/v0.21.0...v0.22.0
399
+ [0.21.0]: https://github.com/ruby/prism/compare/v0.20.0...v0.21.0
400
+ [0.20.0]: https://github.com/ruby/prism/compare/v0.19.0...v0.20.0
300
401
  [0.19.0]: https://github.com/ruby/prism/compare/v0.18.0...v0.19.0
301
402
  [0.18.0]: https://github.com/ruby/prism/compare/v0.17.1...v0.18.0
302
403
  [0.17.1]: https://github.com/ruby/prism/compare/v0.17.0...v0.17.1
data/Makefile CHANGED
@@ -25,6 +25,7 @@ all: shared static
25
25
  shared: build/libprism.$(SOEXT)
26
26
  static: build/libprism.a
27
27
  wasm: javascript/src/prism.wasm
28
+ java-wasm: java-wasm/src/test/resources/prism.wasm
28
29
 
29
30
  build/libprism.$(SOEXT): $(SHARED_OBJECTS)
30
31
  $(ECHO) "linking $@"
@@ -38,6 +39,10 @@ javascript/src/prism.wasm: Makefile $(SOURCES) $(HEADERS)
38
39
  $(ECHO) "building $@"
39
40
  $(Q) $(WASI_SDK_PATH)/bin/clang --sysroot=$(WASI_SDK_PATH)/share/wasi-sysroot/ $(DEBUG_FLAGS) -DPRISM_EXPORT_SYMBOLS -D_WASI_EMULATED_MMAN -lwasi-emulated-mman $(CPPFLAGS) $(CFLAGS) -Wl,--export-all -Wl,--no-entry -mexec-model=reactor -o $@ $(SOURCES)
40
41
 
42
+ java-wasm/src/test/resources/prism.wasm: Makefile $(SOURCES) $(HEADERS)
43
+ $(ECHO) "building $@"
44
+ $(Q) $(WASI_SDK_PATH)/bin/clang $(DEBUG_FLAGS) -DPRISM_EXPORT_SYMBOLS -D_WASI_EMULATED_MMAN -lwasi-emulated-mman $(CPPFLAGS) $(CFLAGS) -Wl,--export-all -Wl,--no-entry -mexec-model=reactor -lc++ -lc++abi -o $@ $(SOURCES)
45
+
41
46
  build/shared/%.o: src/%.c Makefile $(HEADERS)
42
47
  $(ECHO) "compiling $@"
43
48
  $(Q) mkdir -p $(@D)
data/README.md CHANGED
@@ -32,8 +32,8 @@ The repository contains the infrastructure for both a shared library (libprism)
32
32
  │   └── prism.rb main entrypoint for the Ruby library
33
33
  ├── rakelib various Rake tasks for the project
34
34
  ├── rust
35
- │   ├── prism Rustified crate for the shared library
36
- │   └── prism-sys FFI binding for Rust
35
+ │   ├── ruby-prism Rustified crate for the shared library
36
+ │   └── ruby-prism-sys FFI binding for Rust
37
37
  ├── src
38
38
  │   ├── enc various encoding files
39
39
  │   ├── util various utility files
@@ -50,10 +50,9 @@ The repository contains the infrastructure for both a shared library (libprism)
50
50
 
51
51
  To compile the shared library, you will need:
52
52
 
53
- * A C99 compiler
54
- * autotools (autoconf, automake, libtool)
53
+ * C99 compiler
55
54
  * make
56
- * Ruby 3.3.0-preview1 or later
55
+ * Ruby 2.7.0 or later
57
56
 
58
57
  Once you have these dependencies, run:
59
58
 
@@ -79,8 +78,9 @@ to see the syntax tree for the expression `1 + 2`.
79
78
 
80
79
  See the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information. We additionally have documentation about the overall design of the project as well as various subtopics.
81
80
 
82
- * [Building](docs/building.md)
81
+ * [Build system](docs/build_system.md)
83
82
  * [Configuration](docs/configuration.md)
83
+ * [CRuby compilation](docs/cruby_compilation.md)
84
84
  * [Design](docs/design.md)
85
85
  * [Encoding](docs/encoding.md)
86
86
  * [Fuzzing](docs/fuzzing.md)
@@ -88,8 +88,11 @@ See the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information. We additio
88
88
  * [JavaScript](docs/javascript.md)
89
89
  * [Local variable depth](docs/local_variable_depth.md)
90
90
  * [Mapping](docs/mapping.md)
91
+ * [Parser translation](docs/parser_translation.md)
92
+ * [Parsing rules](docs/parsing_rules.md)
91
93
  * [Releasing](docs/releasing.md)
92
94
  * [Ripper](docs/ripper.md)
93
95
  * [Ruby API](docs/ruby_api.md)
96
+ * [RubyParser translation](docs/ruby_parser_translation.md)
94
97
  * [Serialization](docs/serialization.md)
95
98
  * [Testing](docs/testing.md)