oxc 0.1.0-arm-linux-gnu → 0.149.0-arm-linux-gnu

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7003799f248bc17853e84b4573f6666e47ff9a26e705617e2ea0dce66f0e372f
-  data.tar.gz: 89a4facbe706ded06469348779f2996e9edfe0cb6bf63121ede39524b169307a
+  metadata.gz: bbcb94b81fe82d086d425ceaca0c7fc0c54ba0177b22537be764fec951410bb0
+  data.tar.gz: 8ff558d7b1d18b91324de7acfdd45c6907ee61cb0b1e00a424fab743394b8e60
 SHA512:
-  metadata.gz: cd218b1db0f4bb52bc213a9a21751e460f825615ad30e1a1b245ffd2f0fa31eb020cfc4216e162f5bc07c733f22a3507342fb9a96f8be1cea67c072c6114cfd4
-  data.tar.gz: f83c40bb390e93baf83d5169528bd460b94f58aca9ec94be42f4f5c03cc30d6647fc0793e780973572f5a8f7b1b0adfcd0418ececda7044e4c2f339de0c4d42d
+  metadata.gz: fe0b453313f7a14b60fa4697ac1503ef3c1ea6ec15d12cd92e5a9ee2a5f4554ccf224d52c90a81839c14e98cd0fd96ff12f38c1f840670ff55aca578fe58eec2
+  data.tar.gz: 2378b04f48856ec7e25e6ebc09c906a73e9a57ed5a5fa1095e79a0228737fe756a328d628bd14d72737d34b7040251fe1166cb5cecea1667bd3c21064b67d821

data/README.md

@@ -27,6 +27,14 @@ bundle add oxc
 
 Anywhere a precompiled gem is not published, the gem builds from source and needs the [Rust toolchain](https://rustup.rs) at 1.96 or newer.
 
+### Versioning
+
+The gem carries the version of oxc it packages. Gem v0.149.0 bundles oxc v0.149.0, and `Oxc.oxc_version` reports what a given build was compiled against.
+
+So the version tracks oxc, not the Ruby API here. A minor bump is a minor oxc release, and says nothing about whether this gem's own API moved. Pin on the oxc version you want.
+
+If the gem needs releasing again for the same oxc version, that release adds a fourth digit, as in v0.149.0.1.
+
 ### Usage
 
 #### Minifying
@@ -205,6 +213,171 @@ Oxc.parse("let a; let a;", semantic_errors: true).errors.map(&:message)
 #=> ["Identifier `a` has already been declared"]
 ```
 
+#### Walking the AST
+
+`root` answers the program as an `Oxc::Node`, which walks, reads its fields by name, and knows what it sits inside.
+
+```ruby
+root = Oxc.parse(source).root
+
+root.type                 #=> "Program"
+root.keys                 #=> the ESTree fields this node carries
+root.fields               #=> those fields and their values, without the span
+root.child_nodes          #=> the nodes directly under it
+root.every("Identifier")  #=> every identifier in the file
+root.at(offset)           #=> the innermost node covering a byte offset
+root.each                 #=> an Enumerator over every node
+```
+
+Inspecting a node shows every field it carries, so there is always something to reach for next.
+
+```
+#<Oxc::Node VariableDeclaration range=[0, 13] kind="let" declarations=[... 1 item]>
+#<Oxc::Node VariableDeclarator range=[4, 13] id=#<Oxc::Node Identifier> init=#<Oxc::Node Literal>>
+#<Oxc::Node Identifier range=[4, 9] name="count">
+```
+
+Every field is there, so what `inspect` prints and what `keys` answers never disagree. A field holding a node prints as that node's type, one holding a list prints how many it holds, and one holding nothing prints the `nil`, the `false` or the `[]` it holds.
+
+An ESTree field always wins over a method of the gem's own, since `name`, `attributes` and `children` are all real fields. `Identifier#name` is the identifier's name, `JSXElement#children` is what the element wraps, and `ImportDeclaration#attributes` is the import's `with` clause. The walker spells its own versions `underscored_type`, `to_h` and `child_nodes`, which no ESTree field can be called.
+
+A field comes back as a node when it holds one, so reads chain.
+
+```ruby
+declaration = root.child_nodes.first
+
+declaration.kind
+#=> "let"
+
+declaration.declarations.first.id.name
+#=> "count"
+```
+
+ESTree names its fields in camelCase, and a field answers to its snake_case name too, so reading an AST does not mean writing JavaScript casing in Ruby.
+
+```ruby
+node.type_annotation   # the same field as node.typeAnnotation
+node.super_class       # superClass
+root.source_type       # sourceType
+```
+
+Patterns take either name as well, binding what you asked for.
+
+```ruby
+node => { type_annotation: { type: }, readonly: }
+```
+
+`ancestors` is what a rewrite needs, since a reference sits inside the expression that has to be replaced.
+
+```ruby
+reference = root.at(source.index("count +="))
+reference.ancestors.find { |node| node.type == "AssignmentExpression" }.slice
+#=> "count += 1"
+```
+
+A parse result keeps the source it read and hands it down to every node it builds, so `slice` answers with no argument at all.
+
+```ruby
+parsed = Oxc.parse(source)
+
+parsed.source
+#=> "let count = 0\nfunction bump() { count += 1; render(count) }"
+
+parsed.root.every("FunctionDeclaration").first.slice
+#=> "function bump() { count += 1; render(count) }"
+```
+
+It still takes one, for a node assembled by hand or read against a different string.
+
+```ruby
+node.slice(other_source)
+```
+
+Nodes pattern match, and nest, since a field holding a node comes back as one.
+
+```ruby
+node => { type: "VariableDeclarator", id: { name: }, init: { value: } }
+name   #=> "count"
+value  #=> 0
+
+root.select { |node| node in { type: "FunctionDeclaration", id: { name: /^handle/ } } }
+```
+
+`deconstruct_keys` is the whole protocol here. There is no `deconstruct`, since `each` yields every descendant and an array pattern over direct children would disagree with `to_a`.
+
+`to_h` and `to_json` answer the ESTree the node wraps, which is what a snapshot test or a dump to another tool wants.
+
+```ruby
+node.to_h
+#=> { "type" => "Identifier", "name" => "count", "start" => 4, "end" => 9 }
+
+node.to_json
+#=> "{\"type\":\"Identifier\",\"name\":\"count\",\"start\":4,\"end\":9}"
+```
+
+`Oxc::Visitor` answers a node with the method named after its type, and walks through anything nothing answers.
+
+```ruby
+class Reads < Oxc::Visitor
+  def visit_assignment_expression(node)
+    puts "#{node.left["name"]} #{node.operator}"
+
+    visit_children(node)
+  end
+end
+
+Reads.new.visit(Oxc.parse(source))
+```
+
+It takes a parse result or a node, so the common case needs no `root`. A result with no AST is nothing to walk and visits nothing.
+
+There is one node class, not one per type, so a type the gem has never seen still walks and still answers. The types and their fields are [ESTree](https://github.com/estree/estree). For the TypeScript and JSX nodes, which ESTree does not cover, oxc publishes the exact shapes it emits as [`@oxc-project/types`](https://www.npmjs.com/package/@oxc-project/types).
+
+#### Rewriting
+
+`Oxc::MutationVisitor` records what to do to a node and splices the original text at the end, so everything it did not touch survives byte for byte, comments and indentation included.
+
+```ruby
+class Renamer < Oxc::MutationVisitor
+  def visit_identifier(node)
+    replace(node, "renamed") if node["name"] == "count"
+  end
+end
+
+Renamer.new.rewrite("let count = 1 // keep me")
+#=> "let renamed = 1 // keep me"
+```
+
+`replace`, `remove`, `insert_before`, `insert_after` and `wrap` are the operations, and each takes a node. Spans are exact, so removing `debugger;` removes what the node covered and leaves the newline after it alone.
+
+Walking into a node that was replaced would edit text that is no longer there, so it stops. Two edits over the same span raise `Oxc::MutationVisitor::Overlap` instead of quietly producing something broken.
+
+What goes in is text, so a node can become anything, including several statements or nothing at all. Nothing checks it on the way, so the result is read back afterwards and refused if it stopped being JavaScript.
+
+```ruby
+Breaker.new.rewrite("foo(data)")
+#=> Oxc::MutationVisitor::Invalid: what was rewritten no longer reads as JavaScript: Unexpected token
+```
+
+Pass `verify: false` for a fragment that was never going to parse on its own.
+
+`parsed` reaches what the source parsed to, so a rewrite can ask for symbols and drive from them. That is the difference between rewriting a name and rewriting the right one.
+
+```ruby
+class ToState < Oxc::MutationVisitor
+  def rewrite(source) = super(source, symbols: true)
+
+  def visit_identifier(node)
+    reference = parsed.symbols.fetch("declared").flat_map { |symbol| symbol["references"] }
+      .find { |found| found["start"] == node.start }
+
+    replace(node, %(state.get("#{node["name"]}"))) if reference && !reference["write"]
+  end
+end
+```
+
+Drive from references, not from every `Identifier`. That is what keeps a declaration, a shadowed local, and a same-named property out of the rewrite.
+
 #### What a file declared, and what it only used
 
 `symbols: true` answers every binding with the span it was declared at and the spans of every reference to it, plus the names the file used without declaring.

data/ext/oxc/extconf.rb

@@ -65,12 +65,12 @@ if target_platform
 
   system("rustup target add #{target_platform}") || warn("oxc: Failed to add Rust target #{target_platform}")
 
-  cargo_args = "--release --target #{target_platform}"
+  cargo_args = "--release --locked --target #{target_platform}"
   lib_dir = File.join(target_dir, target_platform, "release")
 else
   puts "oxc: Compiling Rust library for native platform..."
 
-  cargo_args = "--release"
+  cargo_args = "--release --locked"
   lib_dir = File.join(target_dir, "release")
 end
 

data/lib/oxc/diagnostic.rb

@@ -80,7 +80,7 @@ module Oxc
 
     #: () -> String
     def inspect
-      "#<#{self.class.name} #{start}..#{finish}#{" #{message.inspect}" if message}>"
+      "#<#{self.class.name} range=[#{start}, #{finish}]#{" #{message.inspect}" if message}>"
     end
   end
 end

data/lib/oxc/mutation_visitor.rb

@@ -0,0 +1,128 @@
+# frozen_string_literal: true
+
+module Oxc
+  # A visitor that rewrites the source it walked, by recording what to do to a node and splicing the
+  # original text at the end. Everything nothing touched survives byte for byte.
+  #
+  #     class Renamer < Oxc::MutationVisitor
+  #       def visit_identifier(node)
+  #         replace(node, "renamed") if node["name"] == "count"
+  #       end
+  #     end
+  #
+  #     Renamer.new.rewrite("let count = 1")  #=> "let renamed = 1"
+  #
+  class MutationVisitor < Visitor
+    class Overlap < StandardError; end
+    class Invalid < Error; end
+
+    Edit = Data.define(
+      :start, #: Integer
+      :finish, #: Integer
+      :text, #: String
+      :order #: Integer
+    )
+
+    attr_reader :source #: String
+    attr_reader :parsed #: Oxc::ParseResult
+
+    #: (String, ?verify: bool, **untyped) -> String
+    def rewrite(source, verify: true, **options)
+      @source = source
+      @edits = [] #: Array[Edit]
+      @replaced = [] #: Array[[Integer, Integer]]
+      @parsed = Oxc.parse(source, **options).validate!
+
+      visit(parsed)
+
+      rewritten = apply
+
+      verify ? verified(rewritten, options) : rewritten
+    end
+
+    #: (Oxc::Node, String) -> void
+    def replace(node, text)
+      @replaced << [node.start, node.finish]
+
+      edit(node.start, node.finish, text)
+    end
+
+    #: (Oxc::Node) -> void
+    def remove(node)
+      replace(node, "")
+    end
+
+    #: (Oxc::Node, String) -> void
+    def insert_before(node, text)
+      edit(node.start, node.start, text)
+    end
+
+    #: (Oxc::Node, String) -> void
+    def insert_after(node, text)
+      edit(node.finish, node.finish, text)
+    end
+
+    #: (Oxc::Node, String, String) -> void
+    def wrap(node, before, after)
+      insert_before(node, before)
+      insert_after(node, after)
+    end
+
+    #: (Oxc::Node) -> void
+    def visit_children(node)
+      return nil if replaced?(node)
+
+      super
+    end
+
+    private
+
+    #: (String, Hash[Symbol, untyped]) -> String
+    def verified(rewritten, options)
+      answer = Oxc.parse(rewritten, **options, ast: false)
+
+      return rewritten unless answer.errors?
+
+      raise Invalid, "what was rewritten no longer reads as JavaScript: #{answer.errors.first&.message}"
+    end
+
+    #: (Oxc::Node) -> bool
+    def replaced?(node)
+      @replaced.any? { |start, finish| node.start >= start && node.finish <= finish }
+    end
+
+    #: (Integer, Integer, String) -> void
+    def edit(start, finish, text)
+      @edits.each do |existing|
+        next unless overlaps?(existing, start, finish)
+
+        raise Overlap, "an edit at #{start}..#{finish} overlaps one at #{existing.start}..#{existing.finish}"
+      end
+
+      @edits << Edit.new(start: start, finish: finish, text: text, order: @edits.length)
+
+      nil
+    end
+
+    #: (Edit, Integer, Integer) -> bool
+    def overlaps?(existing, start, finish)
+      return false if existing.start == existing.finish && start == finish
+
+      start < existing.finish && existing.start < finish
+    end
+
+    #: () -> String
+    def apply
+      taken = 0
+      result = +""
+
+      @edits.sort_by { |edit| [edit.start, edit.order] }.each do |edit|
+        result << source.byteslice(taken, edit.start - taken).to_s << edit.text
+
+        taken = edit.finish
+      end
+
+      result << source.byteslice(taken..).to_s
+    end
+  end
+end

data/lib/oxc/node.rb

@@ -0,0 +1,186 @@
+# frozen_string_literal: true
+
+module Oxc
+  class Node
+    include Enumerable #[Oxc::Node]
+
+    ACRONYM = /([A-Z]+)([A-Z][a-z])/ #: Regexp
+    BOUNDARY = /([a-z\d])([A-Z])/ #: Regexp
+    SPAN = ["type", "start", "end"].freeze #: Array[String]
+
+    attr_reader :parent #: Oxc::Node?
+
+    protected
+
+    attr_reader :attributes #: Hash[String, untyped]
+    attr_reader :text #: String?
+
+    public
+
+    #: (Hash[String, untyped], ?Oxc::Node?, ?String?) -> void
+    def initialize(attributes, parent = nil, text = nil)
+      @attributes = attributes
+      @parent = parent
+      @text = text || parent&.text
+    end
+
+    #: () -> String
+    def type
+      attributes.fetch("type")
+    end
+
+    #: () -> Integer
+    def start
+      attributes.fetch("start")
+    end
+
+    #: () -> Integer
+    def finish
+      attributes.fetch("end")
+    end
+
+    #: () -> String
+    def underscored_type
+      @underscored_type ||= type.gsub(ACRONYM, '\1_\2').gsub(BOUNDARY, '\1_\2').downcase
+    end
+
+    #: (String) -> untyped
+    def [](key)
+      attributes[key]
+    end
+
+    #: (?String?) -> String?
+    def slice(from = text)
+      raise ArgumentError, "this node does not know its source, so #slice needs it" unless from
+
+      from.byteslice(start, finish - start)
+    end
+
+    #: () -> Array[String]
+    def keys
+      attributes.keys
+    end
+
+    #: () -> Hash[String, untyped]
+    def to_h
+      attributes
+    end
+
+    #: (?untyped) -> String
+    def to_json(state = nil)
+      state ? attributes.to_json(state) : attributes.to_json
+    end
+
+    #: (Array[Symbol]?) -> Hash[Symbol, untyped]
+    def deconstruct_keys(keys)
+      found = {} #: Hash[Symbol, untyped]
+
+      if keys
+        keys.each do |name|
+          key = field_for(name.to_s)
+
+          found[name] = wrap(attributes[key]) if key
+        end
+      else
+        attributes.each_key { |key| found[key.to_sym] = wrap(attributes[key]) }
+      end
+
+      found
+    end
+
+    #: () -> Array[Oxc::Node]
+    def child_nodes
+      @child_nodes ||= attributes.each_value
+                                 .flat_map { |value| value.is_a?(Array) ? value : [value] }
+                                 .select { |value| value.is_a?(Hash) && value.key?("type") }
+                                 .map { |value| Node.new(value, self) }
+                                 .freeze
+    end
+
+    #: () { (Oxc::Node) -> void } -> void
+    #: () -> Enumerator[Oxc::Node, void]
+    def each(&)
+      return enum_for(:each) unless block_given?
+
+      yield self
+
+      child_nodes.each { |child| child.each(&) }
+    end
+
+    #: () -> Array[Oxc::Node]
+    def ancestors
+      parent ? [parent, *parent.ancestors] : []
+    end
+
+    #: (Integer) -> Oxc::Node?
+    def at(offset)
+      covering = each.select { |node| offset >= node.start && offset < node.finish }
+
+      covering.min_by { |node| node.finish - node.start }
+    end
+
+    #: (String) -> Array[Oxc::Node]
+    def every(type)
+      each.select { |node| node.type == type }
+    end
+
+    #: () -> Hash[String, untyped]
+    def fields
+      attributes.except(*SPAN)
+    end
+
+    #: () -> String
+    def inspect
+      described = fields.map { |key, value| described_field(key, value) }
+
+      "#<#{self.class.name} #{type} range=[#{start}, #{finish}]#{" #{described.join(" ")}" unless described.empty?}>"
+    end
+
+    #: (Symbol, *untyped) -> untyped
+    def method_missing(name, *arguments)
+      key = field_for(name.to_s)
+
+      return super unless key
+
+      wrap(attributes[key])
+    end
+
+    #: (Symbol, ?bool) -> bool
+    def respond_to_missing?(name, include_private = false)
+      !field_for(name.to_s).nil? || super
+    end
+
+    private
+
+    #: (String, untyped) -> String
+    def described_field(key, value)
+      case value
+      when Array then "#{key}=#{if value.empty?
+                                  "[]"
+                                else
+                                  "[... #{value.length} #{value.length == 1 ? "item" : "items"}]"
+                                end}"
+      when Hash then "#{key}=#<#{self.class.name} #{value["type"]}>"
+      else "#{key}=#{value.inspect}"
+      end
+    end
+
+    #: (String) -> String?
+    def field_for(name)
+      return name if attributes.key?(name)
+
+      camelized = name.gsub(/_([a-z\d])/) { Regexp.last_match(1).to_s.upcase }
+
+      camelized if attributes.key?(camelized)
+    end
+
+    #: (untyped) -> untyped
+    def wrap(value)
+      case value
+      when Hash then value.key?("type") ? Node.new(value, self, text) : value
+      when Array then value.map { |item| wrap(item) }
+      else value
+      end
+    end
+  end
+end

data/lib/oxc/options.rb

@@ -72,9 +72,9 @@ module Oxc
       freeze
     end
 
-    #: (?untyped) -> String
+    #: (?JSON::options?) -> String
     def to_json(state = nil)
-      JSON.generate(to_h, state)
+      state ? JSON.generate(to_h, state) : JSON.generate(to_h)
     end
 
     #: () -> String

data/lib/oxc/parse_result.rb

@@ -4,17 +4,19 @@ module Oxc
   class ParseResult
     include Diagnosed
 
+    attr_reader :source #: String?
     attr_reader :program #: Hash[String, untyped]?
     attr_reader :module_record #: Hash[String, untyped]?
     attr_reader :symbols #: Hash[String, untyped]?
     attr_reader :comments #: Array[Oxc::Comment]
     attr_reader :diagnostics #: Array[Oxc::Diagnostic]
 
-    #: (String) -> Oxc::ParseResult
-    def self.from_json(payload)
+    #: (String, ?String?) -> Oxc::ParseResult
+    def self.from_json(payload, source = nil)
       parsed = JSON.parse(payload)
 
       new(
+        source: source,
         program: parsed["program"],
         module_record: parsed["module_record"],
         symbols: parsed["symbols"],
@@ -24,8 +26,9 @@ module Oxc
       )
     end
 
-    #: (comments: Array[Oxc::Comment], diagnostics: Array[Oxc::Diagnostic], ?program: Hash[String, untyped]?, ?module_record: Hash[String, untyped]?, ?symbols: Hash[String, untyped]?, ?panicked: bool) -> void
-    def initialize(comments:, diagnostics:, program: nil, module_record: nil, symbols: nil, panicked: false)
+    #: (comments: Array[Oxc::Comment], diagnostics: Array[Oxc::Diagnostic], ?source: String?, ?program: Hash[String, untyped]?, ?module_record: Hash[String, untyped]?, ?symbols: Hash[String, untyped]?, ?panicked: bool) -> void
+    def initialize(comments:, diagnostics:, source: nil, program: nil, module_record: nil, symbols: nil, panicked: false)
+      @source = source
       @program = program.freeze
       @module_record = module_record.freeze
       @symbols = symbols.freeze
@@ -36,6 +39,11 @@ module Oxc
       freeze
     end
 
+    #: () -> Oxc::Node?
+    def root
+      program ? Node.new(program, nil, source) : nil
+    end
+
     #: () -> Oxc::ParseResult
     def validate!
       return self unless errors? || panicked?
@@ -46,7 +54,9 @@ module Oxc
     #: () -> String
     def inspect
       parts = [] #: Array[String]
-      parts << "program=#{program["type"]}" if program
+      parts << "#{root&.type} range=[#{root&.start}, #{root&.finish}]" if program
+      parts << "module_record" if module_record
+      parts << "symbols=#{symbols.fetch("declared").length}" if symbols
       parts << "comments=#{comments.length}" unless comments.empty?
       parts << "diagnostics=#{diagnostics.length}" unless diagnostics.empty?
 

data/lib/oxc/result.rb

@@ -18,7 +18,7 @@ module Oxc
       @panicked = panicked
     end
 
-    #: (?strict: untyped) -> self
+    #: (?strict: bool?) -> self
     def validate!(strict: false)
       return self unless errors? || panicked?
       return self unless strict || panicked? || code.empty?

data/lib/oxc/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Oxc
-  VERSION = "0.1.0"
+  VERSION = "0.149.0"
 end

data/lib/oxc/visitor.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+module Oxc
+  class Visitor
+    #: (Oxc::Node | Oxc::ParseResult) -> void
+    def visit(node)
+      node = node.root if node.is_a?(ParseResult)
+
+      return nil unless node
+
+      answer = "visit_#{node.underscored_type}"
+
+      respond_to?(answer) ? public_send(answer, node) : visit_children(node)
+
+      nil
+    end
+
+    #: (Oxc::Node) -> void
+    def visit_children(node)
+      node.child_nodes.each { |child| visit(child) }
+
+      nil
+    end
+  end
+end