prosemirror 0.0.1 → 0.2.0

data/TYPES.md

@@ -0,0 +1,201 @@
+# Node and mark types
+
+Each node and mark of a document has a type, a Ruby class. A type declares its name, its attributes, what it can contain, how it renders to HTML, and how to recognize it when importing HTML. The options follow ProseMirror's [NodeSpec](https://prosemirror.net/docs/ref/#model.NodeSpec) and [MarkSpec](https://prosemirror.net/docs/ref/#model.MarkSpec).
+
+```ruby
+class Callout < Prosemirror::NodeType
+  group :block
+  content "paragraph+"
+
+  prop :tone, _String?
+
+  parse_html(tag: "aside.callout") { |element| {tone: element["data-tone"]} }
+
+  def view_template(&) = aside(class: "callout", data_tone: @tone, &)
+end
+
+Prosemirror.schema = Prosemirror::Schema.tiptap.with(Callout)
+```
+
+A type is a [Phlex](https://www.phlex.fun) component, and its attributes are [Literal](https://literal.fun) properties. `Schema#with` adds a type to a schema, or replaces the type with the same name.
+
+- [Names](#names)
+- [Node type options](#node-type-options)
+- [Mark type options](#mark-type-options)
+- [Content expressions](#content-expressions)
+- [Attributes](#attributes)
+- [Rendering](#rendering)
+- [Parse rules](#parse-rules)
+- [Subclassing built-in types](#subclassing-built-in-types)
+
+## Names
+
+| Option | Meaning |
+|---|---|
+| `type_name :image_block` | The type name. Defaults to the class name in snake_case (`ImageBlock` → `image_block`). |
+| `aliases :figure, :picture` | Other names accepted on input. `Bold` accepts `strong`. |
+
+Type names are snake_case Symbols in Ruby. On input, camelCase works too (`imageBlock`). The schema's `case:` option chooses how they are written in JSON.
+
+## Node type options
+
+| Option | Default | Meaning |
+|---|---|---|
+| `group :block, :list` | none | The groups of the type, used in content expressions and in `marks`. |
+| `content "inline*"` | none: the node is a leaf | The children allowed, as a [content expression](#content-expressions). |
+| `inline` | false | The node is inline, like text. Also implied by `group :inline`. |
+| `marks "bold italic"` | all, in nodes with inline content | The marks allowed inside: names or groups separated by spaces, `"_"` for all, `""` for none. |
+| `code` | false | The node holds code. Implies `whitespace :pre`. |
+| `whitespace :pre` | `:normal` | Keep spaces and newlines when importing HTML. |
+| `leaf_text "\n"` | `""` | The text of a leaf node in `to_text`. Also takes a block, called with the node. |
+| `linebreak_replacement` | false | When importing HTML that keeps whitespace, newlines become nodes of this inline type, like `hard_break`. |
+
+## Mark type options
+
+| Option | Default | Meaning |
+|---|---|---|
+| `group :font` | none | The groups of the type, used in `excludes` and in a node's `marks`. |
+| `excludes "_"` | marks of the same type | The marks this one can't be combined with: names or groups, `"_"` for all. A link excludes other links, and Tiptap's `code` excludes all marks. |
+| `spanning false` | true | Render one element per text node, instead of one for adjacent nodes with the same mark. |
+| `code` | false | The mark holds code, as ProseMirror's `code`. Not used by this gem yet. |
+
+The order of marks in the schema sorts the marks of a node, and nests their elements: the first mark is the outermost element.
+
+## Content expressions
+
+`content` says which children a node can have. It works like a regular expression, with node names or group names instead of characters:
+
+| Expression | Meaning |
+|---|---|
+| `"text*"` | Zero or more text nodes. |
+| `"inline*"` | Zero or more nodes of the `inline` group: text, hard breaks… |
+| `"block+"` | One or more nodes of the `block` group. |
+| `"paragraph block*"` | A paragraph, then zero or more blocks. |
+| `"(table_cell \| table_header)*"` | Zero or more cells or header cells. |
+| `"caption?"` | Zero or one caption. |
+| `"cell{2}"`, `"cell{1,5}"`, `"cell{2,}"` | Exactly two, one to five, two or more cells. |
+
+The built-in types use these groups: `block` for paragraphs, headings, lists, tables…, `inline` for text and hard breaks, and `list` for lists.
+
+HTML import follows content expressions, to wrap content that doesn't fit where it is, or move it up. `to_text` uses them to find the nodes that contain text. Rendering doesn't check them.
+
+## Attributes
+
+Attributes are declared with Literal's `prop`, as keyword properties, and are available as instance variables in `view_template`:
+
+```ruby
+prop :level, _Integer(1..6), default: 1, &Prosemirror::Coerce::INTEGER
+prop :src, _String(length: 1..)
+prop :alt, _String?
+```
+
+- **Type:** a Literal type. A value that doesn't match it is invalid (see [Invalid documents](README.md#invalid-documents)).
+- **Default:** used when the attribute is missing or invalid. Literal needs a frozen value or a Proc: `default: "left".freeze`, or a file with `# frozen_string_literal: true`. An attribute that accepts nil defaults to nil. An attribute without a default is required: without a valid value, the node or mark is removed.
+- **Coercion:** the block converts values before they are checked. `Prosemirror::Coerce::INTEGER` turns `"2"` into `2`.
+
+`prop` takes three more options:
+
+| Option | Meaning |
+|---|---|
+| `json: "textAlign"` | The name in JSON, when it differs from the property name. |
+| `aliases: ["order"]` | Other JSON names accepted on input. |
+| `parse_html: ->(element) { … }` | How to read the attribute from an HTML element, when importing HTML. By default, the HTML attribute with the JSON name is read. |
+
+Attributes that a type doesn't declare are kept in `unknown_attrs`. They are never rendered, and `to_h` gives them back.
+
+## Rendering
+
+A type renders with `view_template`, which yields to render its content. For a mark, the content is the marked text.
+
+```ruby
+def view_template(&) = blockquote(class: "quote", &)
+```
+
+Without `view_template`, the content is rendered without a wrapping element. These methods are also available:
+
+| Method | Returns |
+|---|---|
+| `node` | The node being rendered, a `Prosemirror::Node`. Not available in mark types. |
+| `render_content(nodes)` | Renders any list of nodes, with their marks, for example only some of the node's children. |
+| `safe_url(url)` | The URL after the schema's `resolve_url`, when its scheme is allowed, or nil. Use it for every URL from the document. |
+| `prosemirror_schema` | The schema of the document being rendered. |
+
+Phlex escapes text and attribute values, but doesn't check URLs: pass them through `safe_url`.
+
+## Parse rules
+
+`parse_html` says which elements become this type when importing HTML, like ProseMirror's [parse rules](https://prosemirror.net/docs/ref/#model.ParseRule). A type can have several rules. A type without rules is never created by HTML import.
+
+```ruby
+parse_html tag: "h1", attrs: {level: 1}
+parse_html(tag: "b") { |element| element.style["font-weight"] != "normal" && {} }
+parse_html(style: "font-weight") { |value| value.match?(/bold|[5-9]00/) && {} }
+```
+
+The block is called with the element, or with the value of the style for a style rule. It returns the attributes, `nil` for the defaults, or `false` when the rule doesn't match. A block with two parameters also gets the schema: `{ |element, schema| … }`.
+
+The attributes are built from the rule's `attrs` or block, then from the values each `prop` reads from the element. Values read from the element take precedence, as in Tiptap.
+
+| Option | Meaning |
+|---|---|
+| `tag: "img[src]"` | A CSS selector the element must match. |
+| `style: "font-style=italic"` | For mark types: a CSS property, or a property and value. |
+| `attrs: {level: 1}` | Attributes to give the node or mark. |
+| `priority: 60` | Rules with a higher priority are tried first. Defaults to 50. Rules with the same priority are tried in schema order, mark types first. |
+| `consuming: false` | Let the following rules match the element too, for example to apply several marks. |
+| `context: "list_item/"` | The parents the rule needs. See below. |
+| `ignore: true` | Skip the element and its content. |
+| `skip: true` | Skip the element, but parse its content. |
+| `close_parent: true` | Close the current node: the content that follows goes after it. |
+| `content_element: "figcaption"` | A selector, or a block called with the element, giving the descendant that holds the content. |
+| `get_content: ->(element, schema) { … }` | Returns the nodes to use as content, instead of parsing the element's children. |
+| `preserve_whitespace: true` | Keep spaces. Newlines become `hard_break` nodes where they fit, and spaces otherwise. With `:full`, newlines stay in the text. |
+| `clear_mark: true` | For style rules: remove the marks of this type, as `font-weight: normal` does for bold. Also takes a block called with each mark. |
+
+### Selectors
+
+`tag` takes a list of compound selectors, separated by commas: a type (`p`, `*`), `#id`, `.class`, attribute selectors (`[href]`, `[a=b]`, `[a^=b]`, `[a$=b]`, `[a*=b]`, `[a~=b]`, `[a|=b]`), and `:not(…)`. Combinators like `div > p` aren't supported: use `context`, or check the element in the block.
+
+### Context
+
+`context` matches the nodes the content is being added to, from the innermost one:
+
+| Context | Matches |
+|---|---|
+| `"list_item/"` | Directly in a list item. |
+| `"blockquote//"` | Anywhere inside a blockquote. |
+| `"list_item/paragraph/"` | In a paragraph directly in a list item. |
+| `"blockquote/ \| list_item/"` | One of several contexts. |
+
+Group names work too.
+
+### Elements
+
+The element passed to rules is a `Prosemirror::HTML::Element`, with lowercase names, and attribute values with their character references decoded:
+
+| Method | Returns |
+|---|---|
+| `name` | The tag name, like `"img"`. |
+| `element["src"]` | An attribute value, or nil. |
+| `key?("data-id")` | Whether the attribute is present. |
+| `classes` | The classes, as an Array. |
+| `style` | The declarations of the `style` attribute, as a Hash with lowercase properties: `{"font-weight" => "bold"}`. |
+| `text_content` | The text of the element and its descendants. |
+| `children`, `element_children`, `first_element_child` | The children: elements and texts, elements only, the first element. |
+| `query("img")` | The first descendant matching a selector, or nil. |
+| `matches?("p.lead")` | Whether the element matches a selector. |
+| `parent` | The parent element. |
+
+## Subclassing built-in types
+
+A subclass keeps the name, options, attributes and parse rules of its parent, and can change any of them. Register it with `Schema#with`, which replaces the built-in type with the same name:
+
+```ruby
+class Paragraph < Prosemirror::Nodes::Paragraph
+  def view_template(&) = p(class: "text-lg", &)
+end
+
+Prosemirror.schema = Prosemirror::Schema.tiptap.with(Paragraph)
+```
+
+A subclass that declares parse rules replaces the rules of its parent. See the [cookbook](COOKBOOK.md) for more examples.

data/lib/prosemirror/coerce.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+module Prosemirror
+  # Conversions for property values, used as Literal coercion blocks:
+  #
+  #   prop :level, _Integer(1..6), default: 1, &Coerce::INTEGER
+  module Coerce
+    # "2" becomes 2.
+    INTEGER = proc { |value| (value.is_a?(String) && value.match?(/\A\s*-?\d+\s*\z/)) ? value.to_i : value }
+
+    # "bullet_list" becomes :bullet_list.
+    SYMBOL = proc { |value| value.is_a?(String) ? value.to_sym : value }
+
+    # A frozen copy, unless the value is already frozen.
+    FROZEN = proc { |value| value.frozen? ? value : value.dup.freeze }
+  end
+end

data/lib/prosemirror/content_expression.rb

@@ -0,0 +1,94 @@
+# frozen_string_literal: true
+
+module Prosemirror
+  # A parsed content expression, such as `"paragraph block*"` or `"(table_cell | table_header){1,5}"`.
+  # See https://prosemirror.net/docs/guide/#schema.content_expressions
+  class ContentExpression
+    TOKEN = /\s*(\w+|[()|+*?]|\{\s*\d+\s*(?:,\s*\d*\s*)?\})/
+
+    attr_reader :source, :tree
+
+    def initialize(source)
+      @source = source
+      @tokens = tokenize(source)
+      @tree = parse_choice
+      raise ArgumentError, "unexpected #{@tokens.first.inspect} in content expression #{source.inspect}" if @tokens.any?
+    end
+
+    # The node names and group names the expression refers to.
+    def names
+      collect_names(@tree)
+    end
+
+    private
+
+    def tokenize(source)
+      tokens = []
+      rest = source.strip
+      until rest.empty?
+        match = TOKEN.match(rest)
+        raise ArgumentError, "invalid content expression #{source.inspect}" unless match&.begin(0)&.zero?
+
+        tokens << match[1]
+        rest = match.post_match.strip
+      end
+      tokens
+    end
+
+    def parse_choice
+      branches = [parse_sequence]
+      while @tokens.first == "|"
+        @tokens.shift
+        branches << parse_sequence
+      end
+      (branches.size == 1) ? branches.first : [:choice, branches]
+    end
+
+    def parse_sequence
+      terms = []
+      terms << parse_repeat while @tokens.any? && !["|", ")"].include?(@tokens.first)
+      raise ArgumentError, "empty content expression in #{@source.inspect}" if terms.empty?
+
+      (terms.size == 1) ? terms.first : [:sequence, terms]
+    end
+
+    def parse_repeat
+      atom = parse_atom
+      case @tokens.first
+      when "*" then @tokens.shift && [:repeat, atom, 0, nil]
+      when "+" then @tokens.shift && [:repeat, atom, 1, nil]
+      when "?" then @tokens.shift && [:repeat, atom, 0, 1]
+      when /\A\{/ then [:repeat, atom, *parse_range(@tokens.shift)]
+      else atom
+      end
+    end
+
+    def parse_range(token)
+      min, comma, max = token.delete("{} ").partition(",")
+      return [min.to_i, min.to_i] if comma.empty?
+
+      [min.to_i, max.empty? ? nil : max.to_i]
+    end
+
+    def parse_atom
+      token = @tokens.shift
+      case token
+      when "("
+        expression = parse_choice
+        raise ArgumentError, "missing ) in content expression #{@source.inspect}" unless @tokens.shift == ")"
+
+        expression
+      when /\A\w+\z/ then [:name, token.to_sym]
+      else raise ArgumentError, "unexpected #{token.inspect} in content expression #{@source.inspect}"
+      end
+    end
+
+    def collect_names(tree)
+      case tree.first
+      when :name then [tree[1]]
+      when :repeat then collect_names(tree[1])
+      else tree[1].flat_map { |child| collect_names(child) }.uniq
+      end
+    end
+  end
+end

data/lib/prosemirror/content_match.rb

@@ -0,0 +1,241 @@
+# frozen_string_literal: true
+
+module Prosemirror
+  # A state of a node type's content expression, compiled into a deterministic automaton, as in
+  # ProseMirror's ContentMatch: https://prosemirror.net/docs/ref/#model.ContentMatch
+  #
+  # Edges go from one state to the next for a node type. Their order matters: the first generatable
+  # type is used to fill required content.
+  class ContentMatch
+    attr_reader :edges
+
+    def initialize(valid_end)
+      @valid_end = valid_end
+      @edges = []
+      @wrappings = {}
+    end
+
+    # Compiles a content expression for a schema. Returns EMPTY for leaf types.
+    def self.compile(expression, schema)
+      return EMPTY if expression.nil?
+
+      Compiler.new(expression, schema).call
+    end
+
+    # Whether the content can end here.
+    def valid_end?
+      @valid_end
+    end
+
+    # The state after a node of this type, or nil when it can't come here.
+    def match_type(type)
+      @edges.find { |edge_type, _| edge_type == type }&.last
+    end
+
+    # The state after these node types, or nil when they don't match.
+    def match_types(types)
+      types.reduce(self) { |match, type| match&.match_type(type) }
+    end
+
+    def inline_content?
+      !@edges.empty? && @edges.first.first.inline?
+    end
+
+    # The first type that can come here and can be created empty, or nil.
+    def default_type(schema)
+      @edges.map(&:first).find { |type| generatable?(type, schema) }
+    end
+
+    # The node types to insert before `types` so that they match, possibly none, or nil when no
+    # insertion works. With `to_end`, the content must also be able to end after them.
+    def fill_before(types, schema, to_end: false)
+      seen = [self]
+      search = lambda do |match, filled|
+        finished = match.match_types(types)
+        return filled if finished && (!to_end || finished.valid_end?)
+
+        match.edges.each do |type, following|
+          next if !generatable?(type, schema) || seen.include?(following)
+
+          seen << following
+          found = search.call(following, filled + [type])
+          return found if found
+        end
+        nil
+      end
+      search.call(self, [])
+    end
+
+    # The wrapper types that let a node of this type come here, possibly none, or nil when none do.
+    def find_wrapping(target, schema)
+      return @wrappings[target] if @wrappings.key?(target)
+
+      @wrappings[target] = compute_wrapping(target, schema)
+    end
+
+    def generatable?(type, schema)
+      type.type_name != :text && !type.required_attrs?
+    end
+
+    private
+
+    def compute_wrapping(target, schema)
+      seen = {}
+      active = [[self, nil, nil]]
+      until active.empty?
+        current = active.shift
+        match, type, = current
+        return wrapping_path(current) if match.match_type(target)
+
+        match.edges.each do |next_type, following|
+          next if next_type.leaf? || next_type.required_attrs? || seen[next_type]
+          next if type && !following.valid_end?
+
+          active << [schema.content_match(next_type), next_type, current]
+          seen[next_type] = true
+        end
+      end
+      nil
+    end
+
+    def wrapping_path(step)
+      path = []
+      while step && step[1]
+        path.unshift(step[1])
+        step = step[2]
+      end
+      path
+    end
+
+    EMPTY = new(true)
+
+    # Compiles an expression tree into a non-deterministic automaton, then a deterministic one.
+    class Compiler
+      def initialize(expression, schema)
+        @expression = expression
+        @schema = schema
+        @nfa = [[]]
+      end
+
+      def call
+        connect(compile(@expression.tree, 0), new_state)
+        explore(null_from(0), {})
+      end
+
+      private
+
+      Edge = Struct.new(:term, :to)
+
+      def new_state
+        @nfa << []
+        @nfa.size - 1
+      end
+
+      def edge(from, to = nil, term = nil)
+        Edge.new(term, to).tap { |created| @nfa[from] << created }
+      end
+
+      def connect(edges, to)
+        edges.each { |created| created.to = to }
+      end
+
+      def compile(tree, from)
+        case tree.first
+        when :choice then tree[1].flat_map { |branch| compile(branch, from) }
+        when :sequence then compile_sequence(tree[1], from)
+        when :repeat then compile_repeat(*tree.drop(1), from)
+        when :name then resolve(tree[1]).map { |type| edge(from, nil, type) }
+        end
+      end
+
+      def compile_sequence(parts, from)
+        parts.each_with_index do |part, index|
+          edges = compile(part, from)
+          return edges if index == parts.size - 1
+
+          connect(edges, from = new_state)
+        end
+      end
+
+      def compile_repeat(tree, min, max, from)
+        if min.zero? && max.nil? # *
+          loop_state = new_state
+          edge(from, loop_state)
+          connect(compile(tree, loop_state), loop_state)
+          [edge(loop_state)]
+        elsif min == 1 && max.nil? # +
+          loop_state = new_state
+          connect(compile(tree, from), loop_state)
+          connect(compile(tree, loop_state), loop_state)
+          [edge(loop_state)]
+        elsif min.zero? && max == 1 # ?
+          [edge(from)] + compile(tree, from)
+        else
+          compile_range(tree, min, max, from)
+        end
+      end
+
+      def compile_range(tree, min, max, from)
+        current = from
+        min.times do
+          following = new_state
+          connect(compile(tree, current), following)
+          current = following
+        end
+        if max.nil?
+          connect(compile(tree, current), current)
+        else
+          (max - min).times do
+            following = new_state
+            edge(current, following)
+            connect(compile(tree, current), following)
+            current = following
+          end
+        end
+        [edge(current)]
+      end
+
+      def resolve(name)
+        types = @schema.node_types_named(name)
+        raise ArgumentError, "no node type or group #{name} in content expression #{@expression.source.inspect}" if types.empty?
+
+        types
+      end
+
+      # The states reachable from `state` without consuming a node, sorted.
+      def null_from(state)
+        result = []
+        scan = lambda do |current|
+          edges = @nfa[current]
+          return scan.call(edges.first.to) if edges.size == 1 && edges.first.term.nil?
+
+          result << current
+          edges.each { |created| scan.call(created.to) if created.term.nil? && !result.include?(created.to) }
+        end
+        scan.call(state)
+        result.sort.reverse
+      end
+
+      def explore(states, labeled)
+        out = []
+        states.each do |state|
+          @nfa[state].each do |created|
+            next unless created.term
+
+            set = out.find { |type, _| type == created.term }&.last
+            null_from(created.to).each do |reached|
+              out << [created.term, set = []] unless set
+              set << reached unless set.include?(reached)
+            end
+          end
+        end
+        match = labeled[states.join(",")] = ContentMatch.new(states.include?(@nfa.size - 1))
+        out.each do |type, set|
+          set = set.sort.reverse
+          match.edges << [type, labeled[set.join(",")] || explore(set, labeled)]
+        end
+        match
+      end
+    end
+  end
+end

data/lib/prosemirror/document.rb

@@ -0,0 +1,76 @@
+# frozen_string_literal: true
+
+module Prosemirror
+  # A document and its schema. Enumerating a document yields all its nodes, depth first.
+  #
+  #   doc = Prosemirror.parse(json)
+  #   doc.to_html
+  #   doc.select { |node| node in { type: :heading, attrs: { level: 1 } } }
+  class Document
+    include Enumerable
+
+    attr_reader :root, :schema
+
+    def initialize(root, schema: Prosemirror.schema)
+      @root = root
+      @schema = schema
+    end
+
+    def content
+      root.content
+    end
+
+    def each(&)
+      root.descendants(&)
+    end
+
+    # The document as HTML. The string is `html_safe` when ActiveSupport is loaded.
+    def to_html
+      html = to_phlex.call
+      html.respond_to?(:html_safe) ? html.html_safe : html
+    end
+
+    # A Phlex component rendering the document, to render it inside another Phlex component:
+    #
+    #   render doc.to_phlex
+    def to_phlex
+      HTMLRenderer.new(self)
+    end
+
+    # Lets Rails render the document in ERB templates and ViewComponents: `<%= render doc %>`.
+    def render_in(_view_context, &)
+      to_html
+    end
+
+    # The document as plain text, with blocks separated by `block_separator`.
+    def to_text(block_separator: "\n\n")
+      TextSerializer.new(schema, block_separator: block_separator).call(root)
+    end
+
+    # The document as JSON, as a Hash.
+    def to_h
+      schema.serialize(root)
+    end
+
+    def as_json(*)
+      to_h
+    end
+
+    def to_json(*)
+      to_h.to_json(*)
+    end
+
+    def text_content
+      root.text_content
+    end
+
+    def ==(other)
+      other.is_a?(Document) && root == other.root
+    end
+    alias_method :eql?, :==
+
+    def hash
+      root.hash
+    end
+  end
+end