prosemirror 0.2.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1006b6a2ecb95f5a4d603d5d9321bbdab419f26f882365ff14d1692f27970ce7
-  data.tar.gz: 22734518a2e81ecaba0ff64ed5057f40705dfb55dcd919567673f88e7cbf7e45
+  metadata.gz: c84fca02d8a8c1f32c9162da3a22133f2ef774c035281331d7d92423d24a242f
+  data.tar.gz: ca2ec139725747838a359f43a667c1fc4b5b5249b7dba12775ffbe39985f0c38
 SHA512:
-  metadata.gz: 0f4b24b76e78e49523f068aa8520304c955428d4ff295c0e0f918dfad07ad0bc5b7ee391b8dacaeabf258f871d99f2dcfba15e4ebad4bec961e97529cdd24f34
-  data.tar.gz: be218156ef3b417342de7d393d219b5019fe6f270a9c865e5cce9b5f9a2436008f59d5ada72d0111e09152586a06e21a7e98653ce4002923049eb0e70331daa0
+  metadata.gz: c018119a21927872e1382b1ee05f33c397781c87520b55ee4ca76f06abc2fb728ed4122043f11250cd3f8195c5ede8de4e50619bf98f93405f163fa30bd413aa
+  data.tar.gz: 3a870e5aceacd20c1ec717942f3dc4c75a97f811e8813e73c346aa71b919ab4430645593c6daec6132a1c72cd3f053e192ff812126cf22cd8f567cc5cafb8b98

data/CHANGELOG.md

@@ -1,5 +1,17 @@
 ## [Unreleased]
 
+## [0.4.0] - 2026-10-24
+
+- `Document#transform` builds a new document from a block called with each node, children first, which keeps, replaces, removes or splices nodes. Required content that goes missing is added back and reported.
+- `Document#check` raises `InvalidDocument` for the first problem with the schema, with its path, and `Document#valid?` returns true or false.
+- `Node#blank?` is true for nodes that show nothing, like empty paragraphs.
+
+## [0.3.0] - 2026-10-24
+
+- Import Markdown with `Prosemirror.from_markdown` (or `schema.parse_markdown`). Needs the [`commonmarker`](https://github.com/gjtorikian/commonmarker) gem, which isn't installed with this gem. GitHub Flavored Markdown tables, strikethrough and autolinks are supported, and the schema's parse rules apply, so custom types can match Markdown too. Raw HTML goes through HTML import when the herb gem is installed. YAML front matter is dropped.
+- Import plain text with `Prosemirror.from_text` (or `schema.parse_text`): blank lines separate paragraphs, and other line breaks become hard breaks.
+- HTML, Markdown and text import accept nil and binary strings, and replace invalid UTF-8 with U+FFFD, as browsers do. HTML import raised on invalid UTF-8.
+
 ## [0.2.0] - 2026-10-23
 
 - Import HTML with `Prosemirror.from_html` (or `schema.parse_html`), as ProseMirror's `DOMParser` does. Needs the [`herb`](https://herb-tools.dev) gem, which isn't installed with this gem.

data/COOKBOOK.md

@@ -18,23 +18,35 @@ class Paragraph < Prosemirror::Nodes::Paragraph
 end
 ```
 
-A paragraph that only holds spaces or line breaks is not empty. To skip those too, check the text instead, keeping paragraphs with an inline image or another leaf node:
+A paragraph that only holds spaces or line breaks is not empty. To skip those too, use `blank?`, which keeps paragraphs with an inline image or another leaf node:
 
 ```ruby
 class Paragraph < Prosemirror::Nodes::Paragraph
   def view_template(&)
-    p(&) unless blank?
+    p(&) unless node.blank?(prosemirror_schema)
   end
+end
+```
 
-  private
+`to_text` still separates empty paragraphs with a blank line, as ProseMirror does. `to_h` keeps them, so the editor gets the document back unchanged. To remove them from the document itself, see the next recipe.
 
-  def blank?
-    node.text_content.strip.empty? && node.content.all? { |child| child.text? || child.type == :hard_break }
+## Remove empty paragraphs before saving
+
+To clean a document before saving it, remove blank paragraphs with `transform`, along with the list items and lists they leave empty:
+
+```ruby
+schema = Prosemirror::Schema.tiptap(case: :camel)  # saves `bulletList`, as Tiptap expects
+document = Prosemirror.parse(params[:content], schema:).transform do |node|
+  case node
+  in { type: :paragraph | :list_item } if node.blank?(schema) then nil
+  in { type: :bullet_list | :ordered_list } if node.content.empty? then nil
+  else node
   end
 end
+record.update!(content: document.to_h)
 ```
 
-`to_text` still separates empty paragraphs with a blank line, as ProseMirror does. `to_h` keeps them, so the editor gets the document back unchanged.
+Without `case: :camel`, `to_h` writes snake_case type names like `bullet_list`, which Tiptap doesn't read. `blank?` gets the schema so that it knows your custom types: an image block isn't blank. Children are transformed first, so the list sees its remaining items. Attributes the schema doesn't declare are kept. When something the schema requires is gone, like the only paragraph of a list item kept by another rule, `transform` adds it back and reports it with `on_invalid`.
 
 ## Heading anchors
 

data/README.md

@@ -1,13 +1,13 @@
 # ProseMirror
 
-Render [ProseMirror](https://prosemirror.net) and [Tiptap](https://tiptap.dev) documents to HTML and text, and import HTML into them, in Ruby.
+Render [ProseMirror](https://prosemirror.net) and [Tiptap](https://tiptap.dev) documents to HTML and text, and import HTML, Markdown and text into them, in Ruby.
 
 - HTML is built with [Phlex](https://www.phlex.fun) and escaped as it is written.
 - Attributes are typed with [Literal](https://literal.fun).
 - Link and image URLs must use an allowed scheme.
 - No Rails dependency.
 
-This gem replaces [prosemirror_to_html](https://github.com/etaminstudio/prosemirror_to_html) and [html_to_prosemirror](https://github.com/etaminstudio/html_to_prosemirror). Markdown import is planned: see the [roadmap](ROADMAP.md).
+This gem replaces [prosemirror_to_html](https://github.com/etaminstudio/prosemirror_to_html) and [html_to_prosemirror](https://github.com/etaminstudio/html_to_prosemirror). See the [roadmap](ROADMAP.md) for what comes next.
 
 ## Installation
 
@@ -48,6 +48,32 @@ doc.select { |node| node in { type: :heading, attrs: { level: 1 } } }
 doc.content.first.text_content
 ```
 
+## Transforming documents
+
+`transform` builds a new document from a block called with each node. The block returns the node to keep, another node, an array of nodes to put in its place, or nil to remove it:
+
+```ruby
+doc = doc.transform do |node|
+  case node
+  in { type: :paragraph } if node.blank? then nil
+  in { type: :heading, attrs: { level: 1 } } then node.with(attrs: { level: 2 })
+  in { type: :bullet_list } then node.content.flat_map(&:content)  # unwrap the list
+  else node
+  end
+end
+```
+
+- Children are transformed first: the block sees a node with its transformed content, and the nodes it returns aren't transformed again.
+- Nodes are immutable: `with` returns a copy with some properties changed, like `content`, `attrs`, `marks` or `text`. Attributes the schema doesn't declare are kept.
+- `blank?` is true for nodes that show nothing: no text but whitespace, and no leaf nodes but hard breaks.
+- Adjacent text nodes with the same marks are joined. When a node's content changed and lacks what its type requires, like the only paragraph of a list item, the required nodes are added and reported with `on_invalid`.
+
+`check` raises `Prosemirror::InvalidDocument` for the first problem with the schema, with its path: an unknown type, an invalid attribute, content that doesn't match a [content expression](TYPES.md#content-expressions), or marks a node doesn't allow. `valid?` returns true or false. Rendering doesn't need a valid document.
+
+```ruby
+doc.check  # raises "doc.content[0]: invalid content for blockquote: paragraph text"
+```
+
 ## HTML import
 
 HTML import uses [Herb](https://herb-tools.dev)'s parser, which never raises on malformed HTML. Add it to your Gemfile:
@@ -70,6 +96,35 @@ It works like ProseMirror's `DOMParser`, and for the built-in types gives the sa
 - entities are decoded, and ERB tags (`<%= … %>`) are kept as text;
 - invalid attributes are fixed and reported, as in [Invalid documents](#invalid-documents).
 
+## Markdown import
+
+Markdown import uses [commonmarker](https://github.com/gjtorikian/commonmarker)'s parser, written in Rust. Add it to your Gemfile:
+
+```sh
+bundle add commonmarker
+```
+
+```ruby
+doc = Prosemirror.from_markdown("## Hello\n\nSome **bold** text.")
+```
+
+It reads GitHub Flavored Markdown: tables, strikethrough and autolinks. Each Markdown node becomes the element commonmarker renders for it, and the schema's parse rules apply as in HTML import, so custom types can match Markdown too: a type that matches `img` also gets Markdown images. For the built-in types, it gives the same JSON as rendering the Markdown to HTML and importing it in Tiptap, except that:
+
+- code blocks lose their final newline, as in prosemirror-markdown;
+- task list items (`- [ ] todo`) stay as list items starting with `[ ] ` or `[x] `.
+
+Raw HTML in Markdown goes through HTML import when the herb gem is installed, and stays as text otherwise.
+
+YAML front matter, between `---` lines at the start, is dropped. Footnotes aren't supported yet: `[^1]: Note` reads as a link definition, as in CommonMark. Markdown nested more than 1,000 levels deep raises `Prosemirror::Error`.
+
+## Text import
+
+```ruby
+doc = Prosemirror.from_text("First line\nsecond line\n\nNew paragraph")
+```
+
+Blank lines separate paragraphs, and other line breaks become hard breaks. Spaces are kept.
+
 ## Schemas
 
 Two schemas are included:
@@ -163,7 +218,15 @@ Prosemirror::Schema.tiptap(
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests, and `rake` to also run [Standard](https://github.com/standardrb/standard). You can also run `bin/console` for an interactive prompt that will allow you to experiment.
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+To install this gem onto your local machine, run `bundle exec rake install`.
+
+To release a new version, update the version number in `version.rb` and the changelog, and merge them into main. Then tag the version and push the tag:
+
+```sh
+git tag v0.3.0 && git push origin v0.3.0
+```
+
+The [Push gem](.github/workflows/push_gem.yml) workflow publishes the gem to [rubygems.org](https://rubygems.org) with [trusted publishing](https://guides.rubygems.org/trusted-publishing/), after approval in the `release` environment.
 
 ## Contributing
 

data/ROADMAP.md

@@ -169,15 +169,15 @@ They differ in more than names:
 - Old HTML mapped to the schema: `<b>` and `<i>` to `bold` and `italic`, `<strike>` to `strike`, `<tt>` to `code`. `<font>` and `<center>` are unwrapped.
 - Tests: HTML from Tiptap gives the same JSON as Tiptap.
 
-### 0.3: Markdown and text import
+### 0.3: Markdown and text import (released)
 
-- `Prosemirror.from_markdown`, built from [`commonmarker`](https://github.com/gjtorikian/commonmarker)'s syntax tree, without going through HTML (like `prosemirror-markdown`). commonmarker's parser is written in Rust.
-- GitHub Flavored Markdown: tables, strikethrough, autolinks, task lists.
-- Custom node types can define their Markdown mapping.
+- `Prosemirror.from_markdown`, built from [`commonmarker`](https://github.com/gjtorikian/commonmarker)'s syntax tree. commonmarker's parser is written in Rust. Each Markdown node becomes the element commonmarker renders for it, without going through an HTML string, and HTML import's parse rules build the document.
+- GitHub Flavored Markdown: tables, strikethrough, autolinks. Task list items stay as text until task list nodes (0.6).
+- Custom node types match Markdown with their parse rules.
 - Raw HTML in Markdown goes through the HTML importer if Herb is installed.
 - `Prosemirror.from_text`: blank lines make paragraphs, line breaks make hard breaks.
 
-### 0.4: Working with documents
+### 0.4: Working with documents (released)
 
 - Transformations that return a new document: rewrite links, unwrap lists, split paragraphs, remove empty ones.
 

data/TYPES.md

@@ -26,6 +26,7 @@ A type is a [Phlex](https://www.phlex.fun) component, and its attributes are [Li
 - [Attributes](#attributes)
 - [Rendering](#rendering)
 - [Parse rules](#parse-rules)
+- [Markdown](#markdown)
 - [Subclassing built-in types](#subclassing-built-in-types)
 
 ## Names
@@ -186,6 +187,22 @@ The element passed to rules is a `Prosemirror::HTML::Element`, with lowercase na
 | `matches?("p.lead")` | Whether the element matches a selector. |
 | `parent` | The parent element. |
 
+## Markdown
+
+Markdown import builds the elements commonmarker renders for each Markdown node, and applies the same parse rules. To match Markdown, a type matches these elements:
+
+| Markdown | Element |
+|---|---|
+| Paragraph, heading, blockquote, thematic break | `p`, `h1`…`h6`, `blockquote`, `hr` |
+| Bullet list, ordered list, list item | `ul`, `ol` (with `start` when it isn't 1), `li` |
+| Code block | `pre` holding a `code` with a `language-…` class from the info string |
+| Table | `table`, `tr`, `th` for the first row, `td`, with a `text-align` style for aligned columns |
+| Emphasis, strong, strikethrough, inline code | `em`, `strong`, `s`, `code` |
+| Link, image | `a` with `href` and `title`, `img` with `src`, `alt` and `title` |
+| Hard line break | `br` |
+
+Raw HTML in Markdown goes through HTML import when the herb gem is installed, and stays as text otherwise.
+
 ## Subclassing built-in types
 
 A subclass keeps the name, options, attributes and parse rules of its parent, and can change any of them. Register it with `Schema#with`, which replaces the built-in type with the same name:

data/lib/prosemirror/checker.rb

@@ -0,0 +1,82 @@
+# frozen_string_literal: true
+
+module Prosemirror
+  # Checks a document against its schema, as ProseMirror's `Node#check`, and raises InvalidDocument for the
+  # first problem, with its path. See Document#check.
+  #
+  # Rendering and JSON parsing don't need valid content: they render what they get. This checks what
+  # they don't: node and mark types, attributes, content expressions, and the marks each node allows.
+  class Checker
+    def initialize(schema)
+      @schema = schema
+    end
+
+    def call(root)
+      name = @schema.json_type_name(root.type)
+      invalid("the top node is #{name}, not #{@schema.top_node}", name) unless root.type == @schema.top_node
+      check(root, name)
+    end
+
+    private
+
+    def check(node, path)
+      type = @schema.node_type(node.type) or invalid("unknown node type #{@schema.json_type_name(node.type)}", path)
+      return check_text(node, path) if node.text?
+
+      check_attrs(type, node.attrs, path)
+      node.content.each_with_index do |child, index|
+        @schema.node_type(child.type) or invalid("unknown node type #{@schema.json_type_name(child.type)}", "#{path}.content[#{index}]")
+      end
+      check_content(type, node, path)
+      node.content.each_with_index do |child, index|
+        child_path = "#{path}.content[#{index}]"
+        check_marks(type, child, child_path)
+        check(child, child_path)
+      end
+    end
+
+    def check_text(node, path)
+      invalid("empty text node", path) if node.text.to_s.empty?
+      invalid("text nodes have no content", path) unless node.content.empty?
+    end
+
+    def check_content(type, node, path)
+      if type.leaf?
+        invalid("#{name(type)} has no content, but has #{node.content.size} nodes", path) unless node.content.empty?
+        return
+      end
+
+      types = node.content.map { |child| @schema.node_type(child.type) }
+      return if @schema.content_match(type).match_types(types)&.valid_end?
+
+      found = node.content.map { |child| @schema.json_type_name(child.type) }.join(" ")
+      invalid("invalid content for #{name(type)}: #{found.empty? ? "nothing" : found}", path)
+    end
+
+    def check_marks(parent, node, path)
+      node.marks.each_with_index do |mark, index|
+        mark_type = @schema.mark_type(mark.type) or invalid("unknown mark type #{@schema.json_type_name(mark.type)}", "#{path}.marks[#{index}]")
+        check_attrs(mark_type, mark.attrs, "#{path}.marks[#{index}]")
+        invalid("#{name(parent)} doesn't allow #{name(mark_type)} marks", path) unless parent.allows_mark?(mark_type, @schema)
+      end
+      sorted = node.marks.reduce([]) { |set, mark| @schema.add_mark(mark, set) }
+      invalid("invalid marks: #{node.marks.map { |mark| @schema.json_type_name(mark.type) }.join(" ")}", path) unless sorted == node.marks
+    end
+
+    # Attributes are valid when building them from themselves reports nothing and changes nothing.
+    def check_attrs(type, attrs, path)
+      problems = []
+      built = type.build_attrs(attrs, report: ->(message) { problems << message })
+      invalid(problems.first, path) unless problems.empty?
+      invalid("invalid attributes: #{attrs.inspect}", path) unless built == attrs
+    end
+
+    def name(type)
+      @schema.json_type_name(type.type_name)
+    end
+
+    def invalid(message, path)
+      raise InvalidDocument.new(message, path: path)
+    end
+  end
+end

data/lib/prosemirror/document.rb

@@ -64,6 +64,38 @@ module Prosemirror
       root.text_content
     end
 
+    # A new document built from a block called with each node, children first. The block returns the node
+    # to keep, another node, an array of nodes to put in its place, or nil to remove it:
+    #
+    #   doc.transform do |node|
+    #     case node
+    #     in { type: :paragraph } if node.blank? then nil
+    #     in { type: :heading, attrs: { level: 1 } } then node.with(attrs: { level: 2 })
+    #     else node
+    #     end
+    #   end
+    #
+    # Adjacent text nodes with the same marks are joined. When a node's content changed and lacks what its
+    # content expression requires, the required nodes are added and reported with `on_invalid`.
+    def transform(&)
+      Document.new(Transformer.new(schema, &).call(root), schema: schema)
+    end
+
+    # Raises InvalidDocument for the first problem with the schema: an unknown node or mark type, invalid
+    # attributes, content that doesn't match a content expression, or marks a node doesn't allow. Returns
+    # the document.
+    def check
+      Checker.new(schema).call(root)
+      self
+    end
+
+    def valid?
+      check
+      true
+    rescue InvalidDocument
+      false
+    end
+
     def ==(other)
       other.is_a?(Document) && root == other.root
     end

data/lib/prosemirror/html/builder.rb

@@ -22,11 +22,19 @@ module Prosemirror
         raise Error, 'HTML import needs the herb gem: add `gem "herb"` to your Gemfile'
       end
 
+      # Whether the herb gem is installed.
+      def self.herb?
+        load_herb
+        true
+      rescue Error
+        false
+      end
+
       # A root Element holding the parsed content.
       def call(html)
         self.class.load_herb
         root = Element.new("#document")
-        Splitter.new(PIECE_SIZE).call(html.to_s).each do |piece|
+        Splitter.new(PIECE_SIZE).call(UTF8.string(html)).each do |piece|
           # Without `timeout: 0`, Herb stops after a second and returns what it parsed so far.
           build(Herb.parse(piece, track_locations: false, strict: false, timeout: 0).value.children, root)
         end

data/lib/prosemirror/html_parser.rb

@@ -41,8 +41,13 @@ module Prosemirror
 
     # The document parsed from the HTML, as its top node.
     def parse(html)
+      parse_element(HTML::Builder.new.call(html))
+    end
+
+    # The document for an Element's content, like the one HTML::Builder gives.
+    def parse_element(root)
       context = Context.new(self)
-      context.add_all(HTML::Builder.new.call(html), [])
+      context.add_all(root, [])
       context.finish
     end
 

data/lib/prosemirror/markdown/builder.rb

@@ -0,0 +1,200 @@
+# frozen_string_literal: true
+
+module Prosemirror
+  module Markdown
+    # Builds the Elements and Texts of HTML import from Markdown, with commonmarker's syntax tree
+    # (https://github.com/gjtorikian/commonmarker), so that the schema's parse rules apply to Markdown too.
+    # Each Markdown node becomes the element commonmarker renders for it, without going through HTML.
+    #
+    # - GitHub Flavored Markdown: tables, strikethrough and autolinks. Task list items stay as text (`[ ] `).
+    # - Raw HTML can span several Markdown blocks, like `<del>` around a paragraph: with the herb gem, a
+    #   document holding raw HTML is rendered to HTML by commonmarker and imported. Without it, raw HTML
+    #   stays as text.
+    # - Code blocks lose their final newline, as in prosemirror-markdown.
+    # - YAML front matter, between `---` lines at the start, is dropped.
+    class Builder
+      MAX_DEPTH = HTML::Builder::MAX_DEPTH
+      EXTENSIONS = {
+        strikethrough: true, table: true, autolink: true, tasklist: false, shortcodes: false, header_ids: nil,
+        front_matter_delimiter: "---"
+      }.freeze
+      RENDER_OPTIONS = {extension: EXTENSIONS, render: {unsafe: true, hardbreaks: false, github_pre_lang: false}}.freeze
+      TAGS = {
+        paragraph: "p", block_quote: "blockquote", item: "li", thematic_break: "hr", table_row: "tr",
+        emph: "em", strong: "strong", strikethrough: "s", linebreak: "br"
+      }.freeze
+      # A table's delimiter row, like `| :-- | --: |`, at the end of its line.
+      DELIMITER_ROW = /\|?(?:\s*:?-+:?\s*\|)*\s*:?-+:?\s*\|?\s*\z/
+      # The bytes commonmarker escapes in URLs, as comrak's `escape_href`: all but these, and `%` when it
+      # doesn't start an escape. `&` and `'` are kept: comrak writes them as HTML references.
+      URL_UNSAFE = %r{%(?!\h\h)|[^-_.+!*(),#@?=;:/$~A-Za-z0-9&'%]}n
+      # An IPv6 host at the start of a URL, like `http://[::1]`, kept as it is.
+      IPV6_HOST = %r{\A[A-Za-z][A-Za-z0-9+.-]*://\[[\h:.]+\]}
+
+      def self.load_commonmarker
+        require "commonmarker"
+      rescue LoadError
+        raise Error, 'Markdown import needs the commonmarker gem: add `gem "commonmarker"` to your Gemfile'
+      end
+
+      # A root Element holding the content.
+      def call(markdown)
+        self.class.load_commonmarker
+        markdown = UTF8.string(markdown)
+        document = parse(markdown)
+        return from_html(document) if raw_html?(document) && HTML::Builder.herb?
+
+        @lines = markdown.lines
+        root = HTML::Element.new("#document")
+        build(document, root)
+        root
+      end
+
+      private
+
+      def parse(markdown)
+        Commonmarker.parse(markdown, options: {extension: EXTENSIONS})
+      rescue Commonmarker::NestingDepthExceededError => error
+        raise Error, "Markdown nested too deeply: #{error.message}"
+      end
+
+      # Walks the tree without recursion. Each item of the stack is a node, the element to add it to, and
+      # the depth of that element. Elements nested deeper than MAX_DEPTH are flattened.
+      def build(document, root)
+        stack = document.to_a.reverse.map { |node| [node, root, 0] }
+        until stack.empty?
+          node, target, depth = stack.pop
+          element = add(node, target)
+          next unless element
+
+          if depth >= MAX_DEPTH
+            element = target
+          else
+            target.append(element)
+          end
+          node.to_a.reverse_each { |child| stack << [child, element, element.equal?(target) ? depth : depth + 1] }
+        end
+      end
+
+      # Adds a node to the target. Returns the element to add its children to, when they need adding.
+      def add(node, target)
+        case node.type
+        when *TAGS.keys then HTML::Element.new(TAGS[node.type])
+        when :heading then HTML::Element.new("h#{node.header_level}")
+        when :list then list(node)
+        when :code_block then code_block(node, target)
+        when :table then table(node)
+        when :table_cell then table_cell(node)
+        when :link then HTML::Element.new("a", {"href" => escape_url(node.url), "title" => presence(node.title)}.compact)
+        when :image then image(node, target)
+        when :code then text(target, node.string_content, "code")
+        when :text then text(target, node.string_content)
+        when :softbreak then text(target, "\n")
+        when :html_block then text(target.append(HTML::Element.new("p")), node.literal.chomp)
+        when :html_inline then text(target, node.literal)
+        when :frontmatter then nil
+        else HTML::Element.new("span") # Unwrapped by HTML import, like escaped characters (`\*`).
+        end
+      end
+
+      def list(node)
+        return HTML::Element.new("ul") unless node.list_type == :ordered
+
+        HTML::Element.new("ol", (node.list_start == 1) ? {} : {"start" => node.list_start.to_s})
+      end
+
+      def code_block(node, target)
+        language = node.fence_info.to_s.split.first
+        code = HTML::Element.new("code", language ? {"class" => "language-#{language}"} : {})
+        code.append(HTML::Text.new(node.string_content.chomp))
+        target.append(HTML::Element.new("pre")).append(code)
+        nil
+      end
+
+      def table(node)
+        @alignments = alignments(node)
+        @row = 0
+        HTML::Element.new("table")
+      end
+
+      # The first row holds header cells.
+      def table_cell(node)
+        index = cell_index(node)
+        @row += 1 if index.zero?
+        align = @alignments[index]
+        HTML::Element.new((@row == 1) ? "th" : "td", align ? {"style" => "text-align: #{align}"} : {})
+      end
+
+      def cell_index(node)
+        index = 0
+        index += 1 while (node = node.previous_sibling)
+        index
+      end
+
+      # commonmarker doesn't give the alignments of a table's columns: they are read from its delimiter row,
+      # the line after its first line.
+      def alignments(node)
+        row = @lines[node.source_position[:start_line]].to_s.chomp[DELIMITER_ROW].to_s
+        row.strip.delete_prefix("|").delete_suffix("|").split("|").map do |cell|
+          cell = cell.strip
+          if cell.start_with?(":") && cell.end_with?(":") then "center"
+          elsif cell.end_with?(":") then "right"
+          elsif cell.start_with?(":") then "left"
+          end
+        end
+      end
+
+      def image(node, target)
+        attributes = {"src" => escape_url(node.url), "alt" => plain_text(node), "title" => presence(node.title)}.compact
+        target.append(HTML::Element.new("img", attributes))
+        nil
+      end
+
+      def text(target, value, tag = nil)
+        target = target.append(HTML::Element.new(tag)) if tag
+        target.append(HTML::Text.new(value))
+        nil
+      end
+
+      def raw_html?(document)
+        document.walk { |node| return true if node.type == :html_block || node.type == :html_inline }
+        false
+      end
+
+      def from_html(document)
+        root = HTML::Builder.new.call(document.to_html(options: RENDER_OPTIONS, plugins: {syntax_highlighter: nil}))
+        chomp_code_blocks(root)
+        root
+      end
+
+      # The code blocks commonmarker renders, `<pre><code>`, not the `<pre>` elements of raw HTML.
+      def chomp_code_blocks(element)
+        element.children.each do |child|
+          next unless child.element?
+          next chomp_code_blocks(child) unless child.name == "pre"
+
+          code = child.children.first
+          next unless code&.element? && code.name == "code" && child.children.size == 1
+
+          last = code.children.last
+          code.children[-1] = HTML::Text.new(last.value.chomp).tap { |text| text.parent = code } if last.is_a?(HTML::Text)
+        end
+      end
+
+      # The URL as commonmarker renders it, with the characters URLs can't hold percent-encoded.
+      def escape_url(url)
+        host = url[IPV6_HOST].to_s
+        rest = url.delete_prefix(host).b.gsub(URL_UNSAFE) { |byte| byte.ord.zero? ? "%EF%BF%BD" : format("%%%02X", byte.ord) }
+        host + rest.force_encoding(Encoding::UTF_8)
+      end
+
+      def plain_text(node)
+        node.to_a.map { |child| %i[text code].include?(child.type) ? child.string_content : plain_text(child) }.join
+      end
+
+      def presence(value)
+        value unless value.nil? || value.empty?
+      end
+    end
+  end
+end

data/lib/prosemirror/node.rb

@@ -11,6 +11,9 @@ module Prosemirror
   # in JSON (for example from a Tiptap extension the schema doesn't know): they are never rendered,
   # and `to_h` gives them back unchanged.
   class Node < Literal::Data
+    # Whitespace, including non-breaking spaces.
+    BLANK = /\A[[:space:]]*\z/
+
     prop :type, Symbol, &Coerce::SYMBOL
     prop :attrs, _Hash(Symbol, _Any?), default: {}.freeze, &Coerce::FROZEN
     prop :unknown_attrs, _Hash(String, _Any?), default: {}.freeze, &Coerce::FROZEN
@@ -70,6 +73,18 @@ module Prosemirror
       end
     end
 
+    # Whether the node shows nothing: no text but whitespace, and no leaf nodes but line breaks, like an
+    # empty paragraph, or a list item holding one. The schema says which nodes are leaves: an image isn't
+    # blank, and neither is a node of a type it doesn't know.
+    def blank?(schema = Prosemirror.schema)
+      [self, *descendants].none? do |node|
+        next !BLANK.match?(node.text) if node.text?
+
+        type = schema.node_type(node.type)
+        type.nil? || (type.leaf? && !type.linebreak_replacement?)
+      end
+    end
+
     def same_markup?(other)
       @type == other.type && @attrs == other.attrs && @unknown_attrs == other.unknown_attrs && @marks == other.marks
     end

data/lib/prosemirror/schema.rb

@@ -117,6 +117,27 @@ module Prosemirror
       Document.new(HTMLParser.new(self).parse(html), schema: self)
     end
 
+    # Build a document from Markdown, with the parse rules of the schema's types. Needs the `commonmarker` gem.
+    def parse_markdown(markdown)
+      Document.new(HTMLParser.new(self).parse_element(Markdown::Builder.new.call(markdown)), schema: self)
+    end
+
+    # Build a document from plain text: blank lines separate paragraphs, and other line breaks become the
+    # schema's line break node, like `hard_break`, or spaces without one. Spaces are kept.
+    def parse_text(text)
+      paragraph = node_type(:paragraph) || @nodes.find { |type| textblock?(type) }
+      linebreak = @nodes.find(&:linebreak_replacement?)
+      separator = linebreak ? {"type" => linebreak.type_name.to_s} : {"type" => "text", "text" => " "}
+      blocks = UTF8.string(text).gsub(/\r\n?/, "\n").split(/\n[ \t]*\n/).map { |block| block.gsub(/\A\n+|\n+\z/, "") }
+      content = blocks.reject { |block| block.strip.empty? }.map do |block|
+        inline = block.split("\n").each_with_index.flat_map do |line, index|
+          [(separator if index.positive?), ({"type" => "text", "text" => line} unless line.empty?)].compact
+        end
+        {"type" => paragraph.type_name.to_s, "content" => inline}
+      end
+      parse({"type" => @top_node.to_s, "content" => content.empty? ? [{"type" => paragraph.type_name.to_s}] : content})
+    end
+
     # The type name as written in JSON.
     def json_type_name(name)
       (@case == :camel) ? Naming.camel(name) : name.to_s

data/lib/prosemirror/transformer.rb

@@ -0,0 +1,81 @@
+# frozen_string_literal: true
+
+module Prosemirror
+  # Builds a new document from a block called with each node, children first. See Document#transform.
+  #
+  # The block returns the node to keep, another node, an array of nodes to put in its place, or nil to
+  # remove it. Adjacent text nodes with the same marks are then joined, and empty ones removed. When a
+  # node's content changed and no longer has what its content expression requires, the required nodes
+  # are added, as an empty paragraph in an empty list item, and reported. Other problems are left for
+  # Document#check.
+  class Transformer
+    def initialize(schema, &block)
+      raise ArgumentError, "transform needs a block" unless block
+
+      @schema = schema
+      @block = block
+    end
+
+    # The root, with its transformed content. The root itself isn't given to the block.
+    def call(root)
+      path = @schema.json_type_name(root.type)
+      filled(root, root.with(content: transform_content(root, path)), path)
+    end
+
+    private
+
+    def transform_content(node, path)
+      content = node.content.each_with_index.flat_map { |child, index| transform(child, "#{path}.content[#{index}]") }
+      Node.join_text(content.reject { |child| child.text? && child.text.empty? })
+    end
+
+    # The nodes the block gives for a node, after its content, with their required content added: the block
+    # sees an emptied list item, and can remove it.
+    def transform(node, path)
+      changed = node.content.empty? ? node : node.with(content: transform_content(node, path))
+      nodes = case (result = @block.call(changed))
+      when nil then []
+      when Node then [result]
+      when Array then result
+      end
+      raise ArgumentError, "the transform block returns a node, an array of nodes or nil, not #{result.inspect}" unless nodes&.all?(Node)
+
+      nodes.map { |result_node| filled(node, result_node, path) }
+    end
+
+    # The node with its required content added, when its content changed.
+    def filled(original, node, path)
+      return node if node.content.equal?(original.content) || node.content == original.content
+
+      content = fill(node, node.content, path)
+      content.equal?(node.content) ? node : node.with(content: content)
+    end
+
+    # The content with the nodes its type requires added in front of it, or after it, when they can be.
+    def fill(node, content, path)
+      type = @schema.node_type(node.type)
+      types = content.map { |child| @schema.node_type(child.type) }
+      return content if type.nil? || type.leaf? || types.include?(nil)
+
+      match = @schema.content_match(type)
+      return content if match.match_types(types)&.valid_end?
+
+      if (before = match.fill_before(types, @schema, to_end: true))
+        added = created(before, node, path) and return added + content
+      elsif (after = match.match_types(types)&.fill_before([], @schema, to_end: true))
+        added = created(after, node, path) and return content + added
+      end
+      content
+    end
+
+    # Empty nodes of these types, filled with what they require, or nil when one can't be created.
+    def created(types, node, path)
+      nodes = types.map { |type| @schema.create_and_fill(type) }
+      return nil if nodes.include?(nil)
+
+      names = types.map { |type| @schema.json_type_name(type.type_name) }.join(", ")
+      @schema.report("added required #{names} to #{@schema.json_type_name(node.type)}", path: path)
+      nodes
+    end
+  end
+end

data/lib/prosemirror/utf8.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+module Prosemirror
+  # Imported HTML, Markdown and text, as valid UTF-8.
+  module UTF8
+    module_function
+
+    # The value as a UTF-8 String: nil becomes empty, other encodings are read as UTF-8 (a binary string
+    # from a file or an HTTP body), and invalid bytes become U+FFFD, as browsers do.
+    def string(value)
+      string = value.to_s
+      string = string.dup.force_encoding(Encoding::UTF_8) unless string.encoding == Encoding::UTF_8
+      string.valid_encoding? ? string : string.scrub
+    end
+  end
+end

data/lib/prosemirror/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Prosemirror
-  VERSION = "0.2.0"
+  VERSION = "0.4.0"
 end

data/lib/prosemirror.rb

@@ -10,7 +10,8 @@ require "zeitwerk"
 
 loader = Zeitwerk::Loader.for_gem
 loader.inflector.inflect(
-  "html" => "HTML", "html_parser" => "HTMLParser", "html_renderer" => "HTMLRenderer", "json_parser" => "JSONParser"
+  "html" => "HTML", "html_parser" => "HTMLParser", "html_renderer" => "HTMLRenderer", "json_parser" => "JSONParser",
+  "utf8" => "UTF8"
 )
 loader.setup
 
@@ -45,5 +46,15 @@ module Prosemirror
     def from_html(html, schema: self.schema)
       schema.parse_html(html)
     end
+
+    # Build a document from Markdown, with the schema's parse rules. Needs the `commonmarker` gem.
+    def from_markdown(markdown, schema: self.schema)
+      schema.parse_markdown(markdown)
+    end
+
+    # Build a document from plain text: blank lines separate paragraphs, other line breaks become hard breaks.
+    def from_text(text, schema: self.schema)
+      schema.parse_text(text)
+    end
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: prosemirror
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.4.0
 platform: ruby
 authors:
 - Hans Lemuet
@@ -66,6 +66,7 @@ files:
 - ROADMAP.md
 - TYPES.md
 - lib/prosemirror.rb
+- lib/prosemirror/checker.rb
 - lib/prosemirror/coerce.rb
 - lib/prosemirror/content_expression.rb
 - lib/prosemirror/content_match.rb
@@ -82,6 +83,7 @@ files:
 - lib/prosemirror/json_parser.rb
 - lib/prosemirror/mark.rb
 - lib/prosemirror/mark_type.rb
+- lib/prosemirror/markdown/builder.rb
 - lib/prosemirror/marks/basic/code.rb
 - lib/prosemirror/marks/basic/em.rb
 - lib/prosemirror/marks/basic/link.rb
@@ -121,7 +123,9 @@ files:
 - lib/prosemirror/rendering.rb
 - lib/prosemirror/schema.rb
 - lib/prosemirror/text_serializer.rb
+- lib/prosemirror/transformer.rb
 - lib/prosemirror/type_definition.rb
+- lib/prosemirror/utf8.rb
 - lib/prosemirror/version.rb
 - sig/prosemirror.rbs
 homepage: https://github.com/etaminstudio/prosemirror