prosemirror 0.6.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8aae1987528266c7bd7c4b3c3d8f7c82e74f74cf89ee9e7ac87b1732ff76634d
-  data.tar.gz: d5cb877b264eb0d39cd40cd8b4909ec94b423ac439a6719df23c70eb6cde6182
+  metadata.gz: 76385bda3d9b696b80e976154bf40c5d33ad2b9ac79b48e5f3fa8a6f7bf224e8
+  data.tar.gz: 7acda4288b7bbb2d731a111143dc26b715177551476829ffa6416a7a298180c6
 SHA512:
-  metadata.gz: 947eab5fdc041f059be90be81b94bc66bb1505118b983ed982100faf1ccf2748fdcf753f83384385af72e80cc610473e24eeca31aa6a404ed93f431a8d3138d2
-  data.tar.gz: 4a56748f16cc7ac1af9ad1660518870dfded5f22b089e714417cf74afb970e0ec8b9f53d2d9e1adc82662efac63c449ba1dff10d07988ed8ca110413143c2452
+  metadata.gz: 7c9fdb3905684864d3662c4bdb25c2cee8efb055d59a64b28b4dd414b4a08759deae97a5f071ab605e8c94ce235356c30bb54c364053d5cc2d0b105eee151c0a
+  data.tar.gz: 42054ca794261591e076b654aac9068bf629f7aeb82d18a5eb51fdbf4602d95dd5199907a4caa5363ad883b6b1628490ab848f32310aea4dbcbd10eadd7b7577

data/CHANGELOG.md

@@ -1,5 +1,9 @@
 ## [Unreleased]
 
+## [0.7.0] - 2026-10-24
+
+- `resolve_urls:` resolves all the URLs of a document at once, before rendering it, to look up records in one query, where `resolve_url:` is called once per URL. Attributes holding URLs are declared with `prop … url: true`, like links' `href` and images' `src`, and `Document#urls` lists them.
+
 ## [0.6.0] - 2026-10-24
 
 - Tiptap's YouTube and Twitch embeds, in `Schema.tiptap`: `youtube` and `twitch`, with the video's URL in `src`, rendered as an iframe of its embed URL, built from the video's id. They are imported from Tiptap's markup, and from iframes of embed URLs without it. Subclasses set Twitch's `parent` domain and YouTube's nocookie host. Markdown export writes a link to the video, or the iframe with `html: true`.

data/COOKBOOK.md

@@ -104,7 +104,26 @@ end
 
 ## Links to records with GlobalID
 
-Store links to records as GlobalIDs (`gid://app/Page/1`), so they keep working when a record's URL changes, and turn them into URLs when rendering:
+Store links to records as GlobalIDs (`gid://app/Page/1`), so they keep working when a record's URL changes, and turn them into URLs when rendering. `resolve_urls` gets all the URLs of a document before rendering it, so its records load in one query:
+
+```ruby
+Prosemirror::Schema.tiptap(
+  resolve_urls: lambda do |urls|
+    gids = urls.select { |url| url.start_with?("gid://") }
+    records = GlobalID::Locator.locate_many(gids, ignore_missing: true).index_by { |record| record.to_global_id.to_s }
+
+    urls.to_h do |url|
+      next [url, url] unless url.start_with?("gid://")
+
+      [url, records[url]&.then { |record| Rails.application.routes.url_helpers.polymorphic_path(record) }]
+    end
+  end
+)
+```
+
+URLs are resolved before the allowed URL schemes are checked, so `gid://` doesn't need to be allowed. When a record doesn't exist, its link is removed, its text is kept, and the problem goes to `on_invalid`.
+
+`resolve_url` is simpler, but runs one query per link:
 
 ```ruby
 Prosemirror::Schema.tiptap(
@@ -116,5 +135,3 @@ Prosemirror::Schema.tiptap(
   end
 )
 ```
-
-`resolve_url` runs before the allowed URL schemes are checked, so `gid://` doesn't need to be allowed. When the record doesn't exist, the link is removed, its text is kept, and the problem goes to `on_invalid`. Each link runs one query.

data/MIGRATING.md

@@ -0,0 +1,131 @@
+# Migrating from prosemirror_to_html and html_to_prosemirror
+
+`prosemirror` replaces [`prosemirror_to_html`](https://github.com/etaminstudio/prosemirror_to_html) and [`html_to_prosemirror`](https://github.com/etaminstudio/html_to_prosemirror), which have the same maintainers. Rendering and import use one schema, so a custom type is declared once for both.
+
+```ruby
+# Gemfile
+gem "prosemirror"
+gem "herb" # optional, only if you need HTML import
+```
+
+## Rendering, from prosemirror_to_html
+
+```ruby
+# Before
+ProsemirrorToHtml::Renderer.new.render(json)
+
+# After
+Prosemirror.parse(json).to_html
+```
+
+`parse` takes the same input: a JSON string, or a Hash with String or Symbol keys. Type names can be in snake_case or camelCase.
+
+### Custom nodes and marks
+
+A node type is a [Phlex](https://www.phlex.fun) component, and its attributes are declared with `prop`. It's registered in a schema instead of a renderer:
+
+```ruby
+# Before
+class Callout < ProsemirrorToHtml::Nodes::Node
+  @node_type = "callout"
+
+  def tag
+    [{tag: "aside", attrs: {class: "callout callout-#{@node[:attrs][:tone]}"}}]
+  end
+end
+
+renderer = ProsemirrorToHtml::Renderer.new.add_node(Callout)
+renderer.render(json)
+
+# After
+class Callout < Prosemirror::NodeType
+  group :block
+  content "block+"
+  prop :tone, _String?
+
+  def view_template(&) = aside(class: ["callout", @tone && "callout-#{@tone}"], &)
+end
+
+schema = Prosemirror::Schema.tiptap.with(Callout)
+Prosemirror.parse(json, schema:).to_html
+```
+
+| prosemirror_to_html | prosemirror |
+|---|---|
+| `@node_type = "callout"` | The class name (`Callout` → `callout`), or `type_name :callout` |
+| `@tag_name`, `def tag` | `def view_template(&)`, which yields to render the content |
+| `@node[:attrs][:tone]` | `prop :tone, _String?`, then `@tone` |
+| `def self_closing = true` | No `content`: the node is a leaf |
+| `def text` | `view_template` renders whatever the node shows |
+| `renderer.add_node(Callout)`, `add_mark` | `Prosemirror::Schema.tiptap.with(Callout)` |
+| `renderer.replace_node(Old, New)` | A subclass of the built-in type, registered with `with`, which replaces it |
+| `with_nodes([...])`, `with_marks([...])` | `Prosemirror::Schema.new(nodes: [...], marks: [...])` |
+
+Mark types work the same way, with `Prosemirror::MarkType`. See [Node and mark types](TYPES.md) for everything a type can declare.
+
+### Differences in the HTML
+
+prosemirror_to_html built HTML strings, then cleaned them up with Loofah. `prosemirror` builds the HTML with Phlex, which escapes text and attribute values, and cleans nothing up afterwards:
+
+- **Only declared attributes are rendered.** prosemirror_to_html wrote every attribute of a heading, link, image, ordered list or table cell into the HTML, so `{"level": 2, "id": "intro"}` gave `<h2 id="intro">`. `prosemirror` renders the attributes each type declares. To render more, subclass the type (see [Changing built-in types](README.md#changing-built-in-types)). Other attributes are kept in `to_h`.
+- **Unsafe URLs are dropped.** A link whose `href` isn't allowed renders as its text, instead of an `<a>` without `href`, and an image without an allowed `src` renders nothing. See [URLs](README.md#urls).
+- **Custom types render as written.** Loofah removed elements and attributes it didn't know, like `<iframe>`, `<picture>` or `srcset`. They now render. Pass URLs from the document through `safe_url`.
+- **Invalid attributes are fixed and reported** with `on_invalid`, instead of being written as they are. See [Invalid documents](README.md#invalid-documents).
+- **HTML5:** strike is `<s>`, not `<strike>`. A code block's language is a `language-` class on `<code>`, and an ordered list starting at 1 has no `start`.
+- **Marks are nested in schema order**, as Tiptap nests them, rather than in the order of the JSON: `<strong><em>`.
+- **More types:** highlight, task lists, details, mentions, and YouTube and Twitch embeds render, where prosemirror_to_html rendered only their text, or nothing.
+
+## HTML import, from html_to_prosemirror
+
+```ruby
+# Before
+HtmlToProsemirror::Renderer.new.render(html)                          # snake_case
+HtmlToProsemirror::Renderer.new(case_type: :lower_camel).render(html) # camelCase
+
+# After
+Prosemirror.from_html(html).to_h
+Prosemirror.from_html(html, schema: Prosemirror::Schema.tiptap(case: :camel)).to_h
+```
+
+`to_h` returns a Hash with String keys, where html_to_prosemirror returned Symbol keys. HTML import needs the [herb](https://herb-tools.dev) gem.
+
+### Differences in the JSON
+
+The JSON follows the schema, as Tiptap's `generateJSON` does:
+
+- **Documents are valid.** Text in a list item is wrapped in a paragraph, text at the top level too, and attributes have their types: a heading's `level` is `1`, not `"1"`.
+- **Nested marks keep their text.** html_to_prosemirror dropped text in nested marks, like `<a><b>bold link</b></a>`.
+- **Styles become marks:** `font-weight`, `font-style`, `text-decoration` and `vertical-align`, as in pasted Word or Google Docs content.
+- **Whitespace is kept** as HTML defines it: newlines in `<pre>` stay, and character references like `&copy` are decoded.
+- **More types:** strike, underline, subscript, superscript, highlight, tables, task lists, details, mentions and embeds.
+
+### The `user` node
+
+html_to_prosemirror turned `<user-mention data-id="7">` into a `user` node. `prosemirror` has Tiptap's `mention` instead. To keep the `user` node, declare it:
+
+```ruby
+class User < Prosemirror::NodeType
+  group :inline
+  inline
+  prop :id, _String?, parse_html: ->(element) { element["data-id"] }
+  parse_html tag: "user-mention"
+
+  def view_template = span(class: "user-mention", data_id: @id)
+end
+
+schema = Prosemirror::Schema.tiptap.with(User)
+Prosemirror.from_html(html, schema:).to_h # {"type" => "user", "attrs" => {"id" => "7"}}
+```
+
+## Going further
+
+Beyond what the legacy gems did, `prosemirror` can:
+
+- write a document as [Markdown](README.md#markdown-export), useful for LLMs, or as [plain text](README.md#usage);
+- import [Markdown](README.md#markdown-import) and [plain text](README.md#text-import), with the same schema as HTML import;
+- [query and transform documents](README.md#transforming-documents): nodes are immutable values with pattern matching, and `transform` builds a new document, for example to [remove empty paragraphs before saving](COOKBOOK.md#remove-empty-paragraphs-before-saving);
+- [check documents](README.md#transforming-documents) against the schema, and fix and report [invalid documents](README.md#invalid-documents);
+- render inside Rails views and [Phlex components](README.md#usage), and [your own components](COOKBOOK.md#render-your-own-components) for custom nodes;
+- resolve link URLs, for example [links to records with GlobalID](COOKBOOK.md#links-to-records-with-globalid), with [allowed URL schemes](README.md#urls);
+- use Tiptap's [task lists, details, mentions, YouTube and Twitch embeds, and text alignment](README.md#schemas), or ProseMirror's own schema;
+- declare custom types with [content expressions, attributes and parse rules](TYPES.md).

data/README.md

@@ -7,7 +7,7 @@ Render [ProseMirror](https://prosemirror.net) and [Tiptap](https://tiptap.dev) d
 - Link and image URLs must use an allowed scheme.
 - No Rails dependency.
 
-This gem replaces [prosemirror_to_html](https://github.com/etaminstudio/prosemirror_to_html) and [html_to_prosemirror](https://github.com/etaminstudio/html_to_prosemirror). See the [roadmap](ROADMAP.md) for what comes next.
+This gem replaces [prosemirror_to_html](https://github.com/etaminstudio/prosemirror_to_html) and [html_to_prosemirror](https://github.com/etaminstudio/html_to_prosemirror): see [Migrating](MIGRATING.md). See the [roadmap](ROADMAP.md) for what comes next.
 
 ## Installation
 
@@ -259,6 +259,8 @@ Prosemirror::Schema.tiptap(
 )
 ```
 
+`resolve_urls:` gets all the URLs of a document at once, before rendering it, and returns a Hash from each URL to its rewritten URL, to look up records in one query (see the [cookbook](COOKBOOK.md#links-to-records-with-globalid)). A URL missing from the Hash, or mapped to nil, is removed and reported, as with `resolve_url`. The URLs are those of the attributes declared with `url: true`, like links' `href` and images' `src`, which `doc.urls` lists. URLs that types build while rendering, like YouTube's embed URLs, are resolved one at a time.
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests, and `rake` to also run [Standard](https://github.com/standardrb/standard). You can also run `bin/console` for an interactive prompt that will allow you to experiment.

data/ROADMAP.md

@@ -205,6 +205,11 @@ They differ in more than names:
 - Text alignment, as an option: `Schema.tiptap(text_align: true)`.
 - GitHub Flavored Markdown task lists in Markdown import and export. Details are written as GitHub's `<details>` HTML.
 
+### 0.7: Resolving URLs at once (released)
+
+- `resolve_urls:` resolves all the URLs of a document before rendering it, to look up records in one query. Types declare their URL attributes with `prop … url: true`, and `Document#urls` lists them.
+- A [migration guide](MIGRATING.md) from `prosemirror_to_html` and `html_to_prosemirror`.
+
 ### 1.0
 
 - Stable API, documentation site, benchmarks.

data/TYPES.md

@@ -95,12 +95,13 @@ prop :alt, _String?
 - **Default:** used when the attribute is missing or invalid. Literal needs a frozen value or a Proc: `default: "left".freeze`, or a file with `# frozen_string_literal: true`. An attribute that accepts nil defaults to nil. An attribute without a default is required: without a valid value, the node or mark is removed.
 - **Coercion:** the block converts values before they are checked. `Prosemirror::Coerce::INTEGER` turns `"2"` into `2`.
 
-`prop` takes three more options:
+`prop` takes four more options:
 
 | Option | Meaning |
 |---|---|
 | `json: "textAlign"` | The name in JSON, when it differs from the property name. |
 | `aliases: ["order"]` | Other JSON names accepted on input. |
+| `url: true` | The attribute holds a URL: `doc.urls` lists it, and the schema's `resolve_urls` resolves it with the document's other URLs before rendering. Render it with `safe_url`. |
 | `parse_html: ->(element) { … }` | How to read the attribute from an HTML element, when importing HTML. By default, the HTML attribute with the JSON name is read. |
 
 Attributes that a type doesn't declare are kept in `unknown_attrs`. They are never rendered, and `to_h` gives them back.

data/lib/prosemirror/document.rb

@@ -24,6 +24,15 @@ module Prosemirror
       root.descendants(&)
     end
 
+    # The URLs in the attributes of its nodes and marks declared with `url: true`, like links' `href` and
+    # images' `src`, without duplicates.
+    def urls
+      each_with_object(Set.new) do |node, urls|
+        add_urls(urls, schema.node_type(node.type), node.attrs)
+        node.marks.each { |mark| add_urls(urls, schema.mark_type(mark.type), mark.attrs) }
+      end.to_a
+    end
+
     # The document as HTML. The string is `html_safe` when ActiveSupport is loaded.
     def to_html
       html = to_phlex.call
@@ -110,5 +119,14 @@ module Prosemirror
     def hash
       root.hash
     end
+
+    private
+
+    def add_urls(urls, type, attrs)
+      type&.url_attributes&.each do |name|
+        value = attrs[name]
+        urls << value if value.is_a?(String)
+      end
+    end
   end
 end

data/lib/prosemirror/html_renderer.rb

@@ -13,7 +13,9 @@ module Prosemirror
       @_prosemirror_schema = document.schema
     end
 
+    # The document's URLs are resolved at once, with the schema's `resolve_urls`, for each rendering.
     def view_template
+      @_prosemirror_urls = prosemirror_schema.resolve_urls(@document.urls) if prosemirror_schema.resolve_urls?
       render_content([@document.root])
     end
   end

data/lib/prosemirror/marks/basic/link.rb

@@ -5,7 +5,7 @@ module Prosemirror
     module Basic
       # ProseMirror's link. Without an allowed `href`, its text is rendered without a link.
       class Link < MarkType
-        prop :href, _String(length: 1..)
+        prop :href, _String(length: 1..), url: true
         prop :title, _String?
         parse_html(tag: "a[href]") { |element, schema| schema.allowed_url?(element["href"]) && {} }
 

data/lib/prosemirror/marks/link.rb

@@ -4,7 +4,7 @@ module Prosemirror
   module Marks
     # Tiptap's link. Without an allowed `href`, its text is rendered without a link.
     class Link < MarkType
-      prop :href, _String(length: 1..)
+      prop :href, _String(length: 1..), url: true
       prop :target, _String?
       prop :rel, _String?
       prop :css_class, _String?, json: "class"

data/lib/prosemirror/nodes/basic/image.rb

@@ -8,7 +8,7 @@ module Prosemirror
         group :inline
         inline
 
-        prop :src, _String(length: 1..)
+        prop :src, _String(length: 1..), url: true
         prop :alt, _String?
         prop :title, _String?
         parse_html tag: "img[src]"

data/lib/prosemirror/nodes/image.rb

@@ -6,7 +6,7 @@ module Prosemirror
     class Image < NodeType
       group :block
 
-      prop :src, _String(length: 1..)
+      prop :src, _String(length: 1..), url: true
       prop :alt, _String?
       prop :title, _String?
       prop :width, _Integer?(1..), &Coerce::INTEGER

data/lib/prosemirror/rendering.rb

@@ -14,9 +14,9 @@ module Prosemirror
       @_prosemirror_schema
     end
 
-    # The URL, when its scheme is allowed, after the schema's `resolve_url`. Otherwise nil.
+    # The URL, when its scheme is allowed, after the schema's `resolve_url` or `resolve_urls`. Otherwise nil.
     def safe_url(url)
-      prosemirror_schema.safe_url(url)
+      prosemirror_schema.safe_url(url, @_prosemirror_urls)
     end
 
     private
@@ -64,6 +64,7 @@ module Prosemirror
 
     def prepare(component)
       component.instance_variable_set(:@_prosemirror_schema, prosemirror_schema)
+      component.instance_variable_set(:@_prosemirror_urls, @_prosemirror_urls)
       component
     end
   end

data/lib/prosemirror/schema.rb

@@ -19,6 +19,9 @@ module Prosemirror
     end
     # Called with each URL before it is checked, to rewrite it.
     prop :resolve_url, _Callable?, reader: false
+    # Called with all the URLs of a document at once, before rendering it: returns a Hash from each URL to
+    # its rewritten URL, or nil. Instead of `resolve_url`, to look up records in one query.
+    prop :resolve_urls, _Callable?, reader: false, predicate: :public
     # Raise `InvalidDocument` instead of fixing invalid documents.
     prop :strict, _Boolean, default: false, reader: false, predicate: :public
     # Called with an `InvalidDocument` error for each problem fixed.
@@ -66,6 +69,7 @@ module Prosemirror
       @textblocks = @nodes.to_h { |type| [type, textblock_type?(type)] }
       @content_matches = @nodes.to_h { |type| [type, ContentMatch.compile(type.content, self)] }
       raise ArgumentError, "the schema has no #{@top_node} node type" unless node_type(@top_node)
+      raise ArgumentError, "resolve_url and resolve_urls can't be used together" if @resolve_url && @resolve_urls
     end
 
     # The start state of a node type's content expression.
@@ -174,6 +178,14 @@ module Prosemirror
 
     # Whether a URL's scheme is allowed, or the URL is relative. Browsers ignore leading control characters
     # and spaces, and tabs and newlines anywhere in a URL.
+    # The URLs resolved with `resolve_urls`, as a Hash, or nil without it.
+    def resolve_urls(urls)
+      return unless @resolve_urls
+      return {} if urls.empty?
+
+      @resolve_urls.call(urls).to_h
+    end
+
     def allowed_url?(url)
       return false unless url.is_a?(String)
 
@@ -194,11 +206,13 @@ module Prosemirror
       @textblocks.fetch(type, false)
     end
 
-    # The URL after `resolve_url`, when its scheme is allowed. Otherwise nil, and the problem is reported.
-    def safe_url(url)
+    # The URL after `resolve_url` or `resolve_urls`, when its scheme is allowed. Otherwise nil, and the
+    # problem is reported. `resolved_urls` holds URLs already resolved with `resolve_urls`: others are
+    # resolved one at a time.
+    def safe_url(url, resolved_urls = nil)
       return nil unless url.is_a?(String)
 
-      resolved = @resolve_url ? @resolve_url.call(url) : url
+      resolved = resolve(url, resolved_urls)
       return report("URL could not be resolved: #{url.inspect}") unless resolved.is_a?(String)
       return report("URL scheme not allowed: #{resolved.inspect}") unless allowed_url?(resolved)
 
@@ -227,6 +241,14 @@ module Prosemirror
 
     private
 
+    def resolve(url, resolved_urls)
+      return @resolve_url.call(url) if @resolve_url
+      return url unless @resolve_urls
+
+      resolved_urls = resolve_urls([url]) unless resolved_urls&.key?(url)
+      resolved_urls[url]
+    end
+
     def lookup(types)
       types.each_with_object({}) do |type, table|
         [type.type_name, *type.aliases].each { |name| table[name] ||= type }

data/lib/prosemirror/type_definition.rb

@@ -23,14 +23,16 @@ module Prosemirror
     # - `json:` the attribute name in JSON, when it differs from the property name (`"textAlign"`);
     # - `aliases:` other JSON names accepted on input (`["order"]`);
     # - `parse_html:` a block reading the attribute from an HTML element, when importing HTML. By default,
-    #   the HTML attribute with the JSON name is read.
-    def prop(name, type, kind = :keyword, json: nil, aliases: [], parse_html: nil, **, &)
+    #   the HTML attribute with the JSON name is read;
+    # - `url: true` for an attribute holding a URL, which `Document#urls` lists and `resolve_urls` resolves
+    #   before rendering.
+    def prop(name, type, kind = :keyword, json: nil, aliases: [], parse_html: nil, url: false, **, &)
       json = (json || name).to_s
       aliases = aliases.map(&:to_s)
       # The keys accepted in JSON: each name as written, in snake_case and in camelCase.
       keys = [json, *aliases].flat_map { |key| [key, Naming.snake(key), Naming.camel(key)] }.uniq.map(&:freeze).freeze
-      attribute_names[name] = {json:, aliases:, parse_html:, keys:}.freeze
-      @known_keys = nil
+      attribute_names[name] = {json:, aliases:, parse_html:, keys:, url:}.freeze
+      @known_keys = @url_attributes = nil
       super(name, type, kind, **, &)
     end
 
@@ -108,6 +110,11 @@ module Prosemirror
       @known_keys ||= attribute_names.each_value.flat_map { |names| names[:keys] }.to_set.freeze
     end
 
+    # The names of the attributes declared with `url: true`.
+    def url_attributes
+      @url_attributes ||= attribute_names.filter_map { |name, names| name if names[:url] }.freeze
+    end
+
     # Attributes as JSON, with their JSON names.
     def serialize_attrs(attrs)
       attrs.to_h { |name, value| [attribute_names.dig(name, :json) || name.to_s, value] }

data/lib/prosemirror/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Prosemirror
-  VERSION = "0.6.0"
+  VERSION = "0.7.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: prosemirror
 version: !ruby/object:Gem::Version
-  version: 0.6.0
+  version: 0.7.0
 platform: ruby
 authors:
 - Hans Lemuet
@@ -62,6 +62,7 @@ files:
 - CHANGELOG.md
 - COOKBOOK.md
 - LICENSE.txt
+- MIGRATING.md
 - README.md
 - ROADMAP.md
 - TYPES.md