roadie 3.2.2 → 3.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 5817fde135bcf056bf3f9ef1d5f28b99dd6362ca
-  data.tar.gz: 663c031f765c9a3aff26b247040ce184cd37f234
+  metadata.gz: 2a5d2f3eb66ab40ecf551f53e5b7df59321c2805
+  data.tar.gz: 82c0b5241461df56c8810dca075245a75a0acf99
 SHA512:
-  metadata.gz: 38cf12788dc10e050a87bee1c8ba68a5a21254ff4a049369b22735f39bf9bee08253f4045b7db78c224a425fd3ca7ed430caf7a43a17a84c5142e01d8c6724e5
-  data.tar.gz: 2b5b336462dc8c9a53d39137c563925764dead15242c3f8cc09313d8f5487e242d4ea231fe591a433f0b30b4033d9f96fdc469ef2ca03cc1adc4f52387fad9b3
+  metadata.gz: e8c1a2ed3e74d586b95e341472b4c80e18e58ae01a094955b7dc102cd2cfe7c76ec1172159e1a1138a09adea465ac17ca1b0fcbde1f42680c46252cffee977ec
+  data.tar.gz: 777552240c002247354dad9c1bcc6ac6fb78ab62774d7e4fb3fe2e1a7aa49f855a67f3586b0178d0b99455c5d0e9547d72d3a30f460e3f8ecec6cabd9f437125

data/.travis.yml

@@ -5,8 +5,12 @@ rvm:
   - 2.2
   - 2.3
   - 2.4
+  - 2.5
   - jruby
   - rbx
+before_install:
+  - gem update --system # Need for Ruby 2.5.0. https://github.com/travis-ci/travis-ci/issues/8978
+  - gem install bundler
 matrix:
   allow_failures:
     # Rubinius and JRuby have a lot of trouble and no large following, so I'm going to

data/Changelog.md

@@ -1,11 +1,25 @@
 ### dev
 
-[full changelog](https://github.com/Mange/roadie/compare/v3.2.2...master)
+[full changelog](https://github.com/Mange/roadie/compare/v3.3.0...master)
 
 * Nothing yet.
 
 ### 3.2.2
 
+[full changelog](https://github.com/Mange/roadie/compare/v3.2.2...v3.3.0)
+
+* Enhancements
+  * Allow transforming to XHTML instead of HTML - [Zhivko Draganov](https://github.com/zdraganov) (#144).
+  * Support partial HTML documents (fragments) - #147
+    * With the help of [andfx](https://github.com/andfx) - #115
+    * With the help of [Frida Sjoholm](https://github.com/andf://github.com/FridaSjoholm) - #146
+  * Skip URL rewriting on elements with `data-roadie-ignore` - #154.
+    * With the help of [Hamed Asghari](https://github.com/hasghari) - #138.
+* Bug fixes:
+  * Apply correct string encoding / charset in `NetHttpProvider` - [Jeremy Nagel](https://github.com/jeznag) (#152).
+
+### 3.2.2
+
 [full changelog](https://github.com/Mange/roadie/compare/v3.2.1...v3.2.2)
 
 * Enhancements

data/README.md

@@ -5,7 +5,6 @@ Roadie
 [![Code Climate](https://codeclimate.com/github/Mange/roadie.png)](https://codeclimate.com/github/Mange/roadie)
 [![Code coverage status](https://codecov.io/github/Mange/roadie/coverage.svg?branch=master)](https://codecov.io/github/Mange/roadie?branch=master)
 [![Gem Version](https://badge.fury.io/rb/roadie.png)](http://badge.fury.io/rb/roadie)
-[![Dependency Status](https://gemnasium.com/Mange/roadie.png)](https://gemnasium.com/Mange/roadie)
 
 **Note: This README details the 3.x version of Roadie. You might be using 2.x, which is much older and only for Rails.**
 
@@ -23,7 +22,11 @@ This gem makes this easier by automatically inlining stylesheets into the docume
 
 "Dynamic" selectors (`:hover`, `:visited`, `:focus`, etc.), or selectors not understood by Nokogiri will be inlined into a single `<style>` element for those email clients that support it. This changes specificity a great deal for these rules, so it might not work 100% out of the box. (See more about this below)
 
-Roadie also rewrites all relative URLs in the email to an absolute counterpart, making images you insert and those referenced in your stylesheets work. No more headaches about how to write the stylesheets while still having them work with emails from your acceptance environments.
+Roadie also rewrites all relative URLs in the email to an absolute counterpart,
+making images you insert and those referenced in your stylesheets work. No more
+headaches about how to write the stylesheets while still having them work with
+emails from your acceptance environments. You can disable this on specific
+elements using a `data-roadie-ignore` marker.
 
 Features
 --------
@@ -35,9 +38,12 @@ Features
   * Keeps `:hover` and friends around in a separate `<style>` element.
 * Makes image urls absolute.
   * Hostname and port configurable on a per-environment basis.
+  * Can be disabled on individual elements.
 * Makes link `href`s and `img` `src`s absolute.
 * Automatically adds proper HTML skeleton when missing; you don't have to create a layout for emails.
+  * Also supports HTML fragments / partial documents, where layout is not added.
 * Allows you to inject stylesheets in a number of ways, at runtime.
+* Removes `data-roadie-ignore` markers before finishing the HTML.
 
 Install & Usage
 ---------------
@@ -51,10 +57,17 @@ gem 'roadie', '~> 3.2'
 You can then create a new instance of a Roadie document:
 
 ```ruby
+# Transform full documents with the #transform method.
 document = Roadie::Document.new "<html><body></body></html>"
 document.add_css "body { color: green; }"
 document.transform
     # => "<html><body style=\"color:green;\"></body></html>"
+
+# Transform partial documents with #transform_partial.
+document = Roadie::Document.new "<div>Hello world!</div>"
+document.add_css "div { color: green; }"
+document.transform_partial
+    # => "<div style=\"color:green;\">Hello world!</div>"
 ```
 
 Your document instance can be configured with several options:
@@ -83,6 +96,14 @@ The following URLs will be rewritten for you:
 * `img[src]` (HTML)
 * `url()` (CSS)
 
+You can disable individual elements by adding an `data-roadie-ignore` marker on
+them. CSS will still be inlined on those elements, but URLs will not be
+rewritten.
+
+```html
+<a href="|UNSUBSCRIBE_URL|" data-roadie-ignore>Unsubscribe</a>
+```
+
 ### Referenced stylesheets ###
 
 By default, `style` and `link` elements in the email document's `head` are processed along with the stylesheets and removed from the `head`.
@@ -177,7 +198,9 @@ The `NetHttpProvider` will download the URLs that is is given using Ruby's stand
 You can give it a whitelist of hosts that downloads are allowed from:
 
 ```ruby
-document.external_asset_providers << Roadie::NetHttpProvider.new(whitelist: ["myapp.com", "assets.myapp.com", "cdn.cdnnetwork.co.jp"])
+document.external_asset_providers << Roadie::NetHttpProvider.new(
+  whitelist: ["myapp.com", "assets.myapp.com", "cdn.cdnnetwork.co.jp"],
+)
 document.external_asset_providers << Roadie::NetHttpProvider.new # Allows every host
 ```
 
@@ -331,11 +354,18 @@ end
 
 ### Keeping CSS that is impossible to inline
 
-Some CSS is impossible to inline properly. `:hover` and `::after` comes to mind. Roadie tries its best to keep these around by injecting them inside a new `<style>` element in the `<head>`.
+Some CSS is impossible to inline properly. `:hover` and `::after` comes to
+mind. Roadie tries its best to keep these around by injecting them inside a new
+`<style>` element in the `<head>` (or at the beginning of the partial if
+transforming a partial document).
 
-The problem here is that Roadie cannot possible adjust the specificity for you, so they will not apply the same way as they did before the styles were inlined.
+The problem here is that Roadie cannot possible adjust the specificity for you,
+so they will not apply the same way as they did before the styles were inlined.
 
-Another caveat is that a lot of email clients does not support this (which is the entire point of inlining in the first place), so don't put anything important in here. Always handle the case of these selectors not being part of the email.
+Another caveat is that a lot of email clients does not support this (which is
+the entire point of inlining in the first place), so don't put anything
+important in here. Always handle the case of these selectors not being part of
+the email.
 
 #### Specificity problems ####
 
@@ -374,10 +404,26 @@ class TrackNewsletterLinks
   end
 end
 
-document.before_transformation = proc { |dom, document| logger.debug "Inlining document with title #{dom.at_css('head > title').try(:text)}" }
+document.before_transformation = ->(dom, document) {
+  logger.debug "Inlining document with title #{dom.at_css('head > title').try(:text)}"
+}
 document.after_transformation = TrackNewsletterLinks.new
 ```
 
+### XHTML vs HTML ###
+
+You can configure the underlying HTML/XML engine to output XHTML or HTML (which
+is the default). One usecase for this is that `{` tokens usually gets escaped
+to `&#123;`, which would be a problem if you then pass the resulting HTML on to
+some other templating engine that uses those tokens (like Handlebars or Mustache).
+
+```ruby
+document.mode = :xhtml
+```
+
+This will also affect the emitted `<!DOCTYPE>` if transforming a full document.
+Partial documents does not have a `<!DOCTYPE>`.
+
 Build Status
 ------------
 
@@ -418,6 +464,40 @@ Instructions on how to do this on most platforms, see [Nokogiri's official insta
 
 The CSS Parser used in Roadie does not handle keyframes. I don't think any email clients do either, but if you want to keep on trying you can add them manually to a `<style>` element (or a separate referenced stylesheet) and [tell Roadie not to touch them](#referenced-stylesheets).
 
+### How do I get rid of the `<body>` elements that are added?
+
+It sounds like you want to transform a partial document. Maybe you are building
+partials or template fragments to later place in other documents. Use
+`Document#transform_partial` instead of `Document#transform` in order to treat
+the HTML as a partial document.
+
+### Can I skip URL rewriting on a specific element?
+
+If you add the `data-roadie-ignore` attribute on an element, URL rewriting will
+not be performed on that element. This could be really useful for you if you
+intend to send the email through some other rendering pipeline that replaces
+some placeholders/variables.
+
+```html
+<a href="/about-us">About us</a>
+<a href="|UNSUBSCRIBE_URL|" data-roadie-ignore>Unsubscribe</a>
+```
+
+Note that this will not skip CSS inlining on the element; it will still get the
+correct styles applied.
+
+### What should I do about "Invalid URL" errors?
+
+If the URL is invalid on purpose, see _Can I skip URL rewriting on a specific
+element?_ above. Otherwise, you can try to parse it yourself using Ruby's `URI`
+class and see if you can figure it out.
+
+```ruby
+require "uri"
+URI.parse("https://example.com/best image.jpg") # raises
+URI.parse("https://example.com/best%20image.jpg") # Works!
+```
+
 Documentation
 -------------
 
@@ -462,7 +542,7 @@ License
 
 (The MIT License)
 
-Copyright (c) 2009-2017 Magnus Bergmark, Jim Neath / Purify, and contributors.
+Copyright (c) 2009-2018 Magnus Bergmark, Jim Neath / Purify, and contributors.
 
 * [Magnus Bergmark](https://github.com/Mange) <magnus.bergmark@gmail.com>
 

data/lib/roadie/asset_scanner.rb

@@ -46,7 +46,6 @@ module Roadie
         element.remove if stylesheet
         stylesheet
       }.compact
-      remove_ignore_markers
       stylesheets
     end
 
@@ -102,11 +101,5 @@ module Roadie
 
       true
     end
-
-    def remove_ignore_markers
-      @dom.css("[data-roadie-ignore]").each do |node|
-        node.remove_attribute "data-roadie-ignore"
-      end
-    end
   end
 end

data/lib/roadie/document.rb

@@ -13,6 +13,8 @@ module Roadie
   # The internal stylesheet is used last and gets the highest priority. The
   # rest is used in the same order as browsers are supposed to use them.
   #
+  # The execution methods are {#transform} and {#transform_partial}.
+  #
   # @attr [#call] before_transformation Callback to call just before {#transform}ation begins. Will be called with the parsed DOM tree and the {Document} instance.
   # @attr [#call] after_transformation Callback to call just before {#transform}ation is completed. Will be called with the current DOM tree and the {Document} instance.
   class Document
@@ -27,6 +29,9 @@ module Roadie
     # Should CSS that cannot be inlined be kept in a new `<style>` element in `<head>`?
     attr_accessor :keep_uninlinable_css
 
+    # The mode to generate markup in. Valid values are `:html` (default) and `:xhtml`.
+    attr_reader :mode
+
     # @param [String] html the input HTML
     def initialize(html)
       @keep_uninlinable_css = true
@@ -34,6 +39,7 @@ module Roadie
       @asset_providers = ProviderList.wrap(FilesystemProvider.new)
       @external_asset_providers = ProviderList.empty
       @css = ""
+      @mode = :html
     end
 
     # Append additional CSS to the document's internal stylesheet.
@@ -42,18 +48,21 @@ module Roadie
       @css << "\n\n" << new_css
     end
 
-    # Transform the input HTML and returns the processed HTML.
+    # Transform the input HTML as a full document and returns the processed
+    # HTML.
     #
     # Before the transformation begins, the {#before_transformation} callback
     # will be called with the parsed HTML tree and the {Document} instance, and
     # after all work is complete the {#after_transformation} callback will be
     # invoked in the same way.
     #
-    # Most of the work is delegated to other classes. A list of them can be seen below.
+    # Most of the work is delegated to other classes. A list of them can be
+    # seen below.
     #
     # @see MarkupImprover MarkupImprover (improves the markup of the DOM)
     # @see Inliner Inliner (inlines the stylesheets)
     # @see UrlRewriter UrlRewriter (rewrites URLs and makes them absolute)
+    # @see #transform_partial Transforms partial documents (fragments)
     #
     # @return [String] the transformed HTML
     def transform
@@ -62,7 +71,41 @@ module Roadie
       callback before_transformation, dom
 
       improve dom
-      inline dom
+      inline dom, keep_uninlinable_in: :head
+      rewrite_urls dom
+
+      callback after_transformation, dom
+
+      remove_ignore_markers dom
+      serialize_document dom
+    end
+
+    # Transform the input HTML as a HTML fragment/partial and returns the
+    # processed HTML.
+    #
+    # Before the transformation begins, the {#before_transformation} callback
+    # will be called with the parsed HTML tree and the {Document} instance, and
+    # after all work is complete the {#after_transformation} callback will be
+    # invoked in the same way.
+    #
+    # The main difference between this and {#transform} is that this does not
+    # treat the HTML as a full document and does not try to fix it by adding
+    # doctypes, {<head>} elements, etc.
+    #
+    # Most of the work is delegated to other classes. A list of them can be
+    # seen below.
+    #
+    # @see Inliner Inliner (inlines the stylesheets)
+    # @see UrlRewriter UrlRewriter (rewrites URLs and makes them absolute)
+    # @see #transform Transforms full documents
+    #
+    # @return [String] the transformed HTML
+    def transform_partial
+      dom = Nokogiri::HTML.fragment html
+
+      callback before_transformation, dom
+
+      inline dom, keep_uninlinable_in: :root
       rewrite_urls dom
 
       callback after_transformation, dom
@@ -80,7 +123,23 @@ module Roadie
       @external_asset_providers = ProviderList.wrap(list)
     end
 
+    # Change the mode. The mode affects how the resulting markup is generated.
+    #
+    # Valid modes:
+    #   `:html` (default)
+    #   `:xhtml`
+    def mode=(mode)
+      if VALID_MODES.include?(mode)
+        @mode = mode
+      else
+        raise ArgumentError, "Invalid mode #{mode.inspect}. Valid modes are: #{VALID_MODES.inspect}"
+      end
+    end
+
     private
+    VALID_MODES = %i[html xhtml].freeze
+    private_constant :VALID_MODES
+
     def stylesheet
       Stylesheet.new "(Document styles)", @css
     end
@@ -89,9 +148,13 @@ module Roadie
       MarkupImprover.new(dom, html).improve
     end
 
-    def inline(dom)
+    def inline(dom, options = {})
+      keep_uninlinable_in = options.fetch(:keep_uninlinable_in)
       dom_stylesheets = AssetScanner.new(dom, asset_providers, external_asset_providers).extract_css
-      Inliner.new(dom_stylesheets + [stylesheet], dom).inline(keep_uninlinable_css)
+      Inliner.new(dom_stylesheets + [stylesheet], dom).inline(
+        keep_uninlinable_css: keep_uninlinable_css,
+        keep_uninlinable_in: keep_uninlinable_in,
+      )
     end
 
     def rewrite_urls(dom)
@@ -101,10 +164,17 @@ module Roadie
     def serialize_document(dom)
       # #dup is called since it fixed a few segfaults in certain versions of Nokogiri
       save_options = Nokogiri::XML::Node::SaveOptions
+      format = {
+        html: save_options::AS_HTML,
+        xhtml: save_options::AS_XHTML,
+      }.fetch(mode)
+
       dom.dup.to_html(
         save_with: (
-          save_options::NO_DECLARATION | save_options::NO_EMPTY_TAGS | save_options::AS_HTML
-        )
+          save_options::NO_DECLARATION |
+          save_options::NO_EMPTY_TAGS |
+          format
+        ),
       )
     end
 
@@ -127,5 +197,11 @@ module Roadie
         end
       end
     end
+
+    def remove_ignore_markers(dom)
+      dom.css("[data-roadie-ignore]").each do |node|
+        node.remove_attribute "data-roadie-ignore"
+      end
+    end
   end
 end

data/lib/roadie/inliner.rb

@@ -24,13 +24,20 @@ module Roadie
 
     # Start the inlining, mutating the DOM tree.
     #
-    # @param [true, false] keep_extra_blocks
+    # @option options [true, false] :keep_uninlinable_css
+    # @option options [:root, :head] :keep_uninlinable_in
     # @return [nil]
-    def inline(keep_extra_blocks = true)
+    def inline(options = {})
+      keep_uninlinable_css = options.fetch(:keep_uninlinable_css, true)
+      keep_uninlinable_in = options.fetch(:keep_uninlinable_in, :head)
+
       style_map, extra_blocks = consume_stylesheets
 
       apply_style_map(style_map)
-      add_styles_to_head(extra_blocks) if keep_extra_blocks
+
+      if keep_uninlinable_css
+        add_uninlinable_styles(keep_uninlinable_in, extra_blocks)
+      end
 
       nil
     end
@@ -89,21 +96,31 @@ module Roadie
       nil
     end
 
-    def add_styles_to_head(blocks)
-      unless blocks.empty?
-        create_style_element(blocks, find_head)
-      end
+    def add_uninlinable_styles(parent, blocks)
+      return if blocks.empty?
+
+      parent_node =
+        case parent
+        when :head
+          find_head
+        when :root
+          dom
+        else
+          raise ArgumentError, "Parent must be either :head or :root. Was #{parent.inspect}"
+        end
+
+      create_style_element(blocks, parent_node)
     end
 
     def find_head
       dom.at_xpath('html/head')
     end
 
-    def create_style_element(style_blocks, head)
-      return unless head
-      element = Nokogiri::XML::Node.new("style", head.document)
+    def create_style_element(style_blocks, parent)
+      return unless parent
+      element = Nokogiri::XML::Node.new("style", parent.document)
       element.content = style_blocks.join("\n")
-      head.add_child(element)
+      parent.add_child(element)
     end
 
     # @api private