crass 0.0.2 → 0.1.0

checksums.yaml

@@ -1,15 +1,15 @@
 ---
 !binary "U0hBMQ==":
   metadata.gz: !binary |-
-    Y2NmZWM1MTg3MjVmNjcxN2ZjN2JhZmE4YzU0NjUxYWFjMDI0MjQwMA==
+    OGNiZTc1MDI4ZWMwYmQxZGI4ODE4Yzc1YzNkNTU2MDJjODc1NTI0Mg==
   data.tar.gz: !binary |-
-    NmZmMGJmM2JkYmRiY2JhYjI2N2E1NzVhNDFjNzc3ZjJiZDViNzg3ZQ==
+    NTdlYmJkNTZlYjI3ZjE3Njc2ZDRiMzkxYWZjOTQ0OWE3ZTcxYWRiOQ==
 SHA512:
   metadata.gz: !binary |-
-    ZjRkYzI5Zjk1NGZlOTEzOGI2YTRmMThhMTdiYzNjNDYyNGU4NzhiMjExYTY4
-    NDQ4ZDJlYjlhYTBiNTE0OTcwNTk4ZTgxM2VjNGNkMmVkNjgwYzcwNmZiZGRh
-    MDM5OWE4YzM3NWIwZjhkN2VhODVlZmJkMDYzMGJjMjgyM2NkNjY=
+    ZjA0NTZkOTAwYTYwZmNjMDgyMzE1N2NkZDQ5ZTlkNDg0NzZmYTk4OGNiMjli
+    MDdlNzQwNTAwZWZmMjkzY2U3N2NkMjEyOWU1ZDMyZWZlYjU1NGRjNzg1Y2Ez
+    ZWZiYjg5ZTk2YmFhNWQ4NDk1ZDEzNmUwYjg2NDE0NjkyNzFmMjY=
   data.tar.gz: !binary |-
-    ZmMyOWZiNWNkYmQ5OGE3MGFkOWEyMWJiY2JmODYwM2Y0MTRhNzhhZWM1NGE0
-    NTU2NTI1MGYyNzVlZDE3ZTZmYWQ4NjkyMmY2NWU4NDA4NGNlNDAyOWIxYWY4
-    NmEzNmVkYjYyNTNhMjkwODFkMDU2MmMzOGJkNGZlM2RlYmMwY2E=
+    NTE5NzI2MDViOWU5YzE4NzFmOWNlZmU0MWM5NTJhNGIyZTMyMTBkOGVjNWQ5
+    MmQ5NDZhYWY5YjIzMTgyNzNjNzZmY2I1NDE3YzFmMzM2Njk1NGQzMzQ3OTkw
+    MjFmODllYmM5NDQ5MTk3YjAwN2I3M2NmNzU4NDZkYzQwYzE1NWE=

data/.gitignore

@@ -1,4 +1,5 @@
 .yardoc/
 doc/
+pkg/
 .DS_Store
 Gemfile.lock

data/HISTORY.md

@@ -1,6 +1,34 @@
 Crass Change History
 ====================
 
+0.1.0 (2013-10-04)
+------------------
+
+* Tokenization is a little over 50% faster.
+
+* Added tons of unit tests.
+
+* Added `Crass.parse_properties` and `Crass::Parser.parse_properties`, which can
+  be used to parse the contents of an HTML element's `style` attribute.
+
+* Added `Crass::Parser.parse_rules`, which can be used to parse the contents of
+  an `:at_rule` block like `@media` that may contain style rules.
+
+* Fixed: `Crass::Parser#consume_at_rule` and `#consume_qualified_rule` didn't
+  properly handle already-parsed `:simple_block` nodes in the input, which
+  occurs when parsing rules in the value of an `:at_rule` block.
+
+* Fixed: On `:property` nodes, `:important` is now set to `true` when the
+  property is followed by an "!important" declaration.
+
+* Fixed: "!important" is no longer included in the value of a `:property` node.
+
+* Fixed: A variety of tokenization bugs uncovered by tests.
+
+* Fixed: Added a workaround for a possible spec bug when an `:at_keyword` is
+  encountered while consuming declarations.
+
+
 0.0.2 (2013-09-30)
 ------------------
 
@@ -11,4 +39,3 @@ Crass Change History
 ------------------
 
 * Initial release.
-

data/README.md

@@ -1,20 +1,22 @@
 Crass
 =====
 
-Crass is a Ruby CSS parser based on the [CSS Syntax Module Level 3][css] draft.
+Crass is a Ruby CSS parser based on the [CSS Syntax Level 3][css] draft
+specification.
 
 * [Home](https://github.com/rgrove/crass/)
 * [API Docs](http://rubydoc.info/github/rgrove/crass/master)
 
 [![Build Status](https://travis-ci.org/rgrove/crass.png?branch=master)](https://travis-ci.org/rgrove/crass?branch=master)
+[![Gem Version](https://badge.fury.io/rb/crass.png)](http://badge.fury.io/rb/crass)
 
 Features
 --------
 
 * Pure Ruby, with no runtime dependencies other than Ruby 1.9.x or higher.
 
-* Tokenizes and parses CSS according to the rules defined in the
-  [CSS Syntax Module Level 3][css] draft.
+* Tokenizes and parses CSS according to the rules defined in the 2013 draft of
+  the [CSS Syntax Level 3][css] specification.
 
 * Extremely tolerant of broken or invalid CSS. If a browser can handle it, Crass
   should be able to handle it too.
@@ -32,7 +34,9 @@ Features
 Problems
 --------
 
-* It's pretty slow.
+* Crass isn't terribly fast. I mean, it's Ruby, and it's not really slow by Ruby
+  standards. But compared to the CSS parser in your average browser? Yeah, it's
+  slow.
 
 * Crass only parses the CSS syntax; it doesn't understand what any of it means,
   doesn't coalesce selectors, etc. You can do this yourself by consuming the
@@ -43,9 +47,10 @@ Problems
   (except for wholesale removal of nodes) are not reflected in the serialized
   output.
 
-* Unit tests aren't complete yet.
+* At the moment, Crass only supports UTF-8 input and doesn't respect `@charset`
+  rules. Input in any other encoding will be converted to UTF-8.
 
-* Probably tons of other things. Did I mention it's very new and experimental?
+* Probably other things. Did I mention Crass is pretty new?
 
 Installing
 ----------
@@ -54,9 +59,6 @@ Installing
 gem install crass
 ```
 
-...but only if you're brave. Seriously, this thing will almost certainly kill
-your family and poop on your pets.
-
 Examples
 --------
 
@@ -95,6 +97,7 @@ This returns a big fat ugly parse tree, which looks like this:
     {:node=>:property,
      :name=>"color",
      :value=>"#0d8bfa",
+     :important=>false,
      :tokens=>
       [{:node=>:ident, :pos=>27, :raw=>"color", :value=>"color"},
        {:node=>:colon, :pos=>32, :raw=>":"},
@@ -109,6 +112,7 @@ This returns a big fat ugly parse tree, which looks like this:
     {:node=>:property,
      :name=>"text-decoration",
      :value=>"underline",
+     :important=>false,
      :tokens=>
       [{:node=>:ident,
         :pos=>45,
@@ -168,6 +172,23 @@ hate to have to turn down a pull request you spent a lot of time on.
 
 [issue]: https://github.com/rgrove/crass/issues/new
 
+Acknowledgments
+---------------
+
+I'm deeply, deeply grateful to [Simon Sapin][simon] for his wonderfully
+comprehensive [CSS parsing tests][css-tests], which I adapted to create many of
+Crass's tests. They've been invaluable in helping me fix bugs and handle weird
+edge cases, and Crass would be much crappier without them.
+
+I'm also grateful to [Tab Atkins Jr.][tab] and Simon Sapin (again!) for their
+work on the [CSS Syntax Level 3][spec] specification, which defines the
+tokenizing and parsing rules that Crass implements.
+
+[css-tests]:https://github.com/SimonSapin/css-parsing-tests/
+[simon]:http://exyr.org/about/
+[spec]:http://www.w3.org/TR/css-syntax-3/
+[tab]:http://www.xanthir.com/contact/
+
 License
 -------
 

data/crass.gemspec

@@ -3,8 +3,8 @@ require './lib/crass/version'
 
 Gem::Specification.new do |s|
   s.name        = 'crass'
-  s.summary     = 'CSS parser based on the CSS Syntax Module Level 3 draft.'
-  s.description = 'Crass is a pure Ruby CSS parser based on the CSS Syntax Module Level 3 draft.'
+  s.summary     = 'CSS parser based on the CSS Syntax Level 3 draft.'
+  s.description = 'Crass is a pure Ruby CSS parser based on the CSS Syntax Level 3 draft.'
   s.version     = Crass::VERSION
   s.authors     = ['Ryan Grove']
   s.email       = ['ryan@wonko.com']

data/lib/crass.rb

@@ -11,4 +11,12 @@ module Crass
     Parser.parse_stylesheet(input, options)
   end
 
+  # Parses _input_ as a string of CSS properties (such as the contents of an
+  # HTML element's `style` attribute) and returns a parse tree.
+  #
+  # See {Tokenizer#initialize} for _options_.
+  def self.parse_properties(input, options = {})
+    Parser.parse_properties(input, options)
+  end
+
 end

data/lib/crass/parser.rb

@@ -16,6 +16,36 @@ module Crass
 
     # -- Class Methods ---------------------------------------------------------
 
+    # Parses CSS properties (such as the contents of an HTML element's `style`
+    # attribute) and returns a parse tree.
+    #
+    # See {Tokenizer#initialize} for _options_.
+    #
+    # http://www.w3.org/TR/2013/WD-css-syntax-3-20130919/#parse-a-list-of-declarations
+    def self.parse_properties(input, options = {})
+      Parser.new(input, options).parse_properties
+    end
+
+    # Parses a CSS rules (such as the content of a `@media` block) and returns a
+    # parse tree. The only difference from {#parse_stylesheet} is that CDO/CDC
+    # nodes (`<!--` and `-->`) aren't ignored.
+    #
+    # See {Tokenizer#initialize} for _options_.
+    #
+    # http://www.w3.org/TR/2013/WD-css-syntax-3-20130919/#parse-a-list-of-rules
+    def self.parse_rules(input, options = {})
+      parser = Parser.new(input, options)
+      rules  = parser.consume_rules
+
+      rules.map do |rule|
+        if rule[:node] == :qualified_rule
+          parser.create_style_rule(rule)
+        else
+          rule
+        end
+      end
+    end
+
     # Parses a CSS stylesheet and returns a parse tree.
     #
     # See {Tokenizer#initialize} for _options_.
@@ -26,10 +56,10 @@ module Crass
       rules  = parser.consume_rules(:top_level => true)
 
       rules.map do |rule|
-        case rule[:node]
-        # TODO: handle at-rules
-        when :qualified_rule then parser.create_style_rule(rule)
-        else rule
+        if rule[:node] == :qualified_rule
+          parser.create_style_rule(rule)
+        else
+          rule
         end
       end
     end
@@ -46,20 +76,32 @@ module Crass
       string = ''
 
       nodes.each do |node|
+        next if node.nil?
+
         case node[:node]
+        when :at_rule
+          string << node[:tokens].first[:raw]
+          string << self.stringify(node[:prelude], options)
+
+          if node[:block]
+            string << self.stringify(node[:block], options)
+          end
+
         when :comment
           string << node[:raw] unless options[:exclude_comments]
 
-        when :style_rule
-          string << self.stringify(node[:selector][:tokens], options)
-          string << "{"
-          string << self.stringify(node[:children], options)
-          string << "}"
-
         when :property
-          string << options[:indent] if options[:indent]
           string << self.stringify(node[:tokens], options)
 
+        when :simple_block
+          string << node[:start]
+          string << self.stringify(node[:value], options)
+          string << node[:end]
+
+        when :style_rule
+          string << self.stringify(node[:selector][:tokens], options)
+          string << "{#{self.stringify(node[:children], options)}}"
+
         else
           if node.key?(:raw)
             string << node[:raw]
@@ -74,7 +116,7 @@ module Crass
 
     # -- Instance Methods ------------------------------------------------------
 
-    # Array of tokens generated from this parser's input.
+    # {TokenScanner} wrapping the tokens generated from this parser's input.
     attr_reader :tokens
 
     # Initializes a parser based on the given _input_, which may be a CSS string
@@ -96,7 +138,7 @@ module Crass
       rule = {}
 
       rule[:tokens] = input.collect do
-        rule[:name]    = parse_value(input.consume)
+        rule[:name]    = input.consume[:value]
         rule[:prelude] = []
 
         while token = input.consume
@@ -108,12 +150,14 @@ module Crass
             rule[:block] = consume_simple_block(input)
             break
 
-          # TODO: At this point, the spec says we should check for a "simple
-          # block with an associated token of <<{-token>>", but isn't that
-          # exactly what we just did above? And the tokenizer only ever produces
-          # standalone <<{-token>>s, so how could the token stream ever contain
-          # one that's already associated with a simple block? What am I
-          # missing?
+          when :simple_block
+            if token[:start] == '{'
+              rule[:block] = token
+              break
+            else
+              input.reconsume
+              rule[:prelude] << consume_component_value(input)
+            end
 
           else
             input.reconsume
@@ -140,34 +184,48 @@ module Crass
 
     # Consumes a declaration and returns it, or `nil` on parse error.
     #
-    # http://www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-declaration0
+    # http://www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-declaration
     def consume_declaration(input = @tokens)
       declaration = {}
+      value       = []
 
       declaration[:tokens] = input.collect do
         declaration[:name] = input.consume[:value]
 
-        value = []
         token = input.consume
-        token = input.consume while token[:node] == :whitespace
-
-        return nil if token[:node] != :colon # TODO: parse error
+        token = input.consume while token && token[:node] == :whitespace
 
+        return nil if !token || token[:node] != :colon # TODO: parse error
         value << token while token = input.consume
-        declaration[:value] = value
+      end
 
-        maybe_important = value.reject {|v| v[:node] == :whitespace }[-2, 2]
+      # Look for !important.
+      pos = -1
+      while token = value[pos]
+        type = token[:node]
+
+        if type == :whitespace || type == :comment || type == :semicolon
+          pos -= 1
+          next
+        end
 
-        if maybe_important &&
-            maybe_important[0][:node] == :delim &&
-            maybe_important[0][:value] == '!' &&
-            maybe_important[1][:node] == :ident &&
-            maybe_important[1][:value].downcase == 'important'
+        if type == :ident && token[:value].downcase == 'important'
+          prev_token = value[pos - 1]
 
-          declaration[:important] = true
+          if prev_token && prev_token[:node] == :delim &&
+              prev_token[:value] == '!'
+
+            declaration[:important] = true
+            value.slice!(pos - 1, 2)
+          else
+            break
+          end
+        else
+          break
         end
       end
 
+      declaration[:value] = value
       create_node(:declaration, declaration)
     end
 
@@ -176,7 +234,7 @@ module Crass
     # NOTE: The returned list may include `:comment`, `:semicolon`, and
     # `:whitespace` nodes, which is non-standard.
     #
-    # http://www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-list-of-declarations0
+    # http://www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-list-of-declarations
     def consume_declarations(input = @tokens)
       declarations = []
 
@@ -189,8 +247,13 @@ module Crass
           # TODO: this is technically a parse error when parsing a style rule,
           # but not necessarily at other times.
 
-          # TODO: It seems like we should reconsume the current token here,
-          # since that's what happens when consuming a list of rules.
+          # Note: The spec doesn't say we should reconsume here, but it's
+          # necessary since `consume_at_rule` must consume the `:at_keyword` as
+          # the rule's name or it'll end up in the prelude. The spec *does* say
+          # we should reconsume when an `:at_keyword` is encountered in
+          # `consume_rules`, so we either have to reconsume in both places or in
+          # neither place. I've chosen to reconsume in both places.
+          input.reconsume
           declarations << consume_at_rule(input)
 
         when :ident
@@ -247,7 +310,7 @@ module Crass
     # Consumes a qualified rule and returns it, or `nil` if a parse error
     # occurs.
     #
-    # http://www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-qualified-rule0
+    # http://www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-qualified-rule
     def consume_qualified_rule(input = @tokens)
       rule = {:prelude => []}
 
@@ -258,15 +321,9 @@ module Crass
           if token[:node] == :'{'
             rule[:block] = consume_simple_block(input)
             break
-
-          # elsif [simple block with an associated <<{-token>>??]
-
-          # TODO: At this point, the spec says we should check for a "simple block
-          # with an associated token of <<{-token>>", but isn't that exactly what
-          # we just did above? And the tokenizer only ever produces standalone
-          # <<{-token>>s, so how could the token stream ever contain one that's
-          # already associated with a simple block? What am I missing?
-
+          elsif token[:node] == :simple_block
+            rule[:block] = token
+            break
           else
             input.reconsume
             rule[:prelude] << consume_component_value(input)
@@ -279,7 +336,7 @@ module Crass
 
     # Consumes a list of rules and returns them.
     #
-    # http://www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-list-of-rules0
+    # http://www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-list-of-rules
     def consume_rules(flags = {})
       rules = []
 
@@ -358,33 +415,41 @@ module Crass
     # * http://www.w3.org/TR/2013/WD-css-syntax-3-20130919/#style-rules
     # * http://www.w3.org/TR/2013/WD-css-syntax-3-20130919/#consume-a-list-of-declarations0
     def create_style_rule(rule)
-      children = []
-      tokens   = TokenScanner.new(rule[:block][:value])
+      create_node(:style_rule,
+        :selector => create_selector(rule[:prelude]),
+        :children => parse_properties(rule[:block][:value]))
+    end
 
-      consume_declarations(tokens).each do |decl|
+    # Parses a list of declarations and returns an array of `:property` nodes
+    # (and any non-declaration nodes that were in the input). This is useful for
+    # parsing the contents of an HTML element's `style` attribute.
+    #
+    # http://www.w3.org/TR/2013/WD-css-syntax-3-20130919/#parse-a-list-of-declarations
+    def parse_properties(input = @tokens)
+      input      = TokenScanner.new(input) unless input.is_a?(TokenScanner)
+      properties = []
+
+      consume_declarations(input).each do |decl|
         unless decl[:node] == :declaration
-          children << decl
+          properties << decl
           next
         end
 
-        children << create_node(:property,
-          :name   => decl[:name],
-          :value  => parse_value(decl[:value]),
-          :tokens => decl[:tokens])
+        properties << create_node(:property,
+          :name      => decl[:name],
+          :value     => parse_value(decl[:value]),
+          :important => decl[:important] == true,
+          :tokens    => decl[:tokens])
       end
 
-      create_node(:style_rule,
-        :selector => create_selector(rule[:prelude]),
-        :children => children
-      )
+      properties
     end
 
     # Returns the unescaped value of a selector name or property declaration.
     def parse_value(nodes)
+      nodes  = [nodes] unless nodes.is_a?(Array)
       string = ''
 
-      nodes = [nodes] unless nodes.is_a?(Array)
-
       nodes.each do |node|
         case node[:node]
         when :comment, :semicolon then next