toml 0.0.1 → 0.0.2

data/Gemfile

@@ -0,0 +1,8 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in toml.gemspec
+gemspec
+
+group :test do
+  gem 'minitest'
+end

data/Gemfile.lock

@@ -0,0 +1,20 @@
+PATH
+  remote: .
+  specs:
+    toml (0.0.1)
+      parslet
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    blankslate (2.1.2.4)
+    minitest (4.6.1)
+    parslet (1.5.0)
+      blankslate (~> 2.0)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  minitest
+  toml!

data/README.md

@@ -1,69 +1,35 @@
-RakeGem
-=======
+# TOML
 
-# DESCRIPTION
+A sane configuration format from @mojombo.  More information here: https://github.com/mojombo/toml
 
-Ever wanted to manage your RubyGem in a sane way without having to resort to
-external dependencies like Jeweler or Hoe? Ever thought that Rake and a hand
-crafted gemspec should be enough to deal with these problems? If so, then
-RakeGem is here to make your life awesome!
+## Usage
 
-RakeGem is not a library. It is just a few simple file templates that you can
-copy into your project and easily customize to match your specific needs. It
-ships with a few Rake tasks to help you keep your gemspec up-to-date, build
-a gem, and release your library and gem to the world.
+It's simple, really.
 
-RakeGem assumes you are using Git. This makes the Rake tasks easy to write. If
-you are using something else, you should be able to get RakeGem up and running
-with your system without too much editing.
+```ruby
+content = <<-TOML
 
-The RakeGem tasks were inspired by the
-[Sinatra](http://github.com/sinatra/sinatra) project.
+# Hello, this is an example.
+[things]
+other = "things"
+what = 900000
 
-# INSTALLATION
+TOML
 
-Take a look at `Rakefile` and `NAME.gemspec`. For new projects, you can start
-with these files and edit a few lines to make them fit into your library. If
-you have an existing project, you'll probably want to take the RakeGem
-versions and copy any custom stuff from your existing Rakefile and gemspec
-into them. As long as you're careful, the rake tasks should keep working.
+parser = TOML::Parser.new(content).parsed
+# => { "things" => { "other" => "things", "what" => 900000 } }
+```
 
-# ASSUMPTIONS
+You can also use the same API as `YAML` if you'd like:
 
-RakeGem makes a few assumptions. You will either need to satisfy these
-assumptions or modify the rake tasks to work with your setup.
+```ruby
+TOML.load("thing = 9")
+# => {"thing" => 9}
 
-You should have a file named `lib/NAME.rb` (where NAME is the name of your
-library) that contains a version line. It should look something like this:
+TOML.load_file("my_file.toml")
+# => {"whatever" => "keys"}
+```
 
-    module NAME
-      VERSION = '0.1.0'
-    end
+## Contributors
 
-It is important that you use the constant `VERSION` and that it appear on a
-line by itself.
-
-# UPDATING THE VERSION
-
-In order to make a new release, you'll want to update the version. With
-RakeGem, you only need to do that in the `lib/NAME.rb` file. Everything else
-will use this find the canonical version of the library.
-
-# TASKS
-
-RakeGem provides three rake tasks:
-
-`rake gemspec` will update your gemspec with the latest version (taken from
-the `lib/NAME.rb` file) and file list (as reported by `git ls-files`).
-
-`rake build` will update your gemspec, build your gemspec into a gem, and
-place it in the `pkg` directory.
-
-`rake release` will update your gemspec, build your gem, make a commit with
-the message `Release 0.1.0` (with the correct version, obviously), tag the
-commit with `v0.1.0` (again with the correct version), and push the `master`
-branch and new tag to `origin`.
-
-Keep in mind that these are just simple Rake tasks and you can edit them
-however you please. Don't want to auto-commit or auto-push? Just delete those
-lines. You can bend RakeGem to your own needs. That's the whole point!
+Written by Jeremy McAnally (@jm) and Dirk Gadsden (@dirk) based on TOML from Tom Preston-Werner (@mojombo).

data/lib/toml.rb

@@ -1,8 +1,22 @@
 $:.unshift(File.dirname(__FILE__))
 
 require 'time'
+require 'parslet'
+
+require 'toml/key'
+require 'toml/key_group'
+require 'toml/parslet'
+require 'toml/transformer'
 require 'toml/parser'
 
 module TOML
-  VERSION = '0.0.1'
+  VERSION = '0.0.2'
+
+  def self.load(content)
+    Parser.new(content).parsed
+  end
+
+  def self.load_file(path)
+    Parser.new(File.read(path)).parsed
+  end
 end

data/lib/toml/key.rb

@@ -0,0 +1,9 @@
+module TOML
+  class Key
+    attr_reader :key, :value
+    def initialize(key, value)
+      @key = key
+      @value = value
+    end
+  end
+end

data/lib/toml/key_group.rb

@@ -0,0 +1,9 @@
+module TOML
+  class KeyGroup
+    attr_reader :keys
+    
+    def initialize(keys)
+      @keys = keys
+    end
+  end
+end

data/lib/toml/parser.rb

@@ -1,67 +1,40 @@
-module TOML
+module TOML  
   class Parser
     attr_reader :parsed
 
     def initialize(markup)
-      lines = markup.split("\n").reject {|l| l.start_with?("#") }
+      # Make sure we have a newline on the end
+      markup += "\n" unless markup.end_with?("\n")
 
+      tree = Parslet.new.parse(markup)
+      parts = Transformer.new.apply(tree)
+      
       @parsed = {}
-      @current_key_group = ""
-
-      lines.each do |line|
-        if line.gsub(/\s/, '').empty?
-          close_key_group
-        elsif line =~ /^\s?\[(.*)\]/
-          new_key_group($1)
-        elsif line =~ /\s?(.*)=(.*)/
-          add_key($1, $2)
+      @current = @parsed
+      
+      parts.each do |part|
+        if part.is_a? Key
+          @current[part.key] = part.value
+        elsif part.is_a? KeyGroup
+          resolve_key_group(part)
         else
-          raise "lmao i have no clue what you're doing: #{line}"
+          raise "Unrecognized part: #{part.inspect}"
         end
       end
     end
-
-    def new_key_group(key_name)
-      @current_key_group = key_name
-    end
-
-    def add_key(key, value)
-      @parsed[@current_key_group] ||= {}
-      @parsed[@current_key_group][key.strip] = coerce(strip_comments(value))
-    end
-
-    def strip_comments(text)
-      text.split("#").first
-    end
-
-    def coerce(value)
-      value = value.strip
-      if value =~ /\[(.*)\]/
-        # array
-        array = $1.split(",").map {|s| s.strip.gsub(/\"(.*)\"/, '\1')}
-        return array
-      elsif value =~ /\"(.*)\"/
-        # string
-        return $1
-      end
-
-      begin
-        time = Time.parse(value)
-        return time
-      rescue
-      end
-
-      begin
-        int = Integer(value)
-        return int
-      rescue
+    
+    def resolve_key_group(kg)
+      @current = @parsed
+
+      path = kg.keys.dup
+      while k = path.shift
+        if @current.has_key? k
+          # pass
+        else
+          @current[k] = {}
+        end
+        @current = @current[k]
       end
-
-      raise "lol no clue what [#{value}] is"
-    end
-
-    def close_key_group
-      @current_key_group = ""
     end
   end
-end
+end

data/lib/toml/parslet.rb

@@ -0,0 +1,67 @@
+module TOML
+  class Parslet < ::Parslet::Parser
+    rule(:document) { (key_group | key_value | comment_line).repeat(0) }
+    root :document
+
+    rule(:value) {
+      array.as(:array) |
+      string |
+      datetime.as(:datetime) |
+      float.as(:float) |
+      integer.as(:integer) |
+      boolean
+    }
+
+    rule(:array) {
+      str("[") >> (
+        all_space >> value >>
+        (all_space >> str(",") >> all_space >> value).repeat(0) >>
+        all_space
+      ).maybe >> str("]") 
+    }
+    
+    rule(:key_value) { space >> key.as(:key) >> space >> str("=") >> space >> value >> space >> comment.maybe >> str("\n") >> all_space }
+    rule(:key_group) { space >> str("[") >> key_group_name.as(:key_group) >> str("]") >> space >> comment.maybe >> str("\n") >> all_space }
+    
+    rule(:key) { match("[^. \t\\]]").repeat(1) }
+    rule(:key_group_name) { key.as(:key) >> (str(".") >> key.as(:key)).repeat(0) }
+
+    rule(:comment_line) { comment >> str("\n") >> all_space }
+    rule(:comment) { str("#") >> match("[^\n]").repeat(0) }
+
+    rule(:space) { match("[ \t]").repeat(0) }
+    rule(:all_space) { match("[ \t\r\n]").repeat(0) }
+        
+    rule(:string) {
+      str('"') >> (
+      match("[^\"\\\\]") |
+      (str("\\") >> match("[0tnr\"\\\\]"))
+      ).repeat(0).as(:string) >> str('"')
+    }
+    
+    rule(:sign) { str("-") }
+    rule(:integer) {
+      str("0") | (sign.maybe >> match("[1-9]") >> match("[0-9]").repeat(0))
+    }
+    
+    rule(:float) {
+      sign.maybe >> match("[0-9]").repeat(1) >> str(".") >> match("[0-9]").repeat(1)
+    }
+
+    rule(:boolean) { str("true").as(:true) | str("false").as(:false) }
+    
+    rule(:date) {
+      match("[0-9]").repeat(4,4) >> str("-") >>
+      match("[0-9]").repeat(2,2) >> str("-") >>
+      match("[0-9]").repeat(2,2)
+    }
+
+    rule(:time) {
+      match("[0-9]").repeat(2,2) >> str(":") >>
+      match("[0-9]").repeat(2,2) >> str(":") >>
+      match("[0-9]").repeat(2,2)
+    }
+
+    rule(:datetime) { date >> str("T") >> time >> str("Z") }
+  end
+end

data/lib/toml/transformer.rb

@@ -0,0 +1,74 @@
+module TOML
+  class Transformer < ::Parslet::Transform
+    # Utility to properly handle escape sequences in parsed string.
+    def self.parse_string(val)
+      e = val.length
+      s = 0
+      o = []
+      while s < e
+        if val[s] == "\\"
+          s += 1
+          case val[s]
+          when "t"
+            o << "\t"
+          when "n"
+            o << "\n"
+          when "\\"
+            o << "\\"
+          when '"'
+            o << '"'
+          when "r"
+            o << "\r"
+          when "0"
+            o << "\0"
+          else
+            raise "Unexpected escape character: '\\#{val[s]}'"
+          end
+        else
+          o << val[s]
+        end
+        s += 1
+      end
+      o.join
+    end
+    
+    # Clean up arrays
+    rule(:array => subtree(:ar)) { ar.is_a?(Array) ? ar : [ar] }
+
+    # Clean up simples (inside arrays)
+    rule(:integer => simple(:i)) { i.to_i }
+    rule(:float => simple(:f)) { f.to_f }
+    rule(:string => simple(:s)) {
+      Transformer.parse_string(s.to_s)
+    }
+    rule(:datetime => simple(:d)) { DateTime.iso8601(d) }
+    rule(:true => simple(:b)) { true }
+    rule(:false => simple(:b)) { false }
+    
+    # TODO: Refactor to remove redundancy
+    rule(:key => simple(:k), :array => subtree(:ar)) { Key.new(k.to_s, ar) }
+    rule(:key => simple(:k), :integer => simple(:i)) { Key.new(k.to_s, i.to_i) }
+    rule(:key => simple(:k), :float => simple(:f)) { Key.new(k.to_s, f.to_f) }
+    rule(:key => simple(:k), :string => simple(:s)) {
+      Key.new(k.to_s, Transformer.parse_string(s.to_s))
+    }
+    rule(:key => simple(:k), :datetime => simple(:d)) {
+      Key.new(k.to_s, DateTime.iso8601(d))
+    }
+    rule(:key => simple(:k), :true => simple(:b)) { Key.new(k.to_s, true) }
+    rule(:key => simple(:k), :false => simple(:b)) { Key.new(k.to_s, false) }
+    
+    # Make keys just be strings
+    rule(:key => simple(:k)) { k }
+
+    # Then objectify the key_groups
+    rule(:key_group => simple(:kg)) {
+      KeyGroup.new([kg.to_s])
+    }
+
+    # Captures array-like key-groups
+    rule(:key_group => subtree(:kg)) {
+      KeyGroup.new(kg.map &:to_s)
+    }
+  end
+end

data/test/spec.toml

@@ -0,0 +1,43 @@
+# Comment
+
+# String
+string = "string\n\t\"string"
+
+# Integer
+integer = 42
+
+# Float
+pi = 3.14159
+
+# Booleans
+true = true
+false = false
+
+# DateTime
+datetime = 1979-05-27T07:32:00Z
+
+# Keygroup
+[a.b.c]
+d = "test"
+
+[e]
+f = "test"
+
+# Post line comment
+[comments]
+on = "a line" # with markup
+
+# Multi-line arrays
+[arrays]
+# Simple array
+simple = [1, 2, 3]
+
+# Nested array
+nested = [[[1], 2], 3]
+
+multi = ["lines", "are",
+"super", "cool", "lol",
+"amirite"]
+
+# Uneven spacing
+uneven = [1, 2, 3,  4, 5 ]

data/test/test_parser.rb

@@ -0,0 +1,50 @@
+
+require 'rubygems'
+require 'bundler/setup'
+
+require 'toml'
+require 'minitest/autorun'
+
+class TestParser < MiniTest::Unit::TestCase
+  def setup
+    filepath = File.join(File.dirname(__FILE__), 'spec.toml')
+    @doc = TOML::Parser.new(File.read(filepath)).parsed
+  end
+  
+  def test_string
+    assert_equal @doc["string"], "string\n\t\"string"
+  end
+
+  def test_integer
+    assert_equal @doc["integer"], 42
+  end
+
+  def test_float
+    assert_equal @doc["pi"], 3.14159
+  end
+
+  def test_datetime
+    assert_equal @doc["datetime"], DateTime.iso8601("1979-05-27T07:32:00Z")
+  end
+
+  def test_booleans
+    assert_equal @doc["true"], true
+    assert_equal @doc["false"], false
+  end
+
+  def test_simple_array
+    assert_equal @doc["arrays"]["simple"], [1, 2, 3]
+  end
+
+  def test_nested_array
+    assert_equal @doc["arrays"]["nested"], [[[1], 2], 3]
+  end
+
+  def test_simple_keygroup
+    assert_equal @doc["e"]["f"], "test"
+  end
+
+  def test_nested_keygroup
+    assert_equal @doc["a"]["b"]["c"]["d"], "test"
+  end
+end

data/toml.gemspec

@@ -13,8 +13,8 @@ Gem::Specification.new do |s|
   ## If your rubyforge_project name is different, then edit it and comment out
   ## the sub! line in the Rakefile
   s.name              = 'toml'
-  s.version           = '0.0.1'
-  s.date              = '2013-02-23'
+  s.version           = '0.0.2'
+  s.date              = '2013-02-24'
 
   ## Make sure your summary is short. The description may be as long
   ## as you like.
@@ -24,7 +24,7 @@ Gem::Specification.new do |s|
   ## List the primary authors. If there are a bunch of authors, it's probably
   ## better to set the email to an email list or something. If you don't have
   ## a custom homepage, consider using your GitHub URL or the like.
-  s.authors  = ["Jeremy McAnally"]
+  s.authors  = ["Jeremy McAnally", "Dirk Gadsden"]
   s.email    = 'jeremy@github.com'
   s.homepage = 'http://github.com/jm/toml'
 
@@ -36,17 +36,27 @@ Gem::Specification.new do |s|
   ## LICENSE files to the extra_rdoc_files list.
   s.rdoc_options = ["--charset=UTF-8"]
   s.extra_rdoc_files = %w[README.md LICENSE]
-
+  
+  s.add_dependency 'parslet'
+  
   ## Leave this section as-is. It will be automatically generated from the
   ## contents of your Git repository via the gemspec task. DO NOT REMOVE
   ## THE MANIFEST COMMENTS, they are used as delimiters by the task.
   # = MANIFEST =
   s.files = %w[
+    Gemfile
+    Gemfile.lock
     LICENSE
     README.md
     Rakefile
     lib/toml.rb
+    lib/toml/key.rb
+    lib/toml/key_group.rb
     lib/toml/parser.rb
+    lib/toml/parslet.rb
+    lib/toml/transformer.rb
+    test/spec.toml
+    test/test_parser.rb
     toml.gemspec
   ]
   # = MANIFEST =

metadata

@@ -1,16 +1,33 @@
 --- !ruby/object:Gem::Specification
 name: toml
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.0.2
   prerelease: 
 platform: ruby
 authors:
 - Jeremy McAnally
+- Dirk Gadsden
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-02-23 00:00:00.000000000 Z
-dependencies: []
+date: 2013-02-24 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: parslet
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 description: Parse your TOML, seriously.
 email: jeremy@github.com
 executables: []
@@ -19,11 +36,19 @@ extra_rdoc_files:
 - README.md
 - LICENSE
 files:
+- Gemfile
+- Gemfile.lock
 - LICENSE
 - README.md
 - Rakefile
 - lib/toml.rb
+- lib/toml/key.rb
+- lib/toml/key_group.rb
 - lib/toml/parser.rb
+- lib/toml/parslet.rb
+- lib/toml/transformer.rb
+- test/spec.toml
+- test/test_parser.rb
 - toml.gemspec
 homepage: http://github.com/jm/toml
 licenses: []
@@ -50,4 +75,5 @@ rubygems_version: 1.8.23
 signing_key: 
 specification_version: 2
 summary: Parse your TOML.
-test_files: []
+test_files:
+- test/test_parser.rb