brianmario-yajl-ruby 0.3.4 → 0.4.0

data/.gitignore

@@ -0,0 +1,4 @@
+Makefile
+*.o
+*.dylib
+*.bundle

data/CHANGELOG.rdoc

@@ -1,5 +1,18 @@
 = Changelog
 
+0.4.0 (?)
+ * NOTE: Breaking API change:
+ ** refactored Stream parsing methods out of Yajl::Native into Yajl::Stream
+ ** removed Yajl::Native namespace/module
+ * Addition of Yajl::HttpStream module
+ ** This module is for streaming JSON HTTP responses directly into Yajl (as they're being received) for increased awesomeness
+ ** it currently supports basic get requests with Yajl::HttpStream.get(uri)
+ ** it also supports (and prefers) output compressed (gzip) responses
+ * Addition Yajl::Chunked module
+ ** This module is for feeding Yajl JSON pieces at a time, instead of an entire IO object
+ ** This works very well in environments like an EventMachine app where data is received in chunks by design
+ * decreased read buffer for Yajl::Stream from 8kb to 4kb
+
 0.3.4 (April 24th, 2009)
  * turned Unicode checks back on in the Yajl parser now that it's fixed (thanks Lloyd!)
  ** this also bumps the yajl version dependency requirement to 1.0.4

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2008-2009 Brian Lopez - http://github.com/brianmario
+ 
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+ 
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+ 
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -26,11 +26,36 @@ or maybe STDIN
 
  cat someJsonFile.json | ruby -ryajl -e "puts Yajl::Native.parse(STDIN).inspect"
 
+
+Or lets say you didn't have access to the IO object that contained JSON data, but instead
+only had access to chunks of it at a time. No problem!
+
+(Assume we're in an EventMachine::Connection instance)
+
+ def object_parsed(obj)
+    puts "Sometimes one pays most for the things one gets for nothing. - Albert Einstein"
+    puts obj.inspect
+  end
+  
+ def connection_completed
+   # once a full JSON object has been parsed from the stream
+   # object_parsed will be called, and passed the constructed object
+   Yajl::Chunked.on_parse_complete = method(:object_parsed)
+ end
+ 
+ def receive_data(data)
+   # continue passing chunks
+   Yajl::Chunked.parse_some(data)
+   
+   # Or as an alias, you could have done:
+   # Yajl::Chunked << data
+ end
+
 There are a lot more possibilities, some of which I'm going to write other gems/plugins for.
 
 Some ideas are:
 * parsing logs in JSON format
-* an  ActiveSupport patch (http://github.com/technoweenie/rails/commit/f9abcc93a3660fcf46172c4cd8a2f36ba5a5510d)
+* an ActiveSupport patch (http://github.com/technoweenie/rails/commit/f9abcc93a3660fcf46172c4cd8a2f36ba5a5510d)
 * Rack middleware (ideally the JSON body could be handed to the parser while it's still being received)
 * use with ohai
 * JSON API clients
@@ -40,7 +65,7 @@ Some ideas are:
 
 First, Yajl uses CMake to build itself (yes, the author realizes this isn't the norm for open source and is willing and ready to accept patches, fork away kids!) so you'll need to grab it first from http://www.cmake.org.
 
-After you've got that, grab the latest version of Yajl itself from the Githubs at http://github.com/lloyd/yajl.
+After you've got that, grab the latest version (I suggest at least 1.0.4 as it contains fixes for Unicode parsing) of Yajl itself from the Githubs at http://github.com/lloyd/yajl.
 
 After you have that installed, you should be able to install it like any other gem hosted here like so:
 
@@ -79,4 +104,15 @@ Here's a list of things I need to get done to improve the awesomeness of this li
 * Write up a test suite 
 * Fix known parsing issues with certain JSON strings (Yajl itself works fine, it's the logic in this library that needs a little tweaking)
 * Write more docs
-* Change API so it can accept a String or an IO
+* Change API so it can accept a String or an IO
+
+== Special Thanks
+
+I've had a lot of inspiration, and a lot of help. Thanks to everyone who's been a part of this and those to come!
+
+* Lloyd Hilaiel (http://github.com/lloyd) - for writing Yajl!!
+* Josh Ferguson (http://github.com/besquared) - for peer-pressuring me into getting back into C; it worked ;)
+* Jonathan Novak (http://github.com/cypriss) - C hacking help
+* Tom Smith (http://github.com/rtomsmith) - C hacking help
+* Rick (http://github.com/technoweenie) - for making an ActiveSupport patch with support for this library and teasing me that it might go into Rails 3. You sure lit a fire under my ass and I got a ton of work done because of it! :)
+* The entire Github Crew - my inspiration, time spent writing this, finding Yajl, So many-MANY other things wouldn't have been possible without this awesome service. I owe you guys some whiskey at Kilowatt.

data/Rakefile

@@ -1,3 +1,4 @@
+# encoding: UTF-8
 begin
   require 'jeweler'
   Jeweler::Tasks.new do |gem|
@@ -6,18 +7,9 @@ begin
     gem.email = "seniorlopez@gmail.com"
     gem.homepage = "http://github.com/brianmario/yajl-ruby"
     gem.authors = ["Brian Lopez"]
-    gem.require_paths = ["ext"]
-    gem.extra_rdoc_files = [
-      "README.rdoc",
-      "CHANGELOG.rdoc"
-    ]
-    gem.files = [
-      "CHANGELOG.rdoc",
-      "README.rdoc",
-      "Rakefile",
-      "VERSION.yml",
-      "ext/yajl.c"
-    ]
+    gem.require_paths = ["ext", "lib"]
+    gem.extra_rdoc_files = `git ls-files *.rdoc`.split("\n")
+    gem.files = `git ls-files`.split("\n")
     gem.extensions = ["ext/extconf.rb"]
     gem.files.include %w(lib/jeweler/templates/.document lib/jeweler/templates/.gitignore)
     # gem.rubyforge_project = "yajl-ruby"

data/VERSION.yml

@@ -1,4 +1,4 @@
 --- 
-:patch: 4
+:patch: 0
 :major: 0
-:minor: 3
+:minor: 4

data/benchmark/http.rb

@@ -0,0 +1,28 @@
+# encoding: UTF-8
+require 'rubygems'
+require 'benchmark'
+require 'yajl'
+require 'yajl/http_stream'
+require 'json'
+require 'activesupport'
+require 'uri'
+require 'net/http'
+
+uri = URI.parse('http://search.twitter.com/search.json?q=github')
+
+times = ARGV[0] ? ARGV[0].to_i : 1
+puts "Starting benchmark parsing #{uri.to_s} #{times} times\n\n"
+Benchmark.bm { |x|
+  x.report {
+    puts "Yajl::HttpStream.get"
+    times.times {
+      Yajl::HttpStream.get(uri)
+    }
+  }
+  x.report {
+    puts "JSON.parser"
+    times.times {
+      JSON.parse(Net::HTTP.get_response(uri).body, :max_nesting => false)
+    }
+  }
+}

data/benchmark/stream.rb

@@ -0,0 +1,40 @@
+# encoding: UTF-8
+require 'rubygems'
+require 'benchmark'
+require 'yajl'
+require 'json'
+require 'activesupport'
+
+filename = ARGV[0] || 'subjects/twitter_search.json'
+json = File.new(filename, 'r')
+
+# warm up the filesystem
+json.read
+json.rewind
+
+times = ARGV[1] ? ARGV[1].to_i : 1
+puts "Starting benchmark parsing #{File.size(filename)} bytes of JSON data #{times} times\n\n"
+Benchmark.bm { |x|
+  x.report {
+    puts "Yajl::Native.parse (C)"
+    times.times {
+      json.rewind
+      Yajl::Stream.parse(json)
+    }
+  }
+  x.report {
+    puts "JSON.parser"
+    times.times {
+      json.rewind
+      JSON.parse(json.read, :max_nesting => false)
+    }
+  }
+  x.report {
+    puts "ActiveSupport::JSON.decode"
+    times.times {
+      json.rewind
+      ActiveSupport::JSON.decode(json.read)
+    }
+  }
+}
+json.close