benofsky-yajl-ruby 0.7.6

data/.gitignore

@@ -0,0 +1,9 @@
+Makefile
+benchmark/subjects/contacts.*
+*.rbc
+*.o
+*.dylib
+*.bundle
+TODO.txt
+tmp/*
+pkg/*

data/CHANGELOG.md

@@ -0,0 +1,281 @@
+# Changelog
+
+## 0.7.6 (May 1st, 2010)
+* use memcmp instead of strcmp for invalid Fixnum check
+* add a spec to verify unicode chars can be used as keys
+* twitter examples updated
+* only use -Wextra if ENV['DEBUG'] is set as gcc 3 doesn't know about it
+* fix chunked http encoding parse logic to further comply with the spec (thanks to Sebastian Cohnen <sebastian.cohnen@gmx.net>)
+* removed as_json checks and usage in encoder to prevent infinite loops
+** In Rails a lot of objects return self from the as_json method - which is wrong IMO - and needs to be fixed before this feature will work properly
+
+## 0.7.5 (March 23rd, 2010)
+* check for existence of and use as_json method on custom objects
+* bugfix with read buffer when parsing from an IO (thanks to Pavel Valodzka <pavel@valodzka.name>)
+* merged in latest yajl
+* enable -Wextra during compilation
+* brought back ability to pass a buffer to bzip/gzip/deflate #read helper methods
+
+## 0.7.4 (March 3rd, 2010)
+* bugfix for the JSON gem compatibility API's Object#to_json method improperly encoding strings
+
+## 0.7.3 (February 23rd, 2010)
+* remove trap from HttpStream code, it's really not needed
+
+## 0.7.2 (February 23rd, 2010)
+* fixed broken to_json compatibility
+* removed strlen in a few places in favor of RSTRING_LEN since ruby already knows the length of the string
+* patched Yajl to more efficiently reset it's lexer (no more malloc/free)
+* removed dependency on IO#eof? when parsing from an IO for full Rack-spec compatibility
+* removed some various cruft code in C
+
+## 0.7.1 (February 17th, 2010)
+* revert a patch made to bundled Yajl enabling optional quoting of strings that broke binary API compatibility
+
+## 0.7.0 (February 5th, 2010)
+* ensure utf8 encoding is set on relevant strings during parse/encode in 1.9
+
+## 0.6.9 (January 26th, 2010)
+* HttpStream patches merged in from Luke Redpath <contact@lukeredpath.co.uk>
+* Changed how Yajl::Parser was calling IO#read to better conform to the Rack spec and thus can be used to directly parse a rack.input stream
+
+## 0.6.8 (January 1st, 2010)
+* A couple of small performance patches
+* Allow passing a string to Yajl::HttpStream methods instead of only a URI
+
+## 0.6.7 (December 4th, 2009)
+* Bump internal version constant to the proper value (doh!)
+* Bring over latest from Yajl upstream
+
+## 0.6.6 (December 1st, 2009)
+* Brought over some optimizations from Macruby's use of some yajl-ruby codez
+* Yajl::HttpStream now supports being killed for long-running requests, thanks to Filipe Giusti <filipegiusti@gmail.com>
+
+## 0.6.5 (November 13th, 2009)
+* optimize symbol creation while symbolize_keys is turned on
+* fix for 32bit integer conversion into ruby
+
+## 0.6.4 (November 4th, 2009)
+* All specs pass on Rubinius :)
+* Added Yajl.load and Yajl.dump for compatibility with other various data format API's in ruby
+* Fixed a bug in Yajl::Encoder which allowed direct, unescaped encoding of NaN, Infinity and -Infinity.
+  It will now properly throw a Yajl::EncodeError exception if either of these values are found unescaped.
+* Update bundled Yajl library to 1.0.7
+* Conditionally define RSTRING_* and RARRAY_* for older versions of ruby (1.8.5 is still the default on CentOS)
+* Bugfix for JSON gem exception classes to more accurately match those of the actual JSON gem
+* A few small speed optimizations
+* Updated specs to not run bzip2 related examples if unable to load the bzip2 library
+* Finally added UTF-8 checking specs
+* Removed needless calls to ID2SYM all over the place
+* Updated benchmark scripts to bring the GC into the picture a little more
+
+## 0.6.3 (August 25th, 2009)
+* Fixed a bug in the JSON gem compatibility API where strings weren't being properly escaped
+
+## 0.6.2 (August 25th, 2009)
+* Fixed a bug surfaced by an existing library providing a to_json method, and Yajl would double-quote the values provided
+
+## 0.6.1 (August 20th, 2009)
+* Fixed a bug in Yajl::HttpStream where responses contained multiple JSON strings but weren't Transfer-Encoding: chunked (thanks @dacort!)
+
+## 0.6.0 (August 19th, 2009)
+* Added POST, PUT and DELETE support to Yajl::HttpStream
+** POST support initially contributed by jdg (http://github.com/jdg) - Although oortle (http://github.com/oortle) coded it up in a fork with it as well.
+
+## 0.5.12 (July 31st, 2009)
+* Add another option that can be passed to Yajl::Encoder's constructor (:terminator) to allow the caller some control over
+  when a full JSON string has been generated by the encoder. More information on it's use in the README
+
+## 0.5.11 (July 14th, 2009)
+* fixing a bug Aman found with to_json on non-primitive Ruby objects and double-quoting in the JSON compat API
+
+## 0.5.10 (July 13th, 2009)
+* Bugfix for the JSON gem compatibility API's default Object#to_json helper
+
+## 0.5.9 (July 9th, 2009)
+* Bugfix for Yajl::Encoder where encoding a hash like {:a => :b} would get stuck in an infinite loop
+
+## 0.5.8 (July 6th, 2009)
+* Bugfix in Yajl::HttpStream for proper handling of the Content-type header (Rob Sharp)
+* Yajl::Encoder now has an on_progress callback setter, which can be used to harness the encoder's streaming ability.
+** The passed Proc/lambda will be called, and passed every chunk (currently 8kb) of the encoded JSON string as it's being encoded.
+* API CHANGE WARNING: Yajl::Encoder.encode's block will now be used as (and work the same as) the on_progress callback
+** This means the block will be passed chunks of the JSON string at a time, giving the caller the ability to start processing the encoded data while it's still being encoded.
+* fixed grammatical error in README (Neil Berkman)
+* Added some encoder examples
+
+## 0.5.7 (June 23rd, 2009)
+* You can now pass parser options (like :symbolize_keys for example) to Yajl::HttpStream.get
+* Refactored spec tests a bit, DRYing up the Yajl::HttpStream specs quite a bit.
+* Added a spec rake task, and spec.opts file
+* Updated and renamed rcov rake task, and added rcov.opts file
+
+## 0.5.6 (June 19th, 2009)
+* Added JSON.default_options hash to the JSON gem compatibility API
+* Split out the JSON gem compatibility API's parsing and encoding methods into individually includable files
+** the use case here is if you *only* want parsing, or *only* want encoding
+** also, if you don't include encoding it won't include the #to_json overrides which tend to cause problems in some environments.
+* Removed some large benchmark test files to reduce the size of the packaged gem by 1.5MB!
+
+## 0.5.5 (June 17th, 2009)
+* Introduction of the JSON gem compatibility API
+** NOTE: this isn't a 1:1 compatibility API, the goal was to be compatible with as many of the projects using the JSON gem as possible - not the JSON gem API itself
+** the compatibility API must be explicitly enabled by requiring 'yajl/json_gem' in your project
+** JSON.parse, JSON.generate, and the #to_json instance method extension to ruby's primitive classes are all included
+* Fix Yajl::Encoder to ensure map keys are strings
+* Encoding multiple JSON objects to a single stream doesn't separate by a newline character anymore
+* Yajl::Encoder now checks for the existence of, and will call #to_json on any non-primitive object
+
+## 0.5.4 (June 16th, 2009)
+* Yajl::Parser's :symbolize_keys option now defaults to false
+* remove use of sprintf for a little speed improvement while parsing
+
+## 0.5.3 (June 7th, 2009)
+* The IO parameter for Yajl::Encode#encode is now optional, and accepts a block
+** it will return the resulting JSON string if no IO is passed to stream to
+** if a block is passed, it will call and pass it the resulting JSON string
+* Yajl::Parser#parse can now parse from a String as well as an IO
+* Added and updated lot of in-code documentation.
+** all the C code exposed to Ruby should now have comments
+* Added :symbolize_keys option to the Yajl::Parser class, which defaults to true.
+** Having this option enabled has shown around an 18% speedup in parsing time according to my benchmarks
+
+## 0.5.2 (May 30th, 2009)
+* Added class helper methods Yajl::Encoder.encode(obj, io) and Yajl::Parser.parse(io)
+ * added tests for the above
+* Updated Twitter streaming example to have a less verbose output
+* Patch Yajl so encoding can continue as a stream
+ * IE: multiple objects encoded onto the same IO
+ * added a test for the above
+* Set the internal read buffer size back down to 8kb by default
+* Added an internal write buffer size (set to 8kb by default) which is used to throttle writes to the output stream
+ * This is to fix a major performance bug/issue with the IO#write C method in ruby 1.9.x (I've opened a bug with them about it)
+* Fixed a typo in a one-off parsing spec test
+* Updated benchmarks to work properly in 1.9 (required removal ActiveSupport benchmarking for now)
+* Updated spec tests to respect ridiculous differences in hash key ordering between 1.8 and 1.9
+
+
+## 0.5.1 (May 25th, 2009)
+* added some more tests for the new API
+* inlined a couple of hot functions used in parsing for a little speedup
+* updates to readme, reflecting changes in API
+* version bump to push another gem build
+
+## 0.5.0 (May 25th, 2009)
+* Refactored internal API so the caller can specify initialization options for the Parser and Encoder respectively. Two new classes were introduced as a result - Yajl::Parser and Yajl::Encoder. The newly refactored codebase is cleaner, thread-safe and removed all of the hack-code that was trickled around to make things work in the previous implementation. She's much more seaworthy now cap'n!
+ * Yajl::Parser.new accepts two options, :allow_comments and :check_utf8 which both default to true
+ * Yajl::Encoder.new accepts two options, :pretty and :indent which default to false and "  " respectively
+* cleaned up a lot of state code, that to my knowledge prevented yajl-ruby from being used in a thread-safe environment.
+* added deprecated messaging to Yajl::Stream.parse and Yajl::Stream.encode - these will likely go away before 0.6.0
+* fixed a bug in the chunked http response parser regarding partially received chunks
+* added a Twitter Search API example showing off the HttpStream API
+
+## 0.4.9 (May 20th, 2009)
+* fixed some parser state bugs surfaced by edge cases
+* added support for Chunked HTTP response bodies in Yajl::HttpStream
+ * added support for passing a block to Yajl::HttpStream.get that will be used as a callback whenever a JSON object is parsed off the stream (even if there is more than one!)
+* added an examples folder, and put an example using the Twitter Streaming API in there to start
+* added some more spec tests, this time around Chunked parsing and continuously parsing multiple JSON strings
+
+## 0.4.8 (May 18th, 2009)
+* fixed a totally bone-head compilation problem, I created for myself ;)
+
+## 0.4.7 (May 18th, 2009)
+* Bundling Yajl sources to remove the need to install them (and CMake) separately (Thank you Lloyd!!!) This means you can now simply install the gem and be off and running
+* Added some spec tests for Yajl::HttpStream
+* Added some spec tests for Yajl::Stream.encode
+* added some more thank you's, where credit's due - in the readme
+* updated the unicode.json file to reflect a "real-life" JSON response
+* reorganized spec tests into their functional areas
+* added an rcov rake task to generate code coverage output
+
+## 0.4.6 (May 17th, 2009)
+* Applied a patch from benburkert (http://github.com/benburkert) to fix HTTP Basic Auth in Yajl::HttpStream.get
+
+## 0.4.5 (May 17th, 2009)
+* added Yajl::Stream.encode(hash, io)
+ * generates a JSON string stream, and writes to IO
+ * compressed StreamWriter helpers added as well
+* fixed a pretty lame segfault in (x86_64 only?) ubuntu/linux
+* changed the compiled extension to have a more specific name (yajl_ext) for easier loading
+* removed forced-load of .bundle file, for the rest of the planet aside from OSX users
+* added some more benchmarks to compare to other forms of serialization in Ruby
+* various readme updates
+
+## 0.4.4 (May 12th, 2009)
+* NOTE: Breaking API change:
+ * renamed Yajl::GzipStreamReader to Yajl::Gzip::StreamReader
+* added Yajl::Bzip2::StreamReader
+ * depends on the bzip2-ruby gem if you want to use it, if not Yajl::Bzip2 won't be loaded
+* added Yajl::Deflate::StreamReader
+ * actually uses Zlib::Inflate for stream decompression
+* added parse(io) class methods to Yajl::Gzip::StreamReader and Yajl::Bzip2::StreamReader as a helper for parsing compressed streams.
+* updated Yajl::HttpStream to request responses compressed as deflate and bzip2 in addition to gzip
+* fixed a bug regarding parsing Integers as Floats (so 123456 would have be parsed and returned as 123456.0)
+* fixed a bug which caused a segfault in ruby's GC during string replacement in Yajl::Gzip and Yajl::Bzip2's StreamReader#read methods
+* added support for user-specified User-Agent strings in Yajl::HttpStream
+
+## 0.4.3 (May 2nd, 2009)
+* adding text/plain as an allowed mime-type for Yajl::HttpStream for webservers that respond with it instead of application/json (ahem...Yelp...)
+* renamed specs folder to spec for no reason at all
+
+## 0.4.2 (April 30th, 2009)
+* Yajl::HttpStream is now sending "proper" http request headers
+* Yajl::HttpStream will request HTTP-Basic auth if credentials are provided in the passed URI
+* cleanup requires
+
+## 0.4.1 (April 30th, 2009)
+* fixed a typo in the stream.rb benchmark file
+* fixed a bug in Yajl::Stream.parse that was causing "strange" Ruby malloc errors on large files, with large strings
+* added Yajl::GzipStreamReader as a wrapper around Zlib::GzipReader to allow for standard IO#read behavior
+ * this allows Yajl::Stream to read off of a Gzip stream directly
+
+## 0.4.0 (April 29th, 2009)
+* NOTE: Breaking API change:
+ * refactored Stream parsing methods out of Yajl::Native into Yajl::Stream
+ * removed Yajl::Native namespace/module
+* Addition of Yajl::HttpStream module
+ * This module is for streaming JSON HTTP responses directly into Yajl (as they're being received) for increased awesomeness
+ * it currently supports basic get requests with Yajl::HttpStream.get(uri)
+ * it also supports (and prefers) output compressed (gzip) responses
+* Addition Yajl::Chunked module
+ * This module is for feeding Yajl JSON pieces at a time, instead of an entire IO object
+ * This works very well in environments like an EventMachine app where data is received in chunks by design
+* decreased read buffer for Yajl::Stream from 8kb to 4kb
+
+## 0.3.4 (April 24th, 2009)
+* turned Unicode checks back on in the Yajl parser now that it's fixed (thanks Lloyd!)
+ * this also bumps the yajl version dependency requirement to 1.0.4
+* better guessing of Integer/Float from number found instead of just trying to create a BigNum no matter what
+* changed extconf.rb to fail Makefile creation if yajl isn't found
+* added a test to check for parsing Infinity due to a Float overflow
+
+## 0.3.3 (April 24th, 2009)
+* 1.9 compatibility
+
+## 0.3.2 (April 24th, 2009)
+* version bump: forgot to include yajl.c in the gem
+
+## 0.3.1 (April 23rd, 2009)
+* fixed borked gemspec
+
+## 0.3.0 (April 23rd, 2009)
+* slight refactor of ActiveSupport tests to better reflect how they actually exist in ActiveSupport
+* typo correction in the changelog which had the years in 2008
+* added some initial spec tests
+ * ported some from ActiveSupport to ensure proper compatibility
+ * included 57 JSON fixtures to test against, all of which pass
+* changed parser config to not check for invalid unicode characters as Ruby is going to do this anyway (?). This resolves the remaining test failures around unicode.
+* changed how the parser was dealing with numbers to prevent overflows
+* added an exception class Yajl::ParseError which is now used in place of simply printing to STDERR upon a parsing error
+* renamed a couple of JSON test files in the benchmark folder to better represent their contents
+* misc README updates
+
+## 0.2.1 (April 23rd, 2009)
+* fixed parsing bug - also fixed failing ActiveSupport test failures (except for the unicode one, which is an issue in Yajl itself)
+
+## 0.2.0 (April 22nd, 2009)
+* updated gemspec and README
+
+## 0.1.0 (April 21st, 2009)
+* initial release - gemified

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2008-2010 Brian Lopez - http://github.com/brianmario
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,320 @@
+= YAJL C Bindings for Ruby
+
+This gem is a C binding to the excellent YAJL JSON parsing and generation library.
+
+You can read more info at the project's website http://lloyd.github.com/yajl or check out its code at http://github.com/lloyd/yajl.
+
+== Features
+
+* JSON parsing and encoding directly to and from an IO stream (file, socket, etc) or String. Compressed stream parsing and encoding supported for Bzip2, Gzip and Deflate.
+* Parse and encode *multiple* JSON objects to and from streams or strings continuously.
+* JSON gem compatibility API - allows yajl-ruby to be used as a drop-in replacement for the JSON gem
+* Basic HTTP client (only GET requests supported for now) which parses JSON directly off the response body *as it's being received*
+* ~3.5x faster than JSON.generate
+* ~1.9x faster than JSON.parse
+* ~4.5x faster than YAML.load
+* ~377.5x faster than YAML.dump
+* ~1.5x faster than Marshal.load
+* ~2x faster than Marshal.dump
+
+== How to install
+
+First make sure you've got Gemcutter in your sources list:
+
+ gem sources -a http://gemcutter.org
+
+Then go ahead and install it as usual:
+
+ sudo gem install yajl-ruby
+
+== Example of use
+
+NOTE: I'm building up a collection of small examples in the examples (http://github.com/brianmario/yajl-ruby/tree/master/examples) folder.
+
+First, you're probably gonna want to require it:
+
+ require 'yajl'
+
+=== Parsing
+
+Then maybe parse some JSON from:
+
+a File IO
+
+ json = File.new('test.json', 'r')
+ parser = Yajl::Parser.new
+ hash = parser.parse(json)
+
+or maybe a StringIO
+
+ json = StringIO.new("...some JSON...")
+ parser = Yajl::Parser.new
+ hash = parser.parse(json)
+
+or maybe STDIN
+
+ cat someJsonFile.json | ruby -ryajl -e "puts Yajl::Parser.parse(STDIN).inspect"
+
+
+Or lets say you didn't have access to the IO object that contained JSON data, but instead
+only had access to chunks of it at a time. No problem!
+
+(Assume we're in an EventMachine::Connection instance)
+
+ def post_init
+    @parser = Yajl::Parser.new(:symbolize_keys => true)
+ end
+
+ def object_parsed(obj)
+    puts "Sometimes one pays most for the things one gets for nothing. - Albert Einstein"
+    puts obj.inspect
+  end
+
+ def connection_completed
+   # once a full JSON object has been parsed from the stream
+   # object_parsed will be called, and passed the constructed object
+   @parser.on_parse_complete = method(:object_parsed)
+ end
+
+ def receive_data(data)
+   # continue passing chunks
+   @parser << data
+ end
+
+Or if you don't need to stream it, it'll just return the built object from the parse when it's done.
+ NOTE: if there are going to be multiple JSON strings in the input, you *must* specify a block or callback as this
+ is how yajl-ruby will hand you (the caller) each object as it's parsed off the input.
+
+ obj = Yajl::Parser.parse(str_or_io)
+
+Or how about a JSON API HTTP request?
+This actually makes a request using a raw TCPSocket, then parses the JSON body right off the socket. While it's being received over the wire!
+
+ require 'uri'
+ require 'yajl/http_stream'
+
+ url = URI.parse("http://search.twitter.com/search.json?q=engineyard")
+ results = Yajl::HttpStream.get(url)
+
+Or do the same request, with Gzip and Deflate output compression support (also supports Bzip2, if loaded):
+(this does the same raw socket Request, but transparently parses the compressed response body)
+
+ require 'uri'
+ require 'yajl/gzip'
+ require 'yajl/deflate'
+ require 'yajl/http_stream'
+
+ url = URI.parse("http://search.twitter.com/search.json?q=engineyard")
+ results = Yajl::HttpStream.get(url)
+
+Since yajl-ruby parses JSON as a stream, supporting API's like Twitter's Streaming API are a piece-of-cake.
+You can simply supply a block to Yajl::HttpStream.get, which is used as the callback for when a JSON object has been
+unserialized off the stream. For the case of this Twitter Streaming API call, the callback gets fired a few times a second (depending on your connection speed).
+The code below is all that's needed to make the request and stream unserialized Ruby hashes off the response, continuously.
+You'll note that I've enabled the :symbolize_keys parser option as well. Doing so is much more efficient for parsing JSON streams with
+lots of repetitive keys - for things like result sets or multiple API requests - than the same parse with string keys.
+This is because Ruby will reuse (and never GC) its symbol table. Be that as it may, if you want to parse JSON strings with random key names
+it's much better to leave string keys enabled (the default), so they can get GC'd later.
+
+ require 'uri'
+ require 'yajl/http_stream'
+
+ uri = URI.parse("http://#{username}:#{password}@stream.twitter.com/spritzer.json")
+ Yajl::HttpStream.get(uri, :symbolize_keys => true) do |hash|
+   puts hash.inspect
+ end
+
+Or how about parsing directly from a compressed file?
+
+ require 'yajl/bzip2'
+
+ file = File.new('some.json.bz2', 'r')
+ result = Yajl::Bzip2::StreamReader.parse(file)
+
+=== Encoding
+
+Since yajl-ruby does everything using streams, you simply need to pass the object to encode, and the IO to write the stream to (this happens in chunks).
+
+This allows you to encode JSON as a stream, writing directly to a socket
+
+ socket = TCPSocket.new(192.168.1.101, 9000)
+ hash = {:foo => 12425125, :bar => "some string", ... }
+ encoder = Yajl::Encoder.new
+ Yajl::Encoder.encode(hash, socket)
+
+Or what if you wanted to compress the stream over the wire?
+
+ require 'yajl/gzip'
+ socket = TCPSocket.new(192.168.1.101, 9000)
+ hash = {:foo => 12425125, :bar => "some string", ... }
+ Yajl::Gzip::StreamWriter.encode(hash, socket)
+
+Or what about encoding multiple objects to JSON over the same stream?
+This example will encode and send 50 JSON objects over the same stream, continuously.
+
+ socket = TCPSocket.new(192.168.1.101, 9000)
+ encoder = Yajl::Encoder.new
+ 50.times do
+   hash = {:current_time => Time.now.to_f, :foo => 12425125}
+   encoder.encode(hash, socket)
+ end
+
+Using EventMachine and you want to encode and send in chunks?
+(Assume we're in an EventMachine::Connection instance)
+
+ def post_init
+    # Passing a :terminator character will let us determine when the encoder
+    # is done encoding the current object
+    @encoder = Yajl::Encoder.new
+    motd_contents = File.read("/path/to/motd.txt")
+    status = File.read("/path/to/huge/status_file.txt")
+    @motd = {:motd => motd_contents, :system_status => status}
+ end
+
+ def connection_completed
+   # The encoder will do its best to hand you data in chunks that
+   # are around 8kb (but you may see some that are larger)
+   #
+   # It should be noted that you could have also assigned the _on_progress_ callback
+   # much like you can assign the _on_parse_complete_ callback with the parser class.
+   # Passing a block (like below) essentially tells the encoder to use that block
+   # as the callback normally assigned to _on_progress_.
+   #
+   # Send our MOTD and status
+   @encoder.encode(@motd) do |chunk|
+     if chunk.nil? # got our terminator, encoding is done
+       close_connection_after_writing
+     else
+       send_data(chunk)
+     end
+   end
+ end
+
+But to make things simple, you might just want to let yajl-ruby do all the hard work for you and just hand back
+a string when it's finished. In that case, just don't provide and IO or block (or assign the on_progress callback).
+
+ str = Yajl::Encoder.encode(obj)
+
+You can also use Yajl::Bzip2::StreamWriter and Yajl::Deflate::StreamWriter. So you can pick whichever fits your CPU/bandwidth sweet-spot.
+
+== JSON gem Compatibility API
+
+The JSON gem compatibility API isn't enabled by default. You have to explicitly require it like so:
+
+ require 'yajl/json_gem'
+
+That's right, you can just replace "require 'json'" with the line above and you're done!
+
+This will require yajl-ruby itself, as well as enable its JSON gem compatibility API.
+
+This includes the following API:
+
+ JSON.parse, JSON.generate, JSON.pretty_generate, JSON.load, JSON.dump
+ and all of the #to_json instance method overrides for Ruby's primitive objects
+
+Once the compatibility API is enabled, your existing or new project should work as if the JSON gem itself were being used. Only you'll be using Yajl ;)
+
+There are a lot more possibilities that I'd love to see other gems/plugins for someday.
+
+Some ideas:
+* parsing logs in JSON format
+* a Rails plugin - DONE! (http://github.com/technoweenie/yajl-rails)
+* official support in Rails 3 - DONE (http://github.com/rails/rails/commit/a96bf4ab5e73fccdafb78b99e8a122cc2172b505)
+** and is the default (if installed) - http://github.com/rails/rails/commit/63bb955a99eb46e257655c93dd64e86ebbf05651
+* Rack middleware (ideally the JSON body could be handed to the parser while it's still being received, this is apparently possible with Unicorn)
+* JSON API clients (http://github.com/brianmario/freckle-api)
+* ???
+
+== Benchmarks
+
+After I finished implementation - this library performs close to the same as the current JSON.parse (C gem) does on small/medium files.
+
+But on larger files, and higher amounts of iteration, this library was around 2x faster than JSON.parse.
+
+The main benefit of this library is in its memory usage.
+Since it's able to parse the stream in chunks, its memory requirements are very, very low.
+
+Here's what parsing a 2.43MB JSON file off the filesystem 20 times looks like:
+
+=== Memory Usage
+
+==== Average
+
+* Yajl::Parser#parse: 32MB
+* JSON.parse: 54MB
+* ActiveSupport::JSON.decode: 63MB
+
+==== Peak
+
+* Yajl::Parser#parse: 32MB
+* JSON.parse: 57MB
+* ActiveSupport::JSON.decode: 67MB
+
+=== Parse Time
+
+* Yajl::Parser#parse: 4.54s
+* JSON.parse: 5.47s
+* ActiveSupport::JSON.decode: 64.42s
+
+=== Encode Time
+
+* Yajl::Encoder#encode: 3.59s
+* JSON#to_json: 6.2s
+* ActiveSupport::JSON.encode: 45.58s
+
+=== Compared to YAML
+
+NOTE: I converted the 2.4MB JSON file to YAML for this test.
+
+==== Parse Time (from their respective formats)
+
+* Yajl::Parser#parse: 4.33s
+* JSON.parse: 5.37s
+* YAML.load: 19.47s
+
+==== Encode Time (to their respective formats)
+
+* Yajl::Encoder#encode: 3.47s
+* JSON#to_json: 6.6s
+* YAML.dump(obj, io): 1309.93s
+
+=== Compared to Marshal.load/Marshal.dump
+
+NOTE: I converted the 2.4MB JSON file to a Hash and a dump file from Marshal.dump for this test.
+
+==== Parse Time (from their respective formats)
+
+* Yajl::Parser#parse: 4.54s
+* JSON.parse: 7.40s
+* Marshal.load: 7s
+
+==== Encode Time (to their respective formats)
+
+* Yajl::Encoder#encode: 2.39s
+* JSON#to_json: 8.37s
+* Marshal.dump: 4.66s
+
+== Third Party Sources Bundled
+
+This project includes code from the BSD licensed yajl project, copyright 2007-2009 Lloyd Hilaiel
+
+== Special Thanks & Contributors
+
+For those of you using yajl-ruby out in the wild, please hit me up on Twitter (brianmario) or send me a message here on the Githubs describing the site and how you're using it. I'd love to get a list going!
+
+I've had a lot of inspiration, and a lot of help. Thanks to everyone who's been a part of this and those to come!
+
+* Lloyd Hilaiel - http://github.com/lloyd - for writing Yajl!!
+* Josh Ferguson - http://github.com/besquared - for peer-pressuring me into getting back into C; it worked ;) Also tons of support over IM
+* Jonathan Novak - http://github.com/cypriss - pointer-hacking help
+* Tom Smith - http://github.com/rtomsmith - pointer-hacking help
+* Rick Olson - http://github.com/technoweenie - for making an ActiveSupport patch with support for this library and teasing me that it might go into Rails 3. You sure lit a fire under my ass and I got a ton of work done because of it! :)
+* The entire Github Crew - http://github.com/ - my inspiration, time spent writing this, finding Yajl, So many-MANY other things wouldn't have been possible without this awesome service. I owe you guys some whiskey at Kilowatt.
+* Ben Burkert - http://github.com/benburkert
+* Aman Gupta - http://github.com/tmm1 - tons of suggestions and inspiration for the most recent features, and hopefully more to come ;)
+* Filipe Giusti
+* Jonathan George
+* Luke Redpath
+* Neil Berkman
+* Pavel Valodzka
+* Rob Sharp

data/Rakefile

@@ -0,0 +1,40 @@
+# encoding: UTF-8
+begin
+  require 'jeweler'
+  Jeweler::Tasks.new do |gem|
+    gem.name = "benofsky-yajl-ruby"
+    gem.summary = "Ruby C bindings to the excellent Yajl JSON stream-based parser library."
+    gem.email = "ben@benmcredmond.com"
+    gem.homepage = "http://github.com/benofsky/yajl-ruby"
+    gem.authors = ["Ben McRedmond", "Brian Lopez", "Lloyd Hilaiel"]
+    gem.require_paths = ["lib", "ext"]
+    gem.extra_rdoc_files = `git ls-files *.rdoc`.split("\n") << 'ext/yajl.c'
+    gem.files = `git ls-files`.split("\n")
+    gem.extensions = ["ext/extconf.rb"]
+    gem.files.include %w(lib/jeweler/templates/.document lib/jeweler/templates/.gitignore)
+    gem.rubyforge_project = "yajl-ruby"
+  end
+rescue LoadError
+  puts "Jeweler, or one of its dependencies, is not available. Install it with: sudo gem install jeweler -s http://gems.github.com"
+end
+
+begin
+  require 'rake'
+  require 'spec/rake/spectask'
+
+  desc "Run all examples with RCov"
+  Spec::Rake::SpecTask.new('spec:rcov') do |t|
+    t.spec_files = FileList['spec/']
+    t.rcov = true
+    t.rcov_opts = lambda do
+      IO.readlines("spec/rcov.opts").map {|l| l.chomp.split " "}.flatten
+    end
+  end
+  Spec::Rake::SpecTask.new('spec') do |t|
+    t.spec_files = FileList['spec/']
+    t.libs << 'ext'
+    t.spec_opts << '--options' << 'spec/spec.opts'
+  end
+rescue LoadError
+  puts "RSpec, or one of its dependencies, is not available. Install it with: sudo gem install rspec"
+end

data/VERSION.yml

@@ -0,0 +1,5 @@
+--- 
+:minor: 7
+:patch: 6
+:build: 
+:major: 0