yajl-ruby 1.0.0-x86-mingw32
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of yajl-ruby might be problematic. Click here for more details.
- data/.gitignore +12 -0
- data/.rspec +2 -0
- data/CHANGELOG.md +327 -0
- data/Gemfile +3 -0
- data/MIT-LICENSE +20 -0
- data/README.md +362 -0
- data/Rakefile +2 -0
- data/benchmark/encode.rb +72 -0
- data/benchmark/encode_json_and_marshal.rb +42 -0
- data/benchmark/encode_json_and_yaml.rb +53 -0
- data/benchmark/http.rb +32 -0
- data/benchmark/parse.rb +94 -0
- data/benchmark/parse_json_and_marshal.rb +50 -0
- data/benchmark/parse_json_and_yaml.rb +55 -0
- data/benchmark/parse_stream.rb +54 -0
- data/benchmark/subjects/item.json +1 -0
- data/benchmark/subjects/ohai.json +1216 -0
- data/benchmark/subjects/ohai.marshal_dump +0 -0
- data/benchmark/subjects/ohai.yml +975 -0
- data/benchmark/subjects/twitter_search.json +1 -0
- data/benchmark/subjects/twitter_stream.json +430 -0
- data/benchmark/subjects/unicode.json +1 -0
- data/examples/encoding/chunked_encoding.rb +27 -0
- data/examples/encoding/one_shot.rb +13 -0
- data/examples/encoding/to_an_io.rb +12 -0
- data/examples/http/twitter_search_api.rb +12 -0
- data/examples/http/twitter_stream_api.rb +26 -0
- data/examples/parsing/from_file.rb +14 -0
- data/examples/parsing/from_stdin.rb +9 -0
- data/examples/parsing/from_string.rb +13 -0
- data/ext/yajl/api/yajl_common.h +89 -0
- data/ext/yajl/api/yajl_gen.h +161 -0
- data/ext/yajl/api/yajl_parse.h +196 -0
- data/ext/yajl/api/yajl_version.h +23 -0
- data/ext/yajl/extconf.rb +7 -0
- data/ext/yajl/yajl.c +164 -0
- data/ext/yajl/yajl_alloc.c +65 -0
- data/ext/yajl/yajl_alloc.h +50 -0
- data/ext/yajl/yajl_buf.c +119 -0
- data/ext/yajl/yajl_buf.h +73 -0
- data/ext/yajl/yajl_bytestack.h +85 -0
- data/ext/yajl/yajl_encode.c +201 -0
- data/ext/yajl/yajl_encode.h +52 -0
- data/ext/yajl/yajl_ext.c +905 -0
- data/ext/yajl/yajl_ext.h +135 -0
- data/ext/yajl/yajl_gen.c +344 -0
- data/ext/yajl/yajl_lex.c +748 -0
- data/ext/yajl/yajl_lex.h +135 -0
- data/ext/yajl/yajl_parser.c +450 -0
- data/ext/yajl/yajl_parser.h +82 -0
- data/ext/yajl/yajl_version.c +7 -0
- data/lib/yajl.rb +75 -0
- data/lib/yajl/1.8/yajl.so +0 -0
- data/lib/yajl/1.9/yajl.so +0 -0
- data/lib/yajl/bzip2.rb +11 -0
- data/lib/yajl/bzip2/stream_reader.rb +31 -0
- data/lib/yajl/bzip2/stream_writer.rb +14 -0
- data/lib/yajl/deflate.rb +6 -0
- data/lib/yajl/deflate/stream_reader.rb +43 -0
- data/lib/yajl/deflate/stream_writer.rb +20 -0
- data/lib/yajl/gzip.rb +6 -0
- data/lib/yajl/gzip/stream_reader.rb +30 -0
- data/lib/yajl/gzip/stream_writer.rb +13 -0
- data/lib/yajl/http_stream.rb +212 -0
- data/lib/yajl/json_gem.rb +15 -0
- data/lib/yajl/json_gem/encoding.rb +51 -0
- data/lib/yajl/json_gem/parsing.rb +26 -0
- data/lib/yajl/version.rb +3 -0
- data/lib/yajl/yajl.rb +2 -0
- data/spec/encoding/encoding_spec.rb +271 -0
- data/spec/global/global_spec.rb +54 -0
- data/spec/http/fixtures/http.bzip2.dump +0 -0
- data/spec/http/fixtures/http.chunked.dump +11 -0
- data/spec/http/fixtures/http.deflate.dump +0 -0
- data/spec/http/fixtures/http.error.dump +12 -0
- data/spec/http/fixtures/http.gzip.dump +0 -0
- data/spec/http/fixtures/http.html.dump +1220 -0
- data/spec/http/fixtures/http.raw.dump +1226 -0
- data/spec/http/http_delete_spec.rb +98 -0
- data/spec/http/http_error_spec.rb +32 -0
- data/spec/http/http_get_spec.rb +109 -0
- data/spec/http/http_post_spec.rb +123 -0
- data/spec/http/http_put_spec.rb +105 -0
- data/spec/http/http_stream_options_spec.rb +27 -0
- data/spec/json_gem_compatibility/compatibility_spec.rb +203 -0
- data/spec/parsing/active_support_spec.rb +64 -0
- data/spec/parsing/chunked_spec.rb +96 -0
- data/spec/parsing/fixtures/fail.15.json +1 -0
- data/spec/parsing/fixtures/fail.16.json +1 -0
- data/spec/parsing/fixtures/fail.17.json +1 -0
- data/spec/parsing/fixtures/fail.26.json +1 -0
- data/spec/parsing/fixtures/fail11.json +1 -0
- data/spec/parsing/fixtures/fail12.json +1 -0
- data/spec/parsing/fixtures/fail13.json +1 -0
- data/spec/parsing/fixtures/fail14.json +1 -0
- data/spec/parsing/fixtures/fail19.json +1 -0
- data/spec/parsing/fixtures/fail20.json +1 -0
- data/spec/parsing/fixtures/fail21.json +1 -0
- data/spec/parsing/fixtures/fail22.json +1 -0
- data/spec/parsing/fixtures/fail23.json +1 -0
- data/spec/parsing/fixtures/fail24.json +1 -0
- data/spec/parsing/fixtures/fail25.json +1 -0
- data/spec/parsing/fixtures/fail27.json +2 -0
- data/spec/parsing/fixtures/fail28.json +2 -0
- data/spec/parsing/fixtures/fail3.json +1 -0
- data/spec/parsing/fixtures/fail4.json +1 -0
- data/spec/parsing/fixtures/fail5.json +1 -0
- data/spec/parsing/fixtures/fail6.json +1 -0
- data/spec/parsing/fixtures/fail9.json +1 -0
- data/spec/parsing/fixtures/pass.array.json +6 -0
- data/spec/parsing/fixtures/pass.codepoints_from_unicode_org.json +1 -0
- data/spec/parsing/fixtures/pass.contacts.json +1 -0
- data/spec/parsing/fixtures/pass.db100.xml.json +1 -0
- data/spec/parsing/fixtures/pass.db1000.xml.json +1 -0
- data/spec/parsing/fixtures/pass.dc_simple_with_comments.json +11 -0
- data/spec/parsing/fixtures/pass.deep_arrays.json +1 -0
- data/spec/parsing/fixtures/pass.difficult_json_c_test_case.json +1 -0
- data/spec/parsing/fixtures/pass.difficult_json_c_test_case_with_comments.json +1 -0
- data/spec/parsing/fixtures/pass.doubles.json +1 -0
- data/spec/parsing/fixtures/pass.empty_array.json +1 -0
- data/spec/parsing/fixtures/pass.empty_string.json +1 -0
- data/spec/parsing/fixtures/pass.escaped_bulgarian.json +4 -0
- data/spec/parsing/fixtures/pass.escaped_foobar.json +1 -0
- data/spec/parsing/fixtures/pass.item.json +1 -0
- data/spec/parsing/fixtures/pass.json-org-sample1.json +23 -0
- data/spec/parsing/fixtures/pass.json-org-sample2.json +11 -0
- data/spec/parsing/fixtures/pass.json-org-sample3.json +26 -0
- data/spec/parsing/fixtures/pass.json-org-sample4-nows.json +88 -0
- data/spec/parsing/fixtures/pass.json-org-sample4.json +89 -0
- data/spec/parsing/fixtures/pass.json-org-sample5.json +27 -0
- data/spec/parsing/fixtures/pass.map-spain.xml.json +1 -0
- data/spec/parsing/fixtures/pass.ns-invoice100.xml.json +1 -0
- data/spec/parsing/fixtures/pass.ns-soap.xml.json +1 -0
- data/spec/parsing/fixtures/pass.numbers-fp-4k.json +6 -0
- data/spec/parsing/fixtures/pass.numbers-fp-64k.json +61 -0
- data/spec/parsing/fixtures/pass.numbers-int-4k.json +11 -0
- data/spec/parsing/fixtures/pass.numbers-int-64k.json +154 -0
- data/spec/parsing/fixtures/pass.twitter-search.json +1 -0
- data/spec/parsing/fixtures/pass.twitter-search2.json +1 -0
- data/spec/parsing/fixtures/pass.unicode.json +3315 -0
- data/spec/parsing/fixtures/pass.yelp.json +1 -0
- data/spec/parsing/fixtures/pass1.json +56 -0
- data/spec/parsing/fixtures/pass2.json +1 -0
- data/spec/parsing/fixtures/pass3.json +6 -0
- data/spec/parsing/fixtures_spec.rb +40 -0
- data/spec/parsing/one_off_spec.rb +85 -0
- data/spec/rcov.opts +3 -0
- data/spec/spec_helper.rb +16 -0
- data/tasks/compile.rake +35 -0
- data/tasks/rspec.rake +16 -0
- data/yajl-ruby.gemspec +24 -0
- metadata +335 -0
data/.gitignore
ADDED
data/.rspec
ADDED
data/CHANGELOG.md
ADDED
@@ -0,0 +1,327 @@
|
|
1
|
+
# Changelog
|
2
|
+
|
3
|
+
## 1.0.0 (September 13th, 2011)
|
4
|
+
* add deprecation notice for Yajl's Bzip2 support
|
5
|
+
* add deprecation notice for Yajl's Deflate support
|
6
|
+
* add deprecation notice for Yajl's Gzip support
|
7
|
+
* add deprecation notice for Yajl's JSON gem compatibility API
|
8
|
+
* add deprecation notice for Yajl::HttpStream
|
9
|
+
* change the path the extension is copied into to be 'lib/yajl'
|
10
|
+
* remove 'ext' from the loadpath
|
11
|
+
|
12
|
+
## 0.8.3 (August 16th, 2011)
|
13
|
+
* fix bug where Yajl::HttpStream wouldn't pass through a user-specified socket
|
14
|
+
* fix incorrect Ruby initialization hook method name
|
15
|
+
* Bump bundled YAJL version to 1.0.12
|
16
|
+
* fix to correctly symbolize multibyte characters on 1.9
|
17
|
+
* add `:headers` option to Yajl::HttpStream for user-specified arbitrary headers
|
18
|
+
|
19
|
+
## 0.8.2 (March 22nd, 2011)
|
20
|
+
* define RSTRING_NOT_MODIFIED for rbx to prevent string caching, making things A LOT faster (100x)
|
21
|
+
|
22
|
+
## 0.8.1 (February 11th, 2011)
|
23
|
+
* fixed a retart bug where Yajl::VERSION wasn't defined when explicitly requiring yajl/http_stream
|
24
|
+
|
25
|
+
## 0.8.0 (February 2nd, 2011)
|
26
|
+
* added a new html_safe option to Yajl::Encoder to escape '/' characters for use in the DOM
|
27
|
+
* moved away from Jeweler to a Bundler/manual gemfile management setup
|
28
|
+
|
29
|
+
## 0.7.9 (January 11th, 2011)
|
30
|
+
* moved to rspec2
|
31
|
+
* fixed some compilation warnings on 1.9.3
|
32
|
+
* brought over latest from Yajl upstream
|
33
|
+
* finally removed the deprecated Yajl::Stream methods
|
34
|
+
* moved to rake-compiler
|
35
|
+
* moved to Bundler for development
|
36
|
+
* fix memory corruption bug when using :pretty => true and a custom indent string
|
37
|
+
* fixed memory leak when exceptions were being raised during a parse
|
38
|
+
|
39
|
+
## 0.7.8 (September 27th, 2010)
|
40
|
+
* fix a bug in chunked http response regex (thanks to http://github.com/kevn for catching this)
|
41
|
+
* Make sure json compability doesn't break ActiveSupport#to_json
|
42
|
+
* fix improper usage of rb_define_method
|
43
|
+
|
44
|
+
## 0.7.7 (July 12th, 2010)
|
45
|
+
* full string encoding support for 1.9, respecting Encoding.default_internal
|
46
|
+
* refactor the #to_json function bodies into a C macro
|
47
|
+
* some misc code cleanup in the benchmark scripts
|
48
|
+
|
49
|
+
## 0.7.6 (May 1st, 2010)
|
50
|
+
* use memcmp instead of strcmp for invalid Fixnum check
|
51
|
+
* add a spec to verify unicode chars can be used as keys
|
52
|
+
* twitter examples updated
|
53
|
+
* only use -Wextra if ENV['DEBUG'] is set as gcc 3 doesn't know about it
|
54
|
+
* fix chunked http encoding parse logic to further comply with the spec (thanks to Sebastian Cohnen <sebastian.cohnen@gmx.net>)
|
55
|
+
* removed as_json checks and usage in encoder to prevent infinite loops
|
56
|
+
** In Rails a lot of objects return self from the as_json method - which is wrong IMO - and needs to be fixed before this feature will work properly
|
57
|
+
|
58
|
+
## 0.7.5 (March 23rd, 2010)
|
59
|
+
* check for existence of and use as_json method on custom objects
|
60
|
+
* bugfix with read buffer when parsing from an IO (thanks to Pavel Valodzka <pavel@valodzka.name>)
|
61
|
+
* merged in latest yajl
|
62
|
+
* enable -Wextra during compilation
|
63
|
+
* brought back ability to pass a buffer to bzip/gzip/deflate #read helper methods
|
64
|
+
|
65
|
+
## 0.7.4 (March 3rd, 2010)
|
66
|
+
* bugfix for the JSON gem compatibility API's Object#to_json method improperly encoding strings
|
67
|
+
|
68
|
+
## 0.7.3 (February 23rd, 2010)
|
69
|
+
* remove trap from HttpStream code, it's really not needed
|
70
|
+
|
71
|
+
## 0.7.2 (February 23rd, 2010)
|
72
|
+
* fixed broken to_json compatibility
|
73
|
+
* removed strlen in a few places in favor of RSTRING_LEN since ruby already knows the length of the string
|
74
|
+
* patched Yajl to more efficiently reset it's lexer (no more malloc/free)
|
75
|
+
* removed dependency on IO#eof? when parsing from an IO for full Rack-spec compatibility
|
76
|
+
* removed some various cruft code in C
|
77
|
+
|
78
|
+
## 0.7.1 (February 17th, 2010)
|
79
|
+
* revert a patch made to bundled Yajl enabling optional quoting of strings that broke binary API compatibility
|
80
|
+
|
81
|
+
## 0.7.0 (February 5th, 2010)
|
82
|
+
* ensure utf8 encoding is set on relevant strings during parse/encode in 1.9
|
83
|
+
|
84
|
+
## 0.6.9 (January 26th, 2010)
|
85
|
+
* HttpStream patches merged in from Luke Redpath <contact@lukeredpath.co.uk>
|
86
|
+
* Changed how Yajl::Parser was calling IO#read to better conform to the Rack spec and thus can be used to directly parse a rack.input stream
|
87
|
+
|
88
|
+
## 0.6.8 (January 1st, 2010)
|
89
|
+
* A couple of small performance patches
|
90
|
+
* Allow passing a string to Yajl::HttpStream methods instead of only a URI
|
91
|
+
|
92
|
+
## 0.6.7 (December 4th, 2009)
|
93
|
+
* Bump internal version constant to the proper value (doh!)
|
94
|
+
* Bring over latest from Yajl upstream
|
95
|
+
|
96
|
+
## 0.6.6 (December 1st, 2009)
|
97
|
+
* Brought over some optimizations from Macruby's use of some yajl-ruby codez
|
98
|
+
* Yajl::HttpStream now supports being killed for long-running requests, thanks to Filipe Giusti <filipegiusti@gmail.com>
|
99
|
+
|
100
|
+
## 0.6.5 (November 13th, 2009)
|
101
|
+
* optimize symbol creation while symbolize_keys is turned on
|
102
|
+
* fix for 32bit integer conversion into ruby
|
103
|
+
|
104
|
+
## 0.6.4 (November 4th, 2009)
|
105
|
+
* All specs pass on Rubinius :)
|
106
|
+
* Added Yajl.load and Yajl.dump for compatibility with other various data format API's in ruby
|
107
|
+
* Fixed a bug in Yajl::Encoder which allowed direct, unescaped encoding of NaN, Infinity and -Infinity.
|
108
|
+
It will now properly throw a Yajl::EncodeError exception if either of these values are found unescaped.
|
109
|
+
* Update bundled Yajl library to 1.0.7
|
110
|
+
* Conditionally define RSTRING_* and RARRAY_* for older versions of ruby (1.8.5 is still the default on CentOS)
|
111
|
+
* Bugfix for JSON gem exception classes to more accurately match those of the actual JSON gem
|
112
|
+
* A few small speed optimizations
|
113
|
+
* Updated specs to not run bzip2 related examples if unable to load the bzip2 library
|
114
|
+
* Finally added UTF-8 checking specs
|
115
|
+
* Removed needless calls to ID2SYM all over the place
|
116
|
+
* Updated benchmark scripts to bring the GC into the picture a little more
|
117
|
+
|
118
|
+
## 0.6.3 (August 25th, 2009)
|
119
|
+
* Fixed a bug in the JSON gem compatibility API where strings weren't being properly escaped
|
120
|
+
|
121
|
+
## 0.6.2 (August 25th, 2009)
|
122
|
+
* Fixed a bug surfaced by an existing library providing a to_json method, and Yajl would double-quote the values provided
|
123
|
+
|
124
|
+
## 0.6.1 (August 20th, 2009)
|
125
|
+
* Fixed a bug in Yajl::HttpStream where responses contained multiple JSON strings but weren't Transfer-Encoding: chunked (thanks @dacort!)
|
126
|
+
|
127
|
+
## 0.6.0 (August 19th, 2009)
|
128
|
+
* Added POST, PUT and DELETE support to Yajl::HttpStream
|
129
|
+
** POST support initially contributed by jdg (http://github.com/jdg) - Although oortle (http://github.com/oortle) coded it up in a fork with it as well.
|
130
|
+
|
131
|
+
## 0.5.12 (July 31st, 2009)
|
132
|
+
* Add another option that can be passed to Yajl::Encoder's constructor (:terminator) to allow the caller some control over
|
133
|
+
when a full JSON string has been generated by the encoder. More information on it's use in the README
|
134
|
+
|
135
|
+
## 0.5.11 (July 14th, 2009)
|
136
|
+
* fixing a bug Aman found with to_json on non-primitive Ruby objects and double-quoting in the JSON compat API
|
137
|
+
|
138
|
+
## 0.5.10 (July 13th, 2009)
|
139
|
+
* Bugfix for the JSON gem compatibility API's default Object#to_json helper
|
140
|
+
|
141
|
+
## 0.5.9 (July 9th, 2009)
|
142
|
+
* Bugfix for Yajl::Encoder where encoding a hash like {:a => :b} would get stuck in an infinite loop
|
143
|
+
|
144
|
+
## 0.5.8 (July 6th, 2009)
|
145
|
+
* Bugfix in Yajl::HttpStream for proper handling of the Content-type header (Rob Sharp)
|
146
|
+
* Yajl::Encoder now has an on_progress callback setter, which can be used to harness the encoder's streaming ability.
|
147
|
+
** The passed Proc/lambda will be called, and passed every chunk (currently 8kb) of the encoded JSON string as it's being encoded.
|
148
|
+
* API CHANGE WARNING: Yajl::Encoder.encode's block will now be used as (and work the same as) the on_progress callback
|
149
|
+
** This means the block will be passed chunks of the JSON string at a time, giving the caller the ability to start processing the encoded data while it's still being encoded.
|
150
|
+
* fixed grammatical error in README (Neil Berkman)
|
151
|
+
* Added some encoder examples
|
152
|
+
|
153
|
+
## 0.5.7 (June 23rd, 2009)
|
154
|
+
* You can now pass parser options (like :symbolize_keys for example) to Yajl::HttpStream.get
|
155
|
+
* Refactored spec tests a bit, DRYing up the Yajl::HttpStream specs quite a bit.
|
156
|
+
* Added a spec rake task, and spec.opts file
|
157
|
+
* Updated and renamed rcov rake task, and added rcov.opts file
|
158
|
+
|
159
|
+
## 0.5.6 (June 19th, 2009)
|
160
|
+
* Added JSON.default_options hash to the JSON gem compatibility API
|
161
|
+
* Split out the JSON gem compatibility API's parsing and encoding methods into individually includable files
|
162
|
+
** the use case here is if you *only* want parsing, or *only* want encoding
|
163
|
+
** also, if you don't include encoding it won't include the #to_json overrides which tend to cause problems in some environments.
|
164
|
+
* Removed some large benchmark test files to reduce the size of the packaged gem by 1.5MB!
|
165
|
+
|
166
|
+
## 0.5.5 (June 17th, 2009)
|
167
|
+
* Introduction of the JSON gem compatibility API
|
168
|
+
** NOTE: this isn't a 1:1 compatibility API, the goal was to be compatible with as many of the projects using the JSON gem as possible - not the JSON gem API itself
|
169
|
+
** the compatibility API must be explicitly enabled by requiring 'yajl/json_gem' in your project
|
170
|
+
** JSON.parse, JSON.generate, and the #to_json instance method extension to ruby's primitive classes are all included
|
171
|
+
* Fix Yajl::Encoder to ensure map keys are strings
|
172
|
+
* Encoding multiple JSON objects to a single stream doesn't separate by a newline character anymore
|
173
|
+
* Yajl::Encoder now checks for the existence of, and will call #to_json on any non-primitive object
|
174
|
+
|
175
|
+
## 0.5.4 (June 16th, 2009)
|
176
|
+
* Yajl::Parser's :symbolize_keys option now defaults to false
|
177
|
+
* remove use of sprintf for a little speed improvement while parsing
|
178
|
+
|
179
|
+
## 0.5.3 (June 7th, 2009)
|
180
|
+
* The IO parameter for Yajl::Encode#encode is now optional, and accepts a block
|
181
|
+
** it will return the resulting JSON string if no IO is passed to stream to
|
182
|
+
** if a block is passed, it will call and pass it the resulting JSON string
|
183
|
+
* Yajl::Parser#parse can now parse from a String as well as an IO
|
184
|
+
* Added and updated lot of in-code documentation.
|
185
|
+
** all the C code exposed to Ruby should now have comments
|
186
|
+
* Added :symbolize_keys option to the Yajl::Parser class, which defaults to true.
|
187
|
+
** Having this option enabled has shown around an 18% speedup in parsing time according to my benchmarks
|
188
|
+
|
189
|
+
## 0.5.2 (May 30th, 2009)
|
190
|
+
* Added class helper methods Yajl::Encoder.encode(obj, io) and Yajl::Parser.parse(io)
|
191
|
+
* added tests for the above
|
192
|
+
* Updated Twitter streaming example to have a less verbose output
|
193
|
+
* Patch Yajl so encoding can continue as a stream
|
194
|
+
* IE: multiple objects encoded onto the same IO
|
195
|
+
* added a test for the above
|
196
|
+
* Set the internal read buffer size back down to 8kb by default
|
197
|
+
* Added an internal write buffer size (set to 8kb by default) which is used to throttle writes to the output stream
|
198
|
+
* This is to fix a major performance bug/issue with the IO#write C method in ruby 1.9.x (I've opened a bug with them about it)
|
199
|
+
* Fixed a typo in a one-off parsing spec test
|
200
|
+
* Updated benchmarks to work properly in 1.9 (required removal ActiveSupport benchmarking for now)
|
201
|
+
* Updated spec tests to respect ridiculous differences in hash key ordering between 1.8 and 1.9
|
202
|
+
|
203
|
+
|
204
|
+
## 0.5.1 (May 25th, 2009)
|
205
|
+
* added some more tests for the new API
|
206
|
+
* inlined a couple of hot functions used in parsing for a little speedup
|
207
|
+
* updates to readme, reflecting changes in API
|
208
|
+
* version bump to push another gem build
|
209
|
+
|
210
|
+
## 0.5.0 (May 25th, 2009)
|
211
|
+
* Refactored internal API so the caller can specify initialization options for the Parser and Encoder respectively. Two new classes were introduced as a result - Yajl::Parser and Yajl::Encoder. The newly refactored codebase is cleaner, thread-safe and removed all of the hack-code that was trickled around to make things work in the previous implementation. She's much more seaworthy now cap'n!
|
212
|
+
* Yajl::Parser.new accepts two options, :allow_comments and :check_utf8 which both default to true
|
213
|
+
* Yajl::Encoder.new accepts two options, :pretty and :indent which default to false and " " respectively
|
214
|
+
* cleaned up a lot of state code, that to my knowledge prevented yajl-ruby from being used in a thread-safe environment.
|
215
|
+
* added deprecated messaging to Yajl::Stream.parse and Yajl::Stream.encode - these will likely go away before 0.6.0
|
216
|
+
* fixed a bug in the chunked http response parser regarding partially received chunks
|
217
|
+
* added a Twitter Search API example showing off the HttpStream API
|
218
|
+
|
219
|
+
## 0.4.9 (May 20th, 2009)
|
220
|
+
* fixed some parser state bugs surfaced by edge cases
|
221
|
+
* added support for Chunked HTTP response bodies in Yajl::HttpStream
|
222
|
+
* added support for passing a block to Yajl::HttpStream.get that will be used as a callback whenever a JSON object is parsed off the stream (even if there is more than one!)
|
223
|
+
* added an examples folder, and put an example using the Twitter Streaming API in there to start
|
224
|
+
* added some more spec tests, this time around Chunked parsing and continuously parsing multiple JSON strings
|
225
|
+
|
226
|
+
## 0.4.8 (May 18th, 2009)
|
227
|
+
* fixed a totally bone-head compilation problem, I created for myself ;)
|
228
|
+
|
229
|
+
## 0.4.7 (May 18th, 2009)
|
230
|
+
* Bundling Yajl sources to remove the need to install them (and CMake) separately (Thank you Lloyd!!!) This means you can now simply install the gem and be off and running
|
231
|
+
* Added some spec tests for Yajl::HttpStream
|
232
|
+
* Added some spec tests for Yajl::Stream.encode
|
233
|
+
* added some more thank you's, where credit's due - in the readme
|
234
|
+
* updated the unicode.json file to reflect a "real-life" JSON response
|
235
|
+
* reorganized spec tests into their functional areas
|
236
|
+
* added an rcov rake task to generate code coverage output
|
237
|
+
|
238
|
+
## 0.4.6 (May 17th, 2009)
|
239
|
+
* Applied a patch from benburkert (http://github.com/benburkert) to fix HTTP Basic Auth in Yajl::HttpStream.get
|
240
|
+
|
241
|
+
## 0.4.5 (May 17th, 2009)
|
242
|
+
* added Yajl::Stream.encode(hash, io)
|
243
|
+
* generates a JSON string stream, and writes to IO
|
244
|
+
* compressed StreamWriter helpers added as well
|
245
|
+
* fixed a pretty lame segfault in (x86_64 only?) ubuntu/linux
|
246
|
+
* changed the compiled extension to have a more specific name (yajl_ext) for easier loading
|
247
|
+
* removed forced-load of .bundle file, for the rest of the planet aside from OSX users
|
248
|
+
* added some more benchmarks to compare to other forms of serialization in Ruby
|
249
|
+
* various readme updates
|
250
|
+
|
251
|
+
## 0.4.4 (May 12th, 2009)
|
252
|
+
* NOTE: Breaking API change:
|
253
|
+
* renamed Yajl::GzipStreamReader to Yajl::Gzip::StreamReader
|
254
|
+
* added Yajl::Bzip2::StreamReader
|
255
|
+
* depends on the bzip2-ruby gem if you want to use it, if not Yajl::Bzip2 won't be loaded
|
256
|
+
* added Yajl::Deflate::StreamReader
|
257
|
+
* actually uses Zlib::Inflate for stream decompression
|
258
|
+
* added parse(io) class methods to Yajl::Gzip::StreamReader and Yajl::Bzip2::StreamReader as a helper for parsing compressed streams.
|
259
|
+
* updated Yajl::HttpStream to request responses compressed as deflate and bzip2 in addition to gzip
|
260
|
+
* fixed a bug regarding parsing Integers as Floats (so 123456 would have be parsed and returned as 123456.0)
|
261
|
+
* fixed a bug which caused a segfault in ruby's GC during string replacement in Yajl::Gzip and Yajl::Bzip2's StreamReader#read methods
|
262
|
+
* added support for user-specified User-Agent strings in Yajl::HttpStream
|
263
|
+
|
264
|
+
## 0.4.3 (May 2nd, 2009)
|
265
|
+
* adding text/plain as an allowed mime-type for Yajl::HttpStream for webservers that respond with it instead of application/json (ahem...Yelp...)
|
266
|
+
* renamed specs folder to spec for no reason at all
|
267
|
+
|
268
|
+
## 0.4.2 (April 30th, 2009)
|
269
|
+
* Yajl::HttpStream is now sending "proper" http request headers
|
270
|
+
* Yajl::HttpStream will request HTTP-Basic auth if credentials are provided in the passed URI
|
271
|
+
* cleanup requires
|
272
|
+
|
273
|
+
## 0.4.1 (April 30th, 2009)
|
274
|
+
* fixed a typo in the stream.rb benchmark file
|
275
|
+
* fixed a bug in Yajl::Stream.parse that was causing "strange" Ruby malloc errors on large files, with large strings
|
276
|
+
* added Yajl::GzipStreamReader as a wrapper around Zlib::GzipReader to allow for standard IO#read behavior
|
277
|
+
* this allows Yajl::Stream to read off of a Gzip stream directly
|
278
|
+
|
279
|
+
## 0.4.0 (April 29th, 2009)
|
280
|
+
* NOTE: Breaking API change:
|
281
|
+
* refactored Stream parsing methods out of Yajl::Native into Yajl::Stream
|
282
|
+
* removed Yajl::Native namespace/module
|
283
|
+
* Addition of Yajl::HttpStream module
|
284
|
+
* This module is for streaming JSON HTTP responses directly into Yajl (as they're being received) for increased awesomeness
|
285
|
+
* it currently supports basic get requests with Yajl::HttpStream.get(uri)
|
286
|
+
* it also supports (and prefers) output compressed (gzip) responses
|
287
|
+
* Addition Yajl::Chunked module
|
288
|
+
* This module is for feeding Yajl JSON pieces at a time, instead of an entire IO object
|
289
|
+
* This works very well in environments like an EventMachine app where data is received in chunks by design
|
290
|
+
* decreased read buffer for Yajl::Stream from 8kb to 4kb
|
291
|
+
|
292
|
+
## 0.3.4 (April 24th, 2009)
|
293
|
+
* turned Unicode checks back on in the Yajl parser now that it's fixed (thanks Lloyd!)
|
294
|
+
* this also bumps the yajl version dependency requirement to 1.0.4
|
295
|
+
* better guessing of Integer/Float from number found instead of just trying to create a BigNum no matter what
|
296
|
+
* changed extconf.rb to fail Makefile creation if yajl isn't found
|
297
|
+
* added a test to check for parsing Infinity due to a Float overflow
|
298
|
+
|
299
|
+
## 0.3.3 (April 24th, 2009)
|
300
|
+
* 1.9 compatibility
|
301
|
+
|
302
|
+
## 0.3.2 (April 24th, 2009)
|
303
|
+
* version bump: forgot to include yajl.c in the gem
|
304
|
+
|
305
|
+
## 0.3.1 (April 23rd, 2009)
|
306
|
+
* fixed borked gemspec
|
307
|
+
|
308
|
+
## 0.3.0 (April 23rd, 2009)
|
309
|
+
* slight refactor of ActiveSupport tests to better reflect how they actually exist in ActiveSupport
|
310
|
+
* typo correction in the changelog which had the years in 2008
|
311
|
+
* added some initial spec tests
|
312
|
+
* ported some from ActiveSupport to ensure proper compatibility
|
313
|
+
* included 57 JSON fixtures to test against, all of which pass
|
314
|
+
* changed parser config to not check for invalid unicode characters as Ruby is going to do this anyway (?). This resolves the remaining test failures around unicode.
|
315
|
+
* changed how the parser was dealing with numbers to prevent overflows
|
316
|
+
* added an exception class Yajl::ParseError which is now used in place of simply printing to STDERR upon a parsing error
|
317
|
+
* renamed a couple of JSON test files in the benchmark folder to better represent their contents
|
318
|
+
* misc README updates
|
319
|
+
|
320
|
+
## 0.2.1 (April 23rd, 2009)
|
321
|
+
* fixed parsing bug - also fixed failing ActiveSupport test failures (except for the unicode one, which is an issue in Yajl itself)
|
322
|
+
|
323
|
+
## 0.2.0 (April 22nd, 2009)
|
324
|
+
* updated gemspec and README
|
325
|
+
|
326
|
+
## 0.1.0 (April 21st, 2009)
|
327
|
+
* initial release - gemified
|
data/Gemfile
ADDED
data/MIT-LICENSE
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright (c) 2008-2011 Brian Lopez - http://github.com/brianmario
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
"Software"), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
17
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
18
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
19
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
20
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,362 @@
|
|
1
|
+
# YAJL C Bindings for Ruby
|
2
|
+
|
3
|
+
This gem is a C binding to the excellent YAJL JSON parsing and generation library.
|
4
|
+
|
5
|
+
You can read more info at the project's website http://lloyd.github.com/yajl or check out its code at http://github.com/lloyd/yajl.
|
6
|
+
|
7
|
+
## Features
|
8
|
+
|
9
|
+
* JSON parsing and encoding directly to and from an IO stream (file, socket, etc) or String. Compressed stream parsing and encoding supported for Bzip2, Gzip and Deflate.
|
10
|
+
* Parse and encode *multiple* JSON objects to and from streams or strings continuously.
|
11
|
+
* JSON gem compatibility API - allows yajl-ruby to be used as a drop-in replacement for the JSON gem
|
12
|
+
* Basic HTTP client (only GET requests supported for now) which parses JSON directly off the response body *as it's being received*
|
13
|
+
* ~3.5x faster than JSON.generate
|
14
|
+
* ~1.9x faster than JSON.parse
|
15
|
+
* ~4.5x faster than YAML.load
|
16
|
+
* ~377.5x faster than YAML.dump
|
17
|
+
* ~1.5x faster than Marshal.load
|
18
|
+
* ~2x faster than Marshal.dump
|
19
|
+
|
20
|
+
## How to install
|
21
|
+
|
22
|
+
Go ahead and install it as usual:
|
23
|
+
|
24
|
+
```
|
25
|
+
gem install yajl-ruby
|
26
|
+
```
|
27
|
+
|
28
|
+
## Example of use
|
29
|
+
|
30
|
+
NOTE: I'm building up a collection of small examples in the examples (http://github.com/brianmario/yajl-ruby/tree/master/examples) folder.
|
31
|
+
|
32
|
+
First, you're probably gonna want to require it:
|
33
|
+
|
34
|
+
``` ruby
|
35
|
+
require 'yajl'
|
36
|
+
```
|
37
|
+
|
38
|
+
### Parsing
|
39
|
+
|
40
|
+
Then maybe parse some JSON from:
|
41
|
+
|
42
|
+
a File IO
|
43
|
+
|
44
|
+
``` ruby
|
45
|
+
json = File.new('test.json', 'r')
|
46
|
+
parser = Yajl::Parser.new
|
47
|
+
hash = parser.parse(json)
|
48
|
+
```
|
49
|
+
|
50
|
+
or maybe a StringIO
|
51
|
+
|
52
|
+
``` ruby
|
53
|
+
json = StringIO.new("...some JSON...")
|
54
|
+
parser = Yajl::Parser.new
|
55
|
+
hash = parser.parse(json)
|
56
|
+
```
|
57
|
+
|
58
|
+
or maybe STDIN
|
59
|
+
|
60
|
+
```
|
61
|
+
cat someJsonFile.json | ruby -ryajl -e "puts Yajl::Parser.parse(STDIN).inspect"
|
62
|
+
```
|
63
|
+
|
64
|
+
Or lets say you didn't have access to the IO object that contained JSON data, but instead
|
65
|
+
only had access to chunks of it at a time. No problem!
|
66
|
+
|
67
|
+
(Assume we're in an EventMachine::Connection instance)
|
68
|
+
|
69
|
+
``` ruby
|
70
|
+
def post_init
|
71
|
+
@parser = Yajl::Parser.new(:symbolize_keys => true)
|
72
|
+
end
|
73
|
+
|
74
|
+
def object_parsed(obj)
|
75
|
+
puts "Sometimes one pays most for the things one gets for nothing. - Albert Einstein"
|
76
|
+
puts obj.inspect
|
77
|
+
end
|
78
|
+
|
79
|
+
def connection_completed
|
80
|
+
# once a full JSON object has been parsed from the stream
|
81
|
+
# object_parsed will be called, and passed the constructed object
|
82
|
+
@parser.on_parse_complete = method(:object_parsed)
|
83
|
+
end
|
84
|
+
|
85
|
+
def receive_data(data)
|
86
|
+
# continue passing chunks
|
87
|
+
@parser << data
|
88
|
+
end
|
89
|
+
```
|
90
|
+
|
91
|
+
Or if you don't need to stream it, it'll just return the built object from the parse when it's done.
|
92
|
+
NOTE: if there are going to be multiple JSON strings in the input, you *must* specify a block or callback as this
|
93
|
+
is how yajl-ruby will hand you (the caller) each object as it's parsed off the input.
|
94
|
+
|
95
|
+
``` ruby
|
96
|
+
obj = Yajl::Parser.parse(str_or_io)
|
97
|
+
```
|
98
|
+
|
99
|
+
Or how about a JSON API HTTP request?
|
100
|
+
This actually makes a request using a raw TCPSocket, then parses the JSON body right off the socket. While it's being received over the wire!
|
101
|
+
|
102
|
+
``` ruby
|
103
|
+
require 'uri'
|
104
|
+
require 'yajl/http_stream'
|
105
|
+
|
106
|
+
url = URI.parse("http://search.twitter.com/search.json?q=engineyard")
|
107
|
+
results = Yajl::HttpStream.get(url)
|
108
|
+
```
|
109
|
+
|
110
|
+
Or do the same request, with Gzip and Deflate output compression support (also supports Bzip2, if loaded):
|
111
|
+
(this does the same raw socket Request, but transparently parses the compressed response body)
|
112
|
+
|
113
|
+
``` ruby
|
114
|
+
require 'uri'
|
115
|
+
require 'yajl/gzip'
|
116
|
+
require 'yajl/deflate'
|
117
|
+
require 'yajl/http_stream'
|
118
|
+
|
119
|
+
url = URI.parse("http://search.twitter.com/search.json?q=engineyard")
|
120
|
+
results = Yajl::HttpStream.get(url)
|
121
|
+
```
|
122
|
+
|
123
|
+
Since yajl-ruby parses JSON as a stream, supporting API's like Twitter's Streaming API are a piece-of-cake.
|
124
|
+
You can simply supply a block to `Yajl::HttpStream.get`, which is used as the callback for when a JSON object has been
|
125
|
+
unserialized off the stream. For the case of this Twitter Streaming API call, the callback gets fired a few times a second (depending on your connection speed).
|
126
|
+
The code below is all that's needed to make the request and stream unserialized Ruby hashes off the response, continuously.
|
127
|
+
You'll note that I've enabled the :symbolize_keys parser option as well. Doing so is much more efficient for parsing JSON streams with
|
128
|
+
lots of repetitive keys - for things like result sets or multiple API requests - than the same parse with string keys.
|
129
|
+
This is because Ruby will reuse (and never GC) its symbol table. Be that as it may, if you want to parse JSON strings with random key names
|
130
|
+
it's much better to leave string keys enabled (the default), so they can get GC'd later.
|
131
|
+
|
132
|
+
``` ruby
|
133
|
+
require 'uri'
|
134
|
+
require 'yajl/http_stream'
|
135
|
+
|
136
|
+
uri = URI.parse("http://#{username}:#{password}@stream.twitter.com/spritzer.json")
|
137
|
+
Yajl::HttpStream.get(uri, :symbolize_keys => true) do |hash|
|
138
|
+
puts hash.inspect
|
139
|
+
end
|
140
|
+
```
|
141
|
+
|
142
|
+
Or how about parsing directly from a compressed file?
|
143
|
+
|
144
|
+
``` ruby
|
145
|
+
require 'yajl/bzip2'
|
146
|
+
|
147
|
+
file = File.new('some.json.bz2', 'r')
|
148
|
+
result = Yajl::Bzip2::StreamReader.parse(file)
|
149
|
+
```
|
150
|
+
|
151
|
+
### Encoding
|
152
|
+
|
153
|
+
Since yajl-ruby does everything using streams, you simply need to pass the object to encode, and the IO to write the stream to (this happens in chunks).
|
154
|
+
|
155
|
+
This allows you to encode JSON as a stream, writing directly to a socket
|
156
|
+
|
157
|
+
``` ruby
|
158
|
+
socket = TCPSocket.new('192.168.1.101', 9000)
|
159
|
+
hash = {:foo => 12425125, :bar => "some string", ... }
|
160
|
+
encoder = Yajl::Encoder.new
|
161
|
+
Yajl::Encoder.encode(hash, socket)
|
162
|
+
```
|
163
|
+
|
164
|
+
Or what if you wanted to compress the stream over the wire?
|
165
|
+
|
166
|
+
``` ruby
|
167
|
+
require 'yajl/gzip'
|
168
|
+
socket = TCPSocket.new('192.168.1.101', 9000)
|
169
|
+
hash = {:foo => 12425125, :bar => "some string", ... }
|
170
|
+
Yajl::Gzip::StreamWriter.encode(hash, socket)
|
171
|
+
```
|
172
|
+
|
173
|
+
Or what about encoding multiple objects to JSON over the same stream?
|
174
|
+
This example will encode and send 50 JSON objects over the same stream, continuously.
|
175
|
+
|
176
|
+
``` ruby
|
177
|
+
socket = TCPSocket.new('192.168.1.101', 9000)
|
178
|
+
encoder = Yajl::Encoder.new
|
179
|
+
50.times do
|
180
|
+
hash = {:current_time => Time.now.to_f, :foo => 12425125}
|
181
|
+
encoder.encode(hash, socket)
|
182
|
+
end
|
183
|
+
```
|
184
|
+
|
185
|
+
Using `EventMachine` and you want to encode and send in chunks?
|
186
|
+
(Assume we're in an `EventMachine::Connection` instance)
|
187
|
+
|
188
|
+
``` ruby
|
189
|
+
def post_init
|
190
|
+
# Passing a :terminator character will let us determine when the encoder
|
191
|
+
# is done encoding the current object
|
192
|
+
@encoder = Yajl::Encoder.new
|
193
|
+
motd_contents = File.read("/path/to/motd.txt")
|
194
|
+
status = File.read("/path/to/huge/status_file.txt")
|
195
|
+
@motd = {:motd => motd_contents, :system_status => status}
|
196
|
+
end
|
197
|
+
|
198
|
+
def connection_completed
|
199
|
+
# The encoder will do its best to hand you data in chunks that
|
200
|
+
# are around 8kb (but you may see some that are larger)
|
201
|
+
#
|
202
|
+
# It should be noted that you could have also assigned the _on_progress_ callback
|
203
|
+
# much like you can assign the _on_parse_complete_ callback with the parser class.
|
204
|
+
# Passing a block (like below) essentially tells the encoder to use that block
|
205
|
+
# as the callback normally assigned to _on_progress_.
|
206
|
+
#
|
207
|
+
# Send our MOTD and status
|
208
|
+
@encoder.encode(@motd) do |chunk|
|
209
|
+
if chunk.nil? # got our terminator, encoding is done
|
210
|
+
close_connection_after_writing
|
211
|
+
else
|
212
|
+
send_data(chunk)
|
213
|
+
end
|
214
|
+
end
|
215
|
+
end
|
216
|
+
```
|
217
|
+
|
218
|
+
But to make things simple, you might just want to let yajl-ruby do all the hard work for you and just hand back
|
219
|
+
a string when it's finished. In that case, just don't provide and IO or block (or assign the on_progress callback).
|
220
|
+
|
221
|
+
``` ruby
|
222
|
+
str = Yajl::Encoder.encode(obj)
|
223
|
+
```
|
224
|
+
|
225
|
+
You can also use `Yajl::Bzip2::StreamWriter` and `Yajl::Deflate::StreamWriter`. So you can pick whichever fits your CPU/bandwidth sweet-spot.
|
226
|
+
|
227
|
+
=== HTML Safety
|
228
|
+
|
229
|
+
If you plan on embedding the output from the encoder in the DOM, you'll want to make sure you use the html_safe option on the encoder. This will escape all '/' characters to ensure no closing tags can be injected, preventing XSS.
|
230
|
+
|
231
|
+
Meaning the following should be perfectly safe:
|
232
|
+
|
233
|
+
``` html
|
234
|
+
<script type="text/javascript">
|
235
|
+
var escaped_str = <%= Yajl::Encoder.encode("</script><script>alert('hi!');</script>", :html_safe => true) %>;
|
236
|
+
</script>
|
237
|
+
```
|
238
|
+
|
239
|
+
== JSON gem Compatibility API
|
240
|
+
|
241
|
+
The JSON gem compatibility API isn't enabled by default. You have to explicitly require it like so:
|
242
|
+
|
243
|
+
``` ruby
|
244
|
+
require 'yajl/json_gem'
|
245
|
+
```
|
246
|
+
|
247
|
+
That's right, you can just replace `"require 'json'"` with the line above and you're done!
|
248
|
+
|
249
|
+
This will require yajl-ruby itself, as well as enable its JSON gem compatibility API.
|
250
|
+
|
251
|
+
This includes the following API:
|
252
|
+
|
253
|
+
JSON.parse, JSON.generate, JSON.pretty_generate, JSON.load, JSON.dump
|
254
|
+
and all of the #to_json instance method overrides for Ruby's primitive objects
|
255
|
+
|
256
|
+
|
257
|
+
Once the compatibility API is enabled, your existing or new project should work as if the JSON gem itself were being used. Only you'll be using Yajl ;)
|
258
|
+
|
259
|
+
There are a lot more possibilities that I'd love to see other gems/plugins for someday.
|
260
|
+
|
261
|
+
Some ideas:
|
262
|
+
|
263
|
+
* parsing logs in JSON format
|
264
|
+
* a Rails plugin - DONE! (http://github.com/technoweenie/yajl-rails)
|
265
|
+
* official support in Rails 3 - DONE (http://github.com/rails/rails/commit/a96bf4ab5e73fccdafb78b99e8a122cc2172b505)
|
266
|
+
* and is the default (if installed) - http://github.com/rails/rails/commit/63bb955a99eb46e257655c93dd64e86ebbf05651
|
267
|
+
* Rack middleware (ideally the JSON body could be handed to the parser while it's still being received, this is apparently possible with Unicorn)
|
268
|
+
* JSON API clients (http://github.com/brianmario/freckle-api)
|
269
|
+
|
270
|
+
## Benchmarks
|
271
|
+
|
272
|
+
After I finished implementation - this library performs close to the same as the current JSON.parse (C gem) does on small/medium files.
|
273
|
+
|
274
|
+
But on larger files, and higher amounts of iteration, this library was around 2x faster than JSON.parse.
|
275
|
+
|
276
|
+
The main benefit of this library is in its memory usage.
|
277
|
+
Since it's able to parse the stream in chunks, its memory requirements are very, very low.
|
278
|
+
|
279
|
+
Here's what parsing a 2.43MB JSON file off the filesystem 20 times looks like:
|
280
|
+
|
281
|
+
### Memory Usage
|
282
|
+
|
283
|
+
#### Average
|
284
|
+
|
285
|
+
* Yajl::Parser#parse: 32MB
|
286
|
+
* JSON.parse: 54MB
|
287
|
+
* ActiveSupport::JSON.decode: 63MB
|
288
|
+
|
289
|
+
#### Peak
|
290
|
+
|
291
|
+
* Yajl::Parser#parse: 32MB
|
292
|
+
* JSON.parse: 57MB
|
293
|
+
* ActiveSupport::JSON.decode: 67MB
|
294
|
+
|
295
|
+
### Parse Time
|
296
|
+
|
297
|
+
* Yajl::Parser#parse: 4.54s
|
298
|
+
* JSON.parse: 5.47s
|
299
|
+
* ActiveSupport::JSON.decode: 64.42s
|
300
|
+
|
301
|
+
### Encode Time
|
302
|
+
|
303
|
+
* Yajl::Encoder#encode: 3.59s
|
304
|
+
* JSON#to_json: 6.2s
|
305
|
+
* ActiveSupport::JSON.encode: 45.58s
|
306
|
+
|
307
|
+
### Compared to YAML
|
308
|
+
|
309
|
+
NOTE: I converted the 2.4MB JSON file to YAML for this test.
|
310
|
+
|
311
|
+
#### Parse Time (from their respective formats)
|
312
|
+
|
313
|
+
* Yajl::Parser#parse: 4.33s
|
314
|
+
* JSON.parse: 5.37s
|
315
|
+
* YAML.load: 19.47s
|
316
|
+
|
317
|
+
#### Encode Time (to their respective formats)
|
318
|
+
|
319
|
+
* Yajl::Encoder#encode: 3.47s
|
320
|
+
* JSON#to_json: 6.6s
|
321
|
+
* YAML.dump(obj, io): 1309.93s
|
322
|
+
|
323
|
+
### Compared to Marshal.load/Marshal.dump
|
324
|
+
|
325
|
+
NOTE: I converted the 2.4MB JSON file to a Hash and a dump file from Marshal.dump for this test.
|
326
|
+
|
327
|
+
#### Parse Time (from their respective formats)
|
328
|
+
|
329
|
+
* Yajl::Parser#parse: 4.54s
|
330
|
+
* JSON.parse: 7.40s
|
331
|
+
* Marshal.load: 7s
|
332
|
+
|
333
|
+
#### Encode Time (to their respective formats)
|
334
|
+
|
335
|
+
* Yajl::Encoder#encode: 2.39s
|
336
|
+
* JSON#to_json: 8.37s
|
337
|
+
* Marshal.dump: 4.66s
|
338
|
+
|
339
|
+
## Third Party Sources Bundled
|
340
|
+
|
341
|
+
This project includes code from the BSD licensed yajl project, copyright 2007-2009 Lloyd Hilaiel
|
342
|
+
|
343
|
+
## Special Thanks & Contributors
|
344
|
+
|
345
|
+
For those of you using yajl-ruby out in the wild, please hit me up on Twitter (brianmario) or send me a message here on the Githubs describing the site and how you're using it. I'd love to get a list going!
|
346
|
+
|
347
|
+
I've had a lot of inspiration, and a lot of help. Thanks to everyone who's been a part of this and those to come!
|
348
|
+
|
349
|
+
* Lloyd Hilaiel - http://github.com/lloyd - for writing Yajl!!
|
350
|
+
* Josh Ferguson - http://github.com/besquared - for peer-pressuring me into getting back into C; it worked ;) Also tons of support over IM
|
351
|
+
* Jonathan Novak - http://github.com/cypriss - pointer-hacking help
|
352
|
+
* Tom Smith - http://github.com/rtomsmith - pointer-hacking help
|
353
|
+
* Rick Olson - http://github.com/technoweenie - for making an ActiveSupport patch with support for this library and teasing me that it might go into Rails 3. You sure lit a fire under my ass and I got a ton of work done because of it! :)
|
354
|
+
* The entire Github Crew - http://github.com/ - my inspiration, time spent writing this, finding Yajl, So many-MANY other things wouldn't have been possible without this awesome service. I owe you guys some whiskey at Kilowatt.
|
355
|
+
* Ben Burkert - http://github.com/benburkert
|
356
|
+
* Aman Gupta - http://github.com/tmm1 - tons of suggestions and inspiration for the most recent features, and hopefully more to come ;)
|
357
|
+
* Filipe Giusti
|
358
|
+
* Jonathan George
|
359
|
+
* Luke Redpath
|
360
|
+
* Neil Berkman
|
361
|
+
* Pavel Valodzka
|
362
|
+
* Rob Sharp
|