yajl-ruby 1.0.0-x86-mswin32-60
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of yajl-ruby might be problematic. Click here for more details.
- data/.gitignore +12 -0
- data/.rspec +2 -0
- data/CHANGELOG.md +327 -0
- data/Gemfile +3 -0
- data/MIT-LICENSE +20 -0
- data/README.md +362 -0
- data/Rakefile +2 -0
- data/benchmark/encode.rb +72 -0
- data/benchmark/encode_json_and_marshal.rb +42 -0
- data/benchmark/encode_json_and_yaml.rb +53 -0
- data/benchmark/http.rb +32 -0
- data/benchmark/parse.rb +94 -0
- data/benchmark/parse_json_and_marshal.rb +50 -0
- data/benchmark/parse_json_and_yaml.rb +55 -0
- data/benchmark/parse_stream.rb +54 -0
- data/benchmark/subjects/item.json +1 -0
- data/benchmark/subjects/ohai.json +1216 -0
- data/benchmark/subjects/ohai.marshal_dump +0 -0
- data/benchmark/subjects/ohai.yml +975 -0
- data/benchmark/subjects/twitter_search.json +1 -0
- data/benchmark/subjects/twitter_stream.json +430 -0
- data/benchmark/subjects/unicode.json +1 -0
- data/examples/encoding/chunked_encoding.rb +27 -0
- data/examples/encoding/one_shot.rb +13 -0
- data/examples/encoding/to_an_io.rb +12 -0
- data/examples/http/twitter_search_api.rb +12 -0
- data/examples/http/twitter_stream_api.rb +26 -0
- data/examples/parsing/from_file.rb +14 -0
- data/examples/parsing/from_stdin.rb +9 -0
- data/examples/parsing/from_string.rb +13 -0
- data/ext/yajl/api/yajl_common.h +89 -0
- data/ext/yajl/api/yajl_gen.h +161 -0
- data/ext/yajl/api/yajl_parse.h +196 -0
- data/ext/yajl/api/yajl_version.h +23 -0
- data/ext/yajl/extconf.rb +7 -0
- data/ext/yajl/yajl.c +164 -0
- data/ext/yajl/yajl_alloc.c +65 -0
- data/ext/yajl/yajl_alloc.h +50 -0
- data/ext/yajl/yajl_buf.c +119 -0
- data/ext/yajl/yajl_buf.h +73 -0
- data/ext/yajl/yajl_bytestack.h +85 -0
- data/ext/yajl/yajl_encode.c +201 -0
- data/ext/yajl/yajl_encode.h +52 -0
- data/ext/yajl/yajl_ext.c +905 -0
- data/ext/yajl/yajl_ext.h +135 -0
- data/ext/yajl/yajl_gen.c +344 -0
- data/ext/yajl/yajl_lex.c +748 -0
- data/ext/yajl/yajl_lex.h +135 -0
- data/ext/yajl/yajl_parser.c +450 -0
- data/ext/yajl/yajl_parser.h +82 -0
- data/ext/yajl/yajl_version.c +7 -0
- data/lib/yajl.rb +75 -0
- data/lib/yajl/1.8/yajl.so +0 -0
- data/lib/yajl/1.9/yajl.so +0 -0
- data/lib/yajl/bzip2.rb +11 -0
- data/lib/yajl/bzip2/stream_reader.rb +31 -0
- data/lib/yajl/bzip2/stream_writer.rb +14 -0
- data/lib/yajl/deflate.rb +6 -0
- data/lib/yajl/deflate/stream_reader.rb +43 -0
- data/lib/yajl/deflate/stream_writer.rb +20 -0
- data/lib/yajl/gzip.rb +6 -0
- data/lib/yajl/gzip/stream_reader.rb +30 -0
- data/lib/yajl/gzip/stream_writer.rb +13 -0
- data/lib/yajl/http_stream.rb +212 -0
- data/lib/yajl/json_gem.rb +15 -0
- data/lib/yajl/json_gem/encoding.rb +51 -0
- data/lib/yajl/json_gem/parsing.rb +26 -0
- data/lib/yajl/version.rb +3 -0
- data/lib/yajl/yajl.rb +2 -0
- data/spec/encoding/encoding_spec.rb +271 -0
- data/spec/global/global_spec.rb +54 -0
- data/spec/http/fixtures/http.bzip2.dump +0 -0
- data/spec/http/fixtures/http.chunked.dump +11 -0
- data/spec/http/fixtures/http.deflate.dump +0 -0
- data/spec/http/fixtures/http.error.dump +12 -0
- data/spec/http/fixtures/http.gzip.dump +0 -0
- data/spec/http/fixtures/http.html.dump +1220 -0
- data/spec/http/fixtures/http.raw.dump +1226 -0
- data/spec/http/http_delete_spec.rb +98 -0
- data/spec/http/http_error_spec.rb +32 -0
- data/spec/http/http_get_spec.rb +109 -0
- data/spec/http/http_post_spec.rb +123 -0
- data/spec/http/http_put_spec.rb +105 -0
- data/spec/http/http_stream_options_spec.rb +27 -0
- data/spec/json_gem_compatibility/compatibility_spec.rb +203 -0
- data/spec/parsing/active_support_spec.rb +64 -0
- data/spec/parsing/chunked_spec.rb +96 -0
- data/spec/parsing/fixtures/fail.15.json +1 -0
- data/spec/parsing/fixtures/fail.16.json +1 -0
- data/spec/parsing/fixtures/fail.17.json +1 -0
- data/spec/parsing/fixtures/fail.26.json +1 -0
- data/spec/parsing/fixtures/fail11.json +1 -0
- data/spec/parsing/fixtures/fail12.json +1 -0
- data/spec/parsing/fixtures/fail13.json +1 -0
- data/spec/parsing/fixtures/fail14.json +1 -0
- data/spec/parsing/fixtures/fail19.json +1 -0
- data/spec/parsing/fixtures/fail20.json +1 -0
- data/spec/parsing/fixtures/fail21.json +1 -0
- data/spec/parsing/fixtures/fail22.json +1 -0
- data/spec/parsing/fixtures/fail23.json +1 -0
- data/spec/parsing/fixtures/fail24.json +1 -0
- data/spec/parsing/fixtures/fail25.json +1 -0
- data/spec/parsing/fixtures/fail27.json +2 -0
- data/spec/parsing/fixtures/fail28.json +2 -0
- data/spec/parsing/fixtures/fail3.json +1 -0
- data/spec/parsing/fixtures/fail4.json +1 -0
- data/spec/parsing/fixtures/fail5.json +1 -0
- data/spec/parsing/fixtures/fail6.json +1 -0
- data/spec/parsing/fixtures/fail9.json +1 -0
- data/spec/parsing/fixtures/pass.array.json +6 -0
- data/spec/parsing/fixtures/pass.codepoints_from_unicode_org.json +1 -0
- data/spec/parsing/fixtures/pass.contacts.json +1 -0
- data/spec/parsing/fixtures/pass.db100.xml.json +1 -0
- data/spec/parsing/fixtures/pass.db1000.xml.json +1 -0
- data/spec/parsing/fixtures/pass.dc_simple_with_comments.json +11 -0
- data/spec/parsing/fixtures/pass.deep_arrays.json +1 -0
- data/spec/parsing/fixtures/pass.difficult_json_c_test_case.json +1 -0
- data/spec/parsing/fixtures/pass.difficult_json_c_test_case_with_comments.json +1 -0
- data/spec/parsing/fixtures/pass.doubles.json +1 -0
- data/spec/parsing/fixtures/pass.empty_array.json +1 -0
- data/spec/parsing/fixtures/pass.empty_string.json +1 -0
- data/spec/parsing/fixtures/pass.escaped_bulgarian.json +4 -0
- data/spec/parsing/fixtures/pass.escaped_foobar.json +1 -0
- data/spec/parsing/fixtures/pass.item.json +1 -0
- data/spec/parsing/fixtures/pass.json-org-sample1.json +23 -0
- data/spec/parsing/fixtures/pass.json-org-sample2.json +11 -0
- data/spec/parsing/fixtures/pass.json-org-sample3.json +26 -0
- data/spec/parsing/fixtures/pass.json-org-sample4-nows.json +88 -0
- data/spec/parsing/fixtures/pass.json-org-sample4.json +89 -0
- data/spec/parsing/fixtures/pass.json-org-sample5.json +27 -0
- data/spec/parsing/fixtures/pass.map-spain.xml.json +1 -0
- data/spec/parsing/fixtures/pass.ns-invoice100.xml.json +1 -0
- data/spec/parsing/fixtures/pass.ns-soap.xml.json +1 -0
- data/spec/parsing/fixtures/pass.numbers-fp-4k.json +6 -0
- data/spec/parsing/fixtures/pass.numbers-fp-64k.json +61 -0
- data/spec/parsing/fixtures/pass.numbers-int-4k.json +11 -0
- data/spec/parsing/fixtures/pass.numbers-int-64k.json +154 -0
- data/spec/parsing/fixtures/pass.twitter-search.json +1 -0
- data/spec/parsing/fixtures/pass.twitter-search2.json +1 -0
- data/spec/parsing/fixtures/pass.unicode.json +3315 -0
- data/spec/parsing/fixtures/pass.yelp.json +1 -0
- data/spec/parsing/fixtures/pass1.json +56 -0
- data/spec/parsing/fixtures/pass2.json +1 -0
- data/spec/parsing/fixtures/pass3.json +6 -0
- data/spec/parsing/fixtures_spec.rb +40 -0
- data/spec/parsing/one_off_spec.rb +85 -0
- data/spec/rcov.opts +3 -0
- data/spec/spec_helper.rb +16 -0
- data/tasks/compile.rake +35 -0
- data/tasks/rspec.rake +16 -0
- data/yajl-ruby.gemspec +24 -0
- metadata +335 -0
data/.gitignore
ADDED
data/.rspec
ADDED
data/CHANGELOG.md
ADDED
@@ -0,0 +1,327 @@
|
|
1
|
+
# Changelog
|
2
|
+
|
3
|
+
## 1.0.0 (September 13th, 2011)
|
4
|
+
* add deprecation notice for Yajl's Bzip2 support
|
5
|
+
* add deprecation notice for Yajl's Deflate support
|
6
|
+
* add deprecation notice for Yajl's Gzip support
|
7
|
+
* add deprecation notice for Yajl's JSON gem compatibility API
|
8
|
+
* add deprecation notice for Yajl::HttpStream
|
9
|
+
* change the path the extension is copied into to be 'lib/yajl'
|
10
|
+
* remove 'ext' from the loadpath
|
11
|
+
|
12
|
+
## 0.8.3 (August 16th, 2011)
|
13
|
+
* fix bug where Yajl::HttpStream wouldn't pass through a user-specified socket
|
14
|
+
* fix incorrect Ruby initialization hook method name
|
15
|
+
* Bump bundled YAJL version to 1.0.12
|
16
|
+
* fix to correctly symbolize multibyte characters on 1.9
|
17
|
+
* add `:headers` option to Yajl::HttpStream for user-specified arbitrary headers
|
18
|
+
|
19
|
+
## 0.8.2 (March 22nd, 2011)
|
20
|
+
* define RSTRING_NOT_MODIFIED for rbx to prevent string caching, making things A LOT faster (100x)
|
21
|
+
|
22
|
+
## 0.8.1 (February 11th, 2011)
|
23
|
+
* fixed a retart bug where Yajl::VERSION wasn't defined when explicitly requiring yajl/http_stream
|
24
|
+
|
25
|
+
## 0.8.0 (February 2nd, 2011)
|
26
|
+
* added a new html_safe option to Yajl::Encoder to escape '/' characters for use in the DOM
|
27
|
+
* moved away from Jeweler to a Bundler/manual gemfile management setup
|
28
|
+
|
29
|
+
## 0.7.9 (January 11th, 2011)
|
30
|
+
* moved to rspec2
|
31
|
+
* fixed some compilation warnings on 1.9.3
|
32
|
+
* brought over latest from Yajl upstream
|
33
|
+
* finally removed the deprecated Yajl::Stream methods
|
34
|
+
* moved to rake-compiler
|
35
|
+
* moved to Bundler for development
|
36
|
+
* fix memory corruption bug when using :pretty => true and a custom indent string
|
37
|
+
* fixed memory leak when exceptions were being raised during a parse
|
38
|
+
|
39
|
+
## 0.7.8 (September 27th, 2010)
|
40
|
+
* fix a bug in chunked http response regex (thanks to http://github.com/kevn for catching this)
|
41
|
+
* Make sure json compability doesn't break ActiveSupport#to_json
|
42
|
+
* fix improper usage of rb_define_method
|
43
|
+
|
44
|
+
## 0.7.7 (July 12th, 2010)
|
45
|
+
* full string encoding support for 1.9, respecting Encoding.default_internal
|
46
|
+
* refactor the #to_json function bodies into a C macro
|
47
|
+
* some misc code cleanup in the benchmark scripts
|
48
|
+
|
49
|
+
## 0.7.6 (May 1st, 2010)
|
50
|
+
* use memcmp instead of strcmp for invalid Fixnum check
|
51
|
+
* add a spec to verify unicode chars can be used as keys
|
52
|
+
* twitter examples updated
|
53
|
+
* only use -Wextra if ENV['DEBUG'] is set as gcc 3 doesn't know about it
|
54
|
+
* fix chunked http encoding parse logic to further comply with the spec (thanks to Sebastian Cohnen <sebastian.cohnen@gmx.net>)
|
55
|
+
* removed as_json checks and usage in encoder to prevent infinite loops
|
56
|
+
** In Rails a lot of objects return self from the as_json method - which is wrong IMO - and needs to be fixed before this feature will work properly
|
57
|
+
|
58
|
+
## 0.7.5 (March 23rd, 2010)
|
59
|
+
* check for existence of and use as_json method on custom objects
|
60
|
+
* bugfix with read buffer when parsing from an IO (thanks to Pavel Valodzka <pavel@valodzka.name>)
|
61
|
+
* merged in latest yajl
|
62
|
+
* enable -Wextra during compilation
|
63
|
+
* brought back ability to pass a buffer to bzip/gzip/deflate #read helper methods
|
64
|
+
|
65
|
+
## 0.7.4 (March 3rd, 2010)
|
66
|
+
* bugfix for the JSON gem compatibility API's Object#to_json method improperly encoding strings
|
67
|
+
|
68
|
+
## 0.7.3 (February 23rd, 2010)
|
69
|
+
* remove trap from HttpStream code, it's really not needed
|
70
|
+
|
71
|
+
## 0.7.2 (February 23rd, 2010)
|
72
|
+
* fixed broken to_json compatibility
|
73
|
+
* removed strlen in a few places in favor of RSTRING_LEN since ruby already knows the length of the string
|
74
|
+
* patched Yajl to more efficiently reset it's lexer (no more malloc/free)
|
75
|
+
* removed dependency on IO#eof? when parsing from an IO for full Rack-spec compatibility
|
76
|
+
* removed some various cruft code in C
|
77
|
+
|
78
|
+
## 0.7.1 (February 17th, 2010)
|
79
|
+
* revert a patch made to bundled Yajl enabling optional quoting of strings that broke binary API compatibility
|
80
|
+
|
81
|
+
## 0.7.0 (February 5th, 2010)
|
82
|
+
* ensure utf8 encoding is set on relevant strings during parse/encode in 1.9
|
83
|
+
|
84
|
+
## 0.6.9 (January 26th, 2010)
|
85
|
+
* HttpStream patches merged in from Luke Redpath <contact@lukeredpath.co.uk>
|
86
|
+
* Changed how Yajl::Parser was calling IO#read to better conform to the Rack spec and thus can be used to directly parse a rack.input stream
|
87
|
+
|
88
|
+
## 0.6.8 (January 1st, 2010)
|
89
|
+
* A couple of small performance patches
|
90
|
+
* Allow passing a string to Yajl::HttpStream methods instead of only a URI
|
91
|
+
|
92
|
+
## 0.6.7 (December 4th, 2009)
|
93
|
+
* Bump internal version constant to the proper value (doh!)
|
94
|
+
* Bring over latest from Yajl upstream
|
95
|
+
|
96
|
+
## 0.6.6 (December 1st, 2009)
|
97
|
+
* Brought over some optimizations from Macruby's use of some yajl-ruby codez
|
98
|
+
* Yajl::HttpStream now supports being killed for long-running requests, thanks to Filipe Giusti <filipegiusti@gmail.com>
|
99
|
+
|
100
|
+
## 0.6.5 (November 13th, 2009)
|
101
|
+
* optimize symbol creation while symbolize_keys is turned on
|
102
|
+
* fix for 32bit integer conversion into ruby
|
103
|
+
|
104
|
+
## 0.6.4 (November 4th, 2009)
|
105
|
+
* All specs pass on Rubinius :)
|
106
|
+
* Added Yajl.load and Yajl.dump for compatibility with other various data format API's in ruby
|
107
|
+
* Fixed a bug in Yajl::Encoder which allowed direct, unescaped encoding of NaN, Infinity and -Infinity.
|
108
|
+
It will now properly throw a Yajl::EncodeError exception if either of these values are found unescaped.
|
109
|
+
* Update bundled Yajl library to 1.0.7
|
110
|
+
* Conditionally define RSTRING_* and RARRAY_* for older versions of ruby (1.8.5 is still the default on CentOS)
|
111
|
+
* Bugfix for JSON gem exception classes to more accurately match those of the actual JSON gem
|
112
|
+
* A few small speed optimizations
|
113
|
+
* Updated specs to not run bzip2 related examples if unable to load the bzip2 library
|
114
|
+
* Finally added UTF-8 checking specs
|
115
|
+
* Removed needless calls to ID2SYM all over the place
|
116
|
+
* Updated benchmark scripts to bring the GC into the picture a little more
|
117
|
+
|
118
|
+
## 0.6.3 (August 25th, 2009)
|
119
|
+
* Fixed a bug in the JSON gem compatibility API where strings weren't being properly escaped
|
120
|
+
|
121
|
+
## 0.6.2 (August 25th, 2009)
|
122
|
+
* Fixed a bug surfaced by an existing library providing a to_json method, and Yajl would double-quote the values provided
|
123
|
+
|
124
|
+
## 0.6.1 (August 20th, 2009)
|
125
|
+
* Fixed a bug in Yajl::HttpStream where responses contained multiple JSON strings but weren't Transfer-Encoding: chunked (thanks @dacort!)
|
126
|
+
|
127
|
+
## 0.6.0 (August 19th, 2009)
|
128
|
+
* Added POST, PUT and DELETE support to Yajl::HttpStream
|
129
|
+
** POST support initially contributed by jdg (http://github.com/jdg) - Although oortle (http://github.com/oortle) coded it up in a fork with it as well.
|
130
|
+
|
131
|
+
## 0.5.12 (July 31st, 2009)
|
132
|
+
* Add another option that can be passed to Yajl::Encoder's constructor (:terminator) to allow the caller some control over
|
133
|
+
when a full JSON string has been generated by the encoder. More information on it's use in the README
|
134
|
+
|
135
|
+
## 0.5.11 (July 14th, 2009)
|
136
|
+
* fixing a bug Aman found with to_json on non-primitive Ruby objects and double-quoting in the JSON compat API
|
137
|
+
|
138
|
+
## 0.5.10 (July 13th, 2009)
|
139
|
+
* Bugfix for the JSON gem compatibility API's default Object#to_json helper
|
140
|
+
|
141
|
+
## 0.5.9 (July 9th, 2009)
|
142
|
+
* Bugfix for Yajl::Encoder where encoding a hash like {:a => :b} would get stuck in an infinite loop
|
143
|
+
|
144
|
+
## 0.5.8 (July 6th, 2009)
|
145
|
+
* Bugfix in Yajl::HttpStream for proper handling of the Content-type header (Rob Sharp)
|
146
|
+
* Yajl::Encoder now has an on_progress callback setter, which can be used to harness the encoder's streaming ability.
|
147
|
+
** The passed Proc/lambda will be called, and passed every chunk (currently 8kb) of the encoded JSON string as it's being encoded.
|
148
|
+
* API CHANGE WARNING: Yajl::Encoder.encode's block will now be used as (and work the same as) the on_progress callback
|
149
|
+
** This means the block will be passed chunks of the JSON string at a time, giving the caller the ability to start processing the encoded data while it's still being encoded.
|
150
|
+
* fixed grammatical error in README (Neil Berkman)
|
151
|
+
* Added some encoder examples
|
152
|
+
|
153
|
+
## 0.5.7 (June 23rd, 2009)
|
154
|
+
* You can now pass parser options (like :symbolize_keys for example) to Yajl::HttpStream.get
|
155
|
+
* Refactored spec tests a bit, DRYing up the Yajl::HttpStream specs quite a bit.
|
156
|
+
* Added a spec rake task, and spec.opts file
|
157
|
+
* Updated and renamed rcov rake task, and added rcov.opts file
|
158
|
+
|
159
|
+
## 0.5.6 (June 19th, 2009)
|
160
|
+
* Added JSON.default_options hash to the JSON gem compatibility API
|
161
|
+
* Split out the JSON gem compatibility API's parsing and encoding methods into individually includable files
|
162
|
+
** the use case here is if you *only* want parsing, or *only* want encoding
|
163
|
+
** also, if you don't include encoding it won't include the #to_json overrides which tend to cause problems in some environments.
|
164
|
+
* Removed some large benchmark test files to reduce the size of the packaged gem by 1.5MB!
|
165
|
+
|
166
|
+
## 0.5.5 (June 17th, 2009)
|
167
|
+
* Introduction of the JSON gem compatibility API
|
168
|
+
** NOTE: this isn't a 1:1 compatibility API, the goal was to be compatible with as many of the projects using the JSON gem as possible - not the JSON gem API itself
|
169
|
+
** the compatibility API must be explicitly enabled by requiring 'yajl/json_gem' in your project
|
170
|
+
** JSON.parse, JSON.generate, and the #to_json instance method extension to ruby's primitive classes are all included
|
171
|
+
* Fix Yajl::Encoder to ensure map keys are strings
|
172
|
+
* Encoding multiple JSON objects to a single stream doesn't separate by a newline character anymore
|
173
|
+
* Yajl::Encoder now checks for the existence of, and will call #to_json on any non-primitive object
|
174
|
+
|
175
|
+
## 0.5.4 (June 16th, 2009)
|
176
|
+
* Yajl::Parser's :symbolize_keys option now defaults to false
|
177
|
+
* remove use of sprintf for a little speed improvement while parsing
|
178
|
+
|
179
|
+
## 0.5.3 (June 7th, 2009)
|
180
|
+
* The IO parameter for Yajl::Encode#encode is now optional, and accepts a block
|
181
|
+
** it will return the resulting JSON string if no IO is passed to stream to
|
182
|
+
** if a block is passed, it will call and pass it the resulting JSON string
|
183
|
+
* Yajl::Parser#parse can now parse from a String as well as an IO
|
184
|
+
* Added and updated lot of in-code documentation.
|
185
|
+
** all the C code exposed to Ruby should now have comments
|
186
|
+
* Added :symbolize_keys option to the Yajl::Parser class, which defaults to true.
|
187
|
+
** Having this option enabled has shown around an 18% speedup in parsing time according to my benchmarks
|
188
|
+
|
189
|
+
## 0.5.2 (May 30th, 2009)
|
190
|
+
* Added class helper methods Yajl::Encoder.encode(obj, io) and Yajl::Parser.parse(io)
|
191
|
+
* added tests for the above
|
192
|
+
* Updated Twitter streaming example to have a less verbose output
|
193
|
+
* Patch Yajl so encoding can continue as a stream
|
194
|
+
* IE: multiple objects encoded onto the same IO
|
195
|
+
* added a test for the above
|
196
|
+
* Set the internal read buffer size back down to 8kb by default
|
197
|
+
* Added an internal write buffer size (set to 8kb by default) which is used to throttle writes to the output stream
|
198
|
+
* This is to fix a major performance bug/issue with the IO#write C method in ruby 1.9.x (I've opened a bug with them about it)
|
199
|
+
* Fixed a typo in a one-off parsing spec test
|
200
|
+
* Updated benchmarks to work properly in 1.9 (required removal ActiveSupport benchmarking for now)
|
201
|
+
* Updated spec tests to respect ridiculous differences in hash key ordering between 1.8 and 1.9
|
202
|
+
|
203
|
+
|
204
|
+
## 0.5.1 (May 25th, 2009)
|
205
|
+
* added some more tests for the new API
|
206
|
+
* inlined a couple of hot functions used in parsing for a little speedup
|
207
|
+
* updates to readme, reflecting changes in API
|
208
|
+
* version bump to push another gem build
|
209
|
+
|
210
|
+
## 0.5.0 (May 25th, 2009)
|
211
|
+
* Refactored internal API so the caller can specify initialization options for the Parser and Encoder respectively. Two new classes were introduced as a result - Yajl::Parser and Yajl::Encoder. The newly refactored codebase is cleaner, thread-safe and removed all of the hack-code that was trickled around to make things work in the previous implementation. She's much more seaworthy now cap'n!
|
212
|
+
* Yajl::Parser.new accepts two options, :allow_comments and :check_utf8 which both default to true
|
213
|
+
* Yajl::Encoder.new accepts two options, :pretty and :indent which default to false and " " respectively
|
214
|
+
* cleaned up a lot of state code, that to my knowledge prevented yajl-ruby from being used in a thread-safe environment.
|
215
|
+
* added deprecated messaging to Yajl::Stream.parse and Yajl::Stream.encode - these will likely go away before 0.6.0
|
216
|
+
* fixed a bug in the chunked http response parser regarding partially received chunks
|
217
|
+
* added a Twitter Search API example showing off the HttpStream API
|
218
|
+
|
219
|
+
## 0.4.9 (May 20th, 2009)
|
220
|
+
* fixed some parser state bugs surfaced by edge cases
|
221
|
+
* added support for Chunked HTTP response bodies in Yajl::HttpStream
|
222
|
+
* added support for passing a block to Yajl::HttpStream.get that will be used as a callback whenever a JSON object is parsed off the stream (even if there is more than one!)
|
223
|
+
* added an examples folder, and put an example using the Twitter Streaming API in there to start
|
224
|
+
* added some more spec tests, this time around Chunked parsing and continuously parsing multiple JSON strings
|
225
|
+
|
226
|
+
## 0.4.8 (May 18th, 2009)
|
227
|
+
* fixed a totally bone-head compilation problem, I created for myself ;)
|
228
|
+
|
229
|
+
## 0.4.7 (May 18th, 2009)
|
230
|
+
* Bundling Yajl sources to remove the need to install them (and CMake) separately (Thank you Lloyd!!!) This means you can now simply install the gem and be off and running
|
231
|
+
* Added some spec tests for Yajl::HttpStream
|
232
|
+
* Added some spec tests for Yajl::Stream.encode
|
233
|
+
* added some more thank you's, where credit's due - in the readme
|
234
|
+
* updated the unicode.json file to reflect a "real-life" JSON response
|
235
|
+
* reorganized spec tests into their functional areas
|
236
|
+
* added an rcov rake task to generate code coverage output
|
237
|
+
|
238
|
+
## 0.4.6 (May 17th, 2009)
|
239
|
+
* Applied a patch from benburkert (http://github.com/benburkert) to fix HTTP Basic Auth in Yajl::HttpStream.get
|
240
|
+
|
241
|
+
## 0.4.5 (May 17th, 2009)
|
242
|
+
* added Yajl::Stream.encode(hash, io)
|
243
|
+
* generates a JSON string stream, and writes to IO
|
244
|
+
* compressed StreamWriter helpers added as well
|
245
|
+
* fixed a pretty lame segfault in (x86_64 only?) ubuntu/linux
|
246
|
+
* changed the compiled extension to have a more specific name (yajl_ext) for easier loading
|
247
|
+
* removed forced-load of .bundle file, for the rest of the planet aside from OSX users
|
248
|
+
* added some more benchmarks to compare to other forms of serialization in Ruby
|
249
|
+
* various readme updates
|
250
|
+
|
251
|
+
## 0.4.4 (May 12th, 2009)
|
252
|
+
* NOTE: Breaking API change:
|
253
|
+
* renamed Yajl::GzipStreamReader to Yajl::Gzip::StreamReader
|
254
|
+
* added Yajl::Bzip2::StreamReader
|
255
|
+
* depends on the bzip2-ruby gem if you want to use it, if not Yajl::Bzip2 won't be loaded
|
256
|
+
* added Yajl::Deflate::StreamReader
|
257
|
+
* actually uses Zlib::Inflate for stream decompression
|
258
|
+
* added parse(io) class methods to Yajl::Gzip::StreamReader and Yajl::Bzip2::StreamReader as a helper for parsing compressed streams.
|
259
|
+
* updated Yajl::HttpStream to request responses compressed as deflate and bzip2 in addition to gzip
|
260
|
+
* fixed a bug regarding parsing Integers as Floats (so 123456 would have be parsed and returned as 123456.0)
|
261
|
+
* fixed a bug which caused a segfault in ruby's GC during string replacement in Yajl::Gzip and Yajl::Bzip2's StreamReader#read methods
|
262
|
+
* added support for user-specified User-Agent strings in Yajl::HttpStream
|
263
|
+
|
264
|
+
## 0.4.3 (May 2nd, 2009)
|
265
|
+
* adding text/plain as an allowed mime-type for Yajl::HttpStream for webservers that respond with it instead of application/json (ahem...Yelp...)
|
266
|
+
* renamed specs folder to spec for no reason at all
|
267
|
+
|
268
|
+
## 0.4.2 (April 30th, 2009)
|
269
|
+
* Yajl::HttpStream is now sending "proper" http request headers
|
270
|
+
* Yajl::HttpStream will request HTTP-Basic auth if credentials are provided in the passed URI
|
271
|
+
* cleanup requires
|
272
|
+
|
273
|
+
## 0.4.1 (April 30th, 2009)
|
274
|
+
* fixed a typo in the stream.rb benchmark file
|
275
|
+
* fixed a bug in Yajl::Stream.parse that was causing "strange" Ruby malloc errors on large files, with large strings
|
276
|
+
* added Yajl::GzipStreamReader as a wrapper around Zlib::GzipReader to allow for standard IO#read behavior
|
277
|
+
* this allows Yajl::Stream to read off of a Gzip stream directly
|
278
|
+
|
279
|
+
## 0.4.0 (April 29th, 2009)
|
280
|
+
* NOTE: Breaking API change:
|
281
|
+
* refactored Stream parsing methods out of Yajl::Native into Yajl::Stream
|
282
|
+
* removed Yajl::Native namespace/module
|
283
|
+
* Addition of Yajl::HttpStream module
|
284
|
+
* This module is for streaming JSON HTTP responses directly into Yajl (as they're being received) for increased awesomeness
|
285
|
+
* it currently supports basic get requests with Yajl::HttpStream.get(uri)
|
286
|
+
* it also supports (and prefers) output compressed (gzip) responses
|
287
|
+
* Addition Yajl::Chunked module
|
288
|
+
* This module is for feeding Yajl JSON pieces at a time, instead of an entire IO object
|
289
|
+
* This works very well in environments like an EventMachine app where data is received in chunks by design
|
290
|
+
* decreased read buffer for Yajl::Stream from 8kb to 4kb
|
291
|
+
|
292
|
+
## 0.3.4 (April 24th, 2009)
|
293
|
+
* turned Unicode checks back on in the Yajl parser now that it's fixed (thanks Lloyd!)
|
294
|
+
* this also bumps the yajl version dependency requirement to 1.0.4
|
295
|
+
* better guessing of Integer/Float from number found instead of just trying to create a BigNum no matter what
|
296
|
+
* changed extconf.rb to fail Makefile creation if yajl isn't found
|
297
|
+
* added a test to check for parsing Infinity due to a Float overflow
|
298
|
+
|
299
|
+
## 0.3.3 (April 24th, 2009)
|
300
|
+
* 1.9 compatibility
|
301
|
+
|
302
|
+
## 0.3.2 (April 24th, 2009)
|
303
|
+
* version bump: forgot to include yajl.c in the gem
|
304
|
+
|
305
|
+
## 0.3.1 (April 23rd, 2009)
|
306
|
+
* fixed borked gemspec
|
307
|
+
|
308
|
+
## 0.3.0 (April 23rd, 2009)
|
309
|
+
* slight refactor of ActiveSupport tests to better reflect how they actually exist in ActiveSupport
|
310
|
+
* typo correction in the changelog which had the years in 2008
|
311
|
+
* added some initial spec tests
|
312
|
+
* ported some from ActiveSupport to ensure proper compatibility
|
313
|
+
* included 57 JSON fixtures to test against, all of which pass
|
314
|
+
* changed parser config to not check for invalid unicode characters as Ruby is going to do this anyway (?). This resolves the remaining test failures around unicode.
|
315
|
+
* changed how the parser was dealing with numbers to prevent overflows
|
316
|
+
* added an exception class Yajl::ParseError which is now used in place of simply printing to STDERR upon a parsing error
|
317
|
+
* renamed a couple of JSON test files in the benchmark folder to better represent their contents
|
318
|
+
* misc README updates
|
319
|
+
|
320
|
+
## 0.2.1 (April 23rd, 2009)
|
321
|
+
* fixed parsing bug - also fixed failing ActiveSupport test failures (except for the unicode one, which is an issue in Yajl itself)
|
322
|
+
|
323
|
+
## 0.2.0 (April 22nd, 2009)
|
324
|
+
* updated gemspec and README
|
325
|
+
|
326
|
+
## 0.1.0 (April 21st, 2009)
|
327
|
+
* initial release - gemified
|
data/Gemfile
ADDED
data/MIT-LICENSE
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright (c) 2008-2011 Brian Lopez - http://github.com/brianmario
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
"Software"), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
17
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
18
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
19
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
20
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,362 @@
|
|
1
|
+
# YAJL C Bindings for Ruby
|
2
|
+
|
3
|
+
This gem is a C binding to the excellent YAJL JSON parsing and generation library.
|
4
|
+
|
5
|
+
You can read more info at the project's website http://lloyd.github.com/yajl or check out its code at http://github.com/lloyd/yajl.
|
6
|
+
|
7
|
+
## Features
|
8
|
+
|
9
|
+
* JSON parsing and encoding directly to and from an IO stream (file, socket, etc) or String. Compressed stream parsing and encoding supported for Bzip2, Gzip and Deflate.
|
10
|
+
* Parse and encode *multiple* JSON objects to and from streams or strings continuously.
|
11
|
+
* JSON gem compatibility API - allows yajl-ruby to be used as a drop-in replacement for the JSON gem
|
12
|
+
* Basic HTTP client (only GET requests supported for now) which parses JSON directly off the response body *as it's being received*
|
13
|
+
* ~3.5x faster than JSON.generate
|
14
|
+
* ~1.9x faster than JSON.parse
|
15
|
+
* ~4.5x faster than YAML.load
|
16
|
+
* ~377.5x faster than YAML.dump
|
17
|
+
* ~1.5x faster than Marshal.load
|
18
|
+
* ~2x faster than Marshal.dump
|
19
|
+
|
20
|
+
## How to install
|
21
|
+
|
22
|
+
Go ahead and install it as usual:
|
23
|
+
|
24
|
+
```
|
25
|
+
gem install yajl-ruby
|
26
|
+
```
|
27
|
+
|
28
|
+
## Example of use
|
29
|
+
|
30
|
+
NOTE: I'm building up a collection of small examples in the examples (http://github.com/brianmario/yajl-ruby/tree/master/examples) folder.
|
31
|
+
|
32
|
+
First, you're probably gonna want to require it:
|
33
|
+
|
34
|
+
``` ruby
|
35
|
+
require 'yajl'
|
36
|
+
```
|
37
|
+
|
38
|
+
### Parsing
|
39
|
+
|
40
|
+
Then maybe parse some JSON from:
|
41
|
+
|
42
|
+
a File IO
|
43
|
+
|
44
|
+
``` ruby
|
45
|
+
json = File.new('test.json', 'r')
|
46
|
+
parser = Yajl::Parser.new
|
47
|
+
hash = parser.parse(json)
|
48
|
+
```
|
49
|
+
|
50
|
+
or maybe a StringIO
|
51
|
+
|
52
|
+
``` ruby
|
53
|
+
json = StringIO.new("...some JSON...")
|
54
|
+
parser = Yajl::Parser.new
|
55
|
+
hash = parser.parse(json)
|
56
|
+
```
|
57
|
+
|
58
|
+
or maybe STDIN
|
59
|
+
|
60
|
+
```
|
61
|
+
cat someJsonFile.json | ruby -ryajl -e "puts Yajl::Parser.parse(STDIN).inspect"
|
62
|
+
```
|
63
|
+
|
64
|
+
Or lets say you didn't have access to the IO object that contained JSON data, but instead
|
65
|
+
only had access to chunks of it at a time. No problem!
|
66
|
+
|
67
|
+
(Assume we're in an EventMachine::Connection instance)
|
68
|
+
|
69
|
+
``` ruby
|
70
|
+
def post_init
|
71
|
+
@parser = Yajl::Parser.new(:symbolize_keys => true)
|
72
|
+
end
|
73
|
+
|
74
|
+
def object_parsed(obj)
|
75
|
+
puts "Sometimes one pays most for the things one gets for nothing. - Albert Einstein"
|
76
|
+
puts obj.inspect
|
77
|
+
end
|
78
|
+
|
79
|
+
def connection_completed
|
80
|
+
# once a full JSON object has been parsed from the stream
|
81
|
+
# object_parsed will be called, and passed the constructed object
|
82
|
+
@parser.on_parse_complete = method(:object_parsed)
|
83
|
+
end
|
84
|
+
|
85
|
+
def receive_data(data)
|
86
|
+
# continue passing chunks
|
87
|
+
@parser << data
|
88
|
+
end
|
89
|
+
```
|
90
|
+
|
91
|
+
Or if you don't need to stream it, it'll just return the built object from the parse when it's done.
|
92
|
+
NOTE: if there are going to be multiple JSON strings in the input, you *must* specify a block or callback as this
|
93
|
+
is how yajl-ruby will hand you (the caller) each object as it's parsed off the input.
|
94
|
+
|
95
|
+
``` ruby
|
96
|
+
obj = Yajl::Parser.parse(str_or_io)
|
97
|
+
```
|
98
|
+
|
99
|
+
Or how about a JSON API HTTP request?
|
100
|
+
This actually makes a request using a raw TCPSocket, then parses the JSON body right off the socket. While it's being received over the wire!
|
101
|
+
|
102
|
+
``` ruby
|
103
|
+
require 'uri'
|
104
|
+
require 'yajl/http_stream'
|
105
|
+
|
106
|
+
url = URI.parse("http://search.twitter.com/search.json?q=engineyard")
|
107
|
+
results = Yajl::HttpStream.get(url)
|
108
|
+
```
|
109
|
+
|
110
|
+
Or do the same request, with Gzip and Deflate output compression support (also supports Bzip2, if loaded):
|
111
|
+
(this does the same raw socket Request, but transparently parses the compressed response body)
|
112
|
+
|
113
|
+
``` ruby
|
114
|
+
require 'uri'
|
115
|
+
require 'yajl/gzip'
|
116
|
+
require 'yajl/deflate'
|
117
|
+
require 'yajl/http_stream'
|
118
|
+
|
119
|
+
url = URI.parse("http://search.twitter.com/search.json?q=engineyard")
|
120
|
+
results = Yajl::HttpStream.get(url)
|
121
|
+
```
|
122
|
+
|
123
|
+
Since yajl-ruby parses JSON as a stream, supporting API's like Twitter's Streaming API are a piece-of-cake.
|
124
|
+
You can simply supply a block to `Yajl::HttpStream.get`, which is used as the callback for when a JSON object has been
|
125
|
+
unserialized off the stream. For the case of this Twitter Streaming API call, the callback gets fired a few times a second (depending on your connection speed).
|
126
|
+
The code below is all that's needed to make the request and stream unserialized Ruby hashes off the response, continuously.
|
127
|
+
You'll note that I've enabled the :symbolize_keys parser option as well. Doing so is much more efficient for parsing JSON streams with
|
128
|
+
lots of repetitive keys - for things like result sets or multiple API requests - than the same parse with string keys.
|
129
|
+
This is because Ruby will reuse (and never GC) its symbol table. Be that as it may, if you want to parse JSON strings with random key names
|
130
|
+
it's much better to leave string keys enabled (the default), so they can get GC'd later.
|
131
|
+
|
132
|
+
``` ruby
|
133
|
+
require 'uri'
|
134
|
+
require 'yajl/http_stream'
|
135
|
+
|
136
|
+
uri = URI.parse("http://#{username}:#{password}@stream.twitter.com/spritzer.json")
|
137
|
+
Yajl::HttpStream.get(uri, :symbolize_keys => true) do |hash|
|
138
|
+
puts hash.inspect
|
139
|
+
end
|
140
|
+
```
|
141
|
+
|
142
|
+
Or how about parsing directly from a compressed file?
|
143
|
+
|
144
|
+
``` ruby
|
145
|
+
require 'yajl/bzip2'
|
146
|
+
|
147
|
+
file = File.new('some.json.bz2', 'r')
|
148
|
+
result = Yajl::Bzip2::StreamReader.parse(file)
|
149
|
+
```
|
150
|
+
|
151
|
+
### Encoding
|
152
|
+
|
153
|
+
Since yajl-ruby does everything using streams, you simply need to pass the object to encode, and the IO to write the stream to (this happens in chunks).
|
154
|
+
|
155
|
+
This allows you to encode JSON as a stream, writing directly to a socket
|
156
|
+
|
157
|
+
``` ruby
|
158
|
+
socket = TCPSocket.new('192.168.1.101', 9000)
|
159
|
+
hash = {:foo => 12425125, :bar => "some string", ... }
|
160
|
+
encoder = Yajl::Encoder.new
|
161
|
+
Yajl::Encoder.encode(hash, socket)
|
162
|
+
```
|
163
|
+
|
164
|
+
Or what if you wanted to compress the stream over the wire?
|
165
|
+
|
166
|
+
``` ruby
|
167
|
+
require 'yajl/gzip'
|
168
|
+
socket = TCPSocket.new('192.168.1.101', 9000)
|
169
|
+
hash = {:foo => 12425125, :bar => "some string", ... }
|
170
|
+
Yajl::Gzip::StreamWriter.encode(hash, socket)
|
171
|
+
```
|
172
|
+
|
173
|
+
Or what about encoding multiple objects to JSON over the same stream?
|
174
|
+
This example will encode and send 50 JSON objects over the same stream, continuously.
|
175
|
+
|
176
|
+
``` ruby
|
177
|
+
socket = TCPSocket.new('192.168.1.101', 9000)
|
178
|
+
encoder = Yajl::Encoder.new
|
179
|
+
50.times do
|
180
|
+
hash = {:current_time => Time.now.to_f, :foo => 12425125}
|
181
|
+
encoder.encode(hash, socket)
|
182
|
+
end
|
183
|
+
```
|
184
|
+
|
185
|
+
Using `EventMachine` and you want to encode and send in chunks?
|
186
|
+
(Assume we're in an `EventMachine::Connection` instance)
|
187
|
+
|
188
|
+
``` ruby
|
189
|
+
def post_init
|
190
|
+
# Passing a :terminator character will let us determine when the encoder
|
191
|
+
# is done encoding the current object
|
192
|
+
@encoder = Yajl::Encoder.new
|
193
|
+
motd_contents = File.read("/path/to/motd.txt")
|
194
|
+
status = File.read("/path/to/huge/status_file.txt")
|
195
|
+
@motd = {:motd => motd_contents, :system_status => status}
|
196
|
+
end
|
197
|
+
|
198
|
+
def connection_completed
|
199
|
+
# The encoder will do its best to hand you data in chunks that
|
200
|
+
# are around 8kb (but you may see some that are larger)
|
201
|
+
#
|
202
|
+
# It should be noted that you could have also assigned the _on_progress_ callback
|
203
|
+
# much like you can assign the _on_parse_complete_ callback with the parser class.
|
204
|
+
# Passing a block (like below) essentially tells the encoder to use that block
|
205
|
+
# as the callback normally assigned to _on_progress_.
|
206
|
+
#
|
207
|
+
# Send our MOTD and status
|
208
|
+
@encoder.encode(@motd) do |chunk|
|
209
|
+
if chunk.nil? # got our terminator, encoding is done
|
210
|
+
close_connection_after_writing
|
211
|
+
else
|
212
|
+
send_data(chunk)
|
213
|
+
end
|
214
|
+
end
|
215
|
+
end
|
216
|
+
```
|
217
|
+
|
218
|
+
But to make things simple, you might just want to let yajl-ruby do all the hard work for you and just hand back
|
219
|
+
a string when it's finished. In that case, just don't provide and IO or block (or assign the on_progress callback).
|
220
|
+
|
221
|
+
``` ruby
|
222
|
+
str = Yajl::Encoder.encode(obj)
|
223
|
+
```
|
224
|
+
|
225
|
+
You can also use `Yajl::Bzip2::StreamWriter` and `Yajl::Deflate::StreamWriter`. So you can pick whichever fits your CPU/bandwidth sweet-spot.
|
226
|
+
|
227
|
+
=== HTML Safety
|
228
|
+
|
229
|
+
If you plan on embedding the output from the encoder in the DOM, you'll want to make sure you use the html_safe option on the encoder. This will escape all '/' characters to ensure no closing tags can be injected, preventing XSS.
|
230
|
+
|
231
|
+
Meaning the following should be perfectly safe:
|
232
|
+
|
233
|
+
``` html
|
234
|
+
<script type="text/javascript">
|
235
|
+
var escaped_str = <%= Yajl::Encoder.encode("</script><script>alert('hi!');</script>", :html_safe => true) %>;
|
236
|
+
</script>
|
237
|
+
```
|
238
|
+
|
239
|
+
== JSON gem Compatibility API
|
240
|
+
|
241
|
+
The JSON gem compatibility API isn't enabled by default. You have to explicitly require it like so:
|
242
|
+
|
243
|
+
``` ruby
|
244
|
+
require 'yajl/json_gem'
|
245
|
+
```
|
246
|
+
|
247
|
+
That's right, you can just replace `"require 'json'"` with the line above and you're done!
|
248
|
+
|
249
|
+
This will require yajl-ruby itself, as well as enable its JSON gem compatibility API.
|
250
|
+
|
251
|
+
This includes the following API:
|
252
|
+
|
253
|
+
JSON.parse, JSON.generate, JSON.pretty_generate, JSON.load, JSON.dump
|
254
|
+
and all of the #to_json instance method overrides for Ruby's primitive objects
|
255
|
+
|
256
|
+
|
257
|
+
Once the compatibility API is enabled, your existing or new project should work as if the JSON gem itself were being used. Only you'll be using Yajl ;)
|
258
|
+
|
259
|
+
There are a lot more possibilities that I'd love to see other gems/plugins for someday.
|
260
|
+
|
261
|
+
Some ideas:
|
262
|
+
|
263
|
+
* parsing logs in JSON format
|
264
|
+
* a Rails plugin - DONE! (http://github.com/technoweenie/yajl-rails)
|
265
|
+
* official support in Rails 3 - DONE (http://github.com/rails/rails/commit/a96bf4ab5e73fccdafb78b99e8a122cc2172b505)
|
266
|
+
* and is the default (if installed) - http://github.com/rails/rails/commit/63bb955a99eb46e257655c93dd64e86ebbf05651
|
267
|
+
* Rack middleware (ideally the JSON body could be handed to the parser while it's still being received, this is apparently possible with Unicorn)
|
268
|
+
* JSON API clients (http://github.com/brianmario/freckle-api)
|
269
|
+
|
270
|
+
## Benchmarks
|
271
|
+
|
272
|
+
After I finished implementation - this library performs close to the same as the current JSON.parse (C gem) does on small/medium files.
|
273
|
+
|
274
|
+
But on larger files, and higher amounts of iteration, this library was around 2x faster than JSON.parse.
|
275
|
+
|
276
|
+
The main benefit of this library is in its memory usage.
|
277
|
+
Since it's able to parse the stream in chunks, its memory requirements are very, very low.
|
278
|
+
|
279
|
+
Here's what parsing a 2.43MB JSON file off the filesystem 20 times looks like:
|
280
|
+
|
281
|
+
### Memory Usage
|
282
|
+
|
283
|
+
#### Average
|
284
|
+
|
285
|
+
* Yajl::Parser#parse: 32MB
|
286
|
+
* JSON.parse: 54MB
|
287
|
+
* ActiveSupport::JSON.decode: 63MB
|
288
|
+
|
289
|
+
#### Peak
|
290
|
+
|
291
|
+
* Yajl::Parser#parse: 32MB
|
292
|
+
* JSON.parse: 57MB
|
293
|
+
* ActiveSupport::JSON.decode: 67MB
|
294
|
+
|
295
|
+
### Parse Time
|
296
|
+
|
297
|
+
* Yajl::Parser#parse: 4.54s
|
298
|
+
* JSON.parse: 5.47s
|
299
|
+
* ActiveSupport::JSON.decode: 64.42s
|
300
|
+
|
301
|
+
### Encode Time
|
302
|
+
|
303
|
+
* Yajl::Encoder#encode: 3.59s
|
304
|
+
* JSON#to_json: 6.2s
|
305
|
+
* ActiveSupport::JSON.encode: 45.58s
|
306
|
+
|
307
|
+
### Compared to YAML
|
308
|
+
|
309
|
+
NOTE: I converted the 2.4MB JSON file to YAML for this test.
|
310
|
+
|
311
|
+
#### Parse Time (from their respective formats)
|
312
|
+
|
313
|
+
* Yajl::Parser#parse: 4.33s
|
314
|
+
* JSON.parse: 5.37s
|
315
|
+
* YAML.load: 19.47s
|
316
|
+
|
317
|
+
#### Encode Time (to their respective formats)
|
318
|
+
|
319
|
+
* Yajl::Encoder#encode: 3.47s
|
320
|
+
* JSON#to_json: 6.6s
|
321
|
+
* YAML.dump(obj, io): 1309.93s
|
322
|
+
|
323
|
+
### Compared to Marshal.load/Marshal.dump
|
324
|
+
|
325
|
+
NOTE: I converted the 2.4MB JSON file to a Hash and a dump file from Marshal.dump for this test.
|
326
|
+
|
327
|
+
#### Parse Time (from their respective formats)
|
328
|
+
|
329
|
+
* Yajl::Parser#parse: 4.54s
|
330
|
+
* JSON.parse: 7.40s
|
331
|
+
* Marshal.load: 7s
|
332
|
+
|
333
|
+
#### Encode Time (to their respective formats)
|
334
|
+
|
335
|
+
* Yajl::Encoder#encode: 2.39s
|
336
|
+
* JSON#to_json: 8.37s
|
337
|
+
* Marshal.dump: 4.66s
|
338
|
+
|
339
|
+
## Third Party Sources Bundled
|
340
|
+
|
341
|
+
This project includes code from the BSD licensed yajl project, copyright 2007-2009 Lloyd Hilaiel
|
342
|
+
|
343
|
+
## Special Thanks & Contributors
|
344
|
+
|
345
|
+
For those of you using yajl-ruby out in the wild, please hit me up on Twitter (brianmario) or send me a message here on the Githubs describing the site and how you're using it. I'd love to get a list going!
|
346
|
+
|
347
|
+
I've had a lot of inspiration, and a lot of help. Thanks to everyone who's been a part of this and those to come!
|
348
|
+
|
349
|
+
* Lloyd Hilaiel - http://github.com/lloyd - for writing Yajl!!
|
350
|
+
* Josh Ferguson - http://github.com/besquared - for peer-pressuring me into getting back into C; it worked ;) Also tons of support over IM
|
351
|
+
* Jonathan Novak - http://github.com/cypriss - pointer-hacking help
|
352
|
+
* Tom Smith - http://github.com/rtomsmith - pointer-hacking help
|
353
|
+
* Rick Olson - http://github.com/technoweenie - for making an ActiveSupport patch with support for this library and teasing me that it might go into Rails 3. You sure lit a fire under my ass and I got a ton of work done because of it! :)
|
354
|
+
* The entire Github Crew - http://github.com/ - my inspiration, time spent writing this, finding Yajl, So many-MANY other things wouldn't have been possible without this awesome service. I owe you guys some whiskey at Kilowatt.
|
355
|
+
* Ben Burkert - http://github.com/benburkert
|
356
|
+
* Aman Gupta - http://github.com/tmm1 - tons of suggestions and inspiration for the most recent features, and hopefully more to come ;)
|
357
|
+
* Filipe Giusti
|
358
|
+
* Jonathan George
|
359
|
+
* Luke Redpath
|
360
|
+
* Neil Berkman
|
361
|
+
* Pavel Valodzka
|
362
|
+
* Rob Sharp
|