ffi-yajl 2.1.0-universal-java → 2.2.0-universal-java

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 405d187ca7b5ec111ac8efbe005109f8bba785a8
4
- data.tar.gz: f749ab5a3e388d3c069825b00cd37c30796db6d9
3
+ metadata.gz: 2b4326cfe36fdc90c1f60e9ed14e601de6898dc5
4
+ data.tar.gz: cfc7023e68822af31e4e634024fa9477064516f6
5
5
  SHA512:
6
- metadata.gz: 4f67c6da0c07d49a2fdb20a2bdbedd9e93eb161a81ebad06601fb9b4200ee1413a24418487fad7d8bb3b0845b72adf83f398a200bdb4d5965322a49205ed98a7
7
- data.tar.gz: 9c47d061b3055164397a98666cf34786ff0341c0ccccb9fdb95655927150af3adbedc9d6ffcace61883cfb3f39e45102ee3fc990891cc5b4769de86f044a7f93
6
+ metadata.gz: e8bb555094aad2ea90b2501483ee411dee20cda9ee3ae6968660db047f6af4d4a93675860e27ca4d9dcc290100b8cc2b5291f78a9a17dd0f0d3cb6cbe5d1d52e
7
+ data.tar.gz: 73f5223e85d8ac1d15951eac652f02de799034b5698e0f4a2fe04e2e0e62401175a4deff8a687f85e7ca4e303ca041bdf19c9366be212ee0b0a12119424cff78
data/README.md CHANGED
@@ -9,16 +9,109 @@ extension mechanisms, including both MRI native extensions and FFI in
9
9
  order to be compatible with as many Ruby implementations as possible
10
10
  while providing good performance where possible.
11
11
 
12
+ ## How to Install
13
+
14
+ Install from the command-line:
15
+
16
+ ```
17
+ gem install ffi-yajl
18
+ ```
19
+
20
+ Or use a Gemfile:
21
+
22
+ ```
23
+ gem 'ffi-yajl'
24
+ ```
25
+
26
+ ## Supported Ruby VMs:
27
+
28
+ * Ruby MRI 1.9.3/2.0.0/2.1.x/2.2.x
29
+ * rbx 2.2.x (possibly earlier)
30
+ * Jruby 1.7.x (possibly earlier)
31
+
32
+ Ruby 1.8.7 support was dropped in 2.2.0
33
+
34
+ ## Supported Distros:
35
+
36
+ * Ubuntu 10.04 through 14.10
37
+ * Debian 7.x
38
+ * RHEL/CentOS/Oracle 5.x/6.x/7.x
39
+ * Solaris 9/10/11 (gcc, sun compiler untested)
40
+ * AIX 6.x/7.x (gcc or xlc)
41
+ * Windows 2008r2/2012 (and Win2k/2k3 and consumer versions should work)
42
+
12
43
  ## Basic Usage
13
44
 
45
+ Start by requiring it:
46
+
14
47
  ```ruby
15
48
  require 'ffi-yajl'
16
- json_out = FFI_Yajl::Encoder.encode( { "foo" => [ "bar", "baz" ] } )
17
- # => "{\"foo\":[\"bar\",\"baz\"]}"
18
- data_in = FFI_Yajl::Parser.parse( json_out )
19
- # => {"foo"=>["bar", "baz"]}
20
49
  ```
21
50
 
51
+ You can encode and parse with class objects:
52
+
53
+ ```ruby
54
+ options_hash = {}
55
+ json = FFI_Yajl::Encoder.encode( {"foo"=>["bar","baz"]}, options_hash )
56
+ hash = FFI_Yajl::Parser.parse( json, options_hash )
57
+ ```
58
+
59
+ Or you can be more object oriented:
60
+
61
+ ```ruby
62
+ options_hash = {}
63
+ encoder = FFI_Yajl::Encoder.new( options_hash )
64
+ json = encoder.encode( {"foo"=>["bar","baz"]} )
65
+ parser = FFI_Yajl::Parser.new( options_hash )
66
+ hash = parser.parse( json )
67
+ ```
68
+
69
+ ## Parser Options
70
+
71
+ * `:check_utf8`
72
+ * `:dont_validate_strings`
73
+ * `:symbolize_keys` (default = false): JSON keys are parsed into symbols instead of strings.
74
+ * `:symbolize_names` (default = false): Alias for `:symbolize_keys`.
75
+ * `:allow_trailing_garbage`
76
+ * `:allow_multiple_values`
77
+ * `:allow_partial_values`
78
+ * `:unique_key_checking` (default = false): Will raise an exception if keys
79
+ are repeated in hashes in the input JSON. Without this, repeated keys will
80
+ silently replace the previous key.
81
+
82
+ ## Encoder Options
83
+
84
+ * `:pretty` (default = false): Produces more human readable 'pretty' output.
85
+ * `:validate_utf8` (default = true): Will raise an exception when trying to
86
+ encode strings that are invalid UTF-8. When set to false this still will
87
+ produce valid UTF-8 JSON but will replace invalid characters.
88
+
89
+ ## Forcing FFI or C Extension
90
+
91
+ You can set the environment variable `FORCE_FFI_YAJL` to `ext` or `ffi` to
92
+ control if the C extension or FFI bindings are used.
93
+
94
+ ## Yajl Library Packaging
95
+
96
+ This library prefers to use the embedded yajl 2.x C library packaged in the
97
+ libyajl2 gem. In order to use the operating system yajl library (which must be
98
+ yajl 2.x) the environment variable `USE_SYSTEM_LIBYAJL2` can be set before
99
+ installing or bundling libyajl2. This will force the libyajl2 gem to skip
100
+ compiling its embedded library and the ffi-yajl gem will fallback to using the
101
+ system yajl library.
102
+
103
+ ## No JSON Gem Compatiblity layer
104
+
105
+ This library does not offer a feature to patch `#to_json` methods into all
106
+ the ruby classes similarly to the JSON gem or yajl-ruby's `yajl/json_gem`
107
+ compatibility API. The differences in encoding between the JSON gem and the
108
+ Yajl C library can produce different output on different systems and makes
109
+ testing brittle. Such a feature will not be included. It was removed in
110
+ this pull request and could be easily extracted to its own gem (we have
111
+ no interest in maintaining that gem):
112
+
113
+ https://github.com/chef/ffi-yajl/pull/47/files
114
+
22
115
  ## Why This Instead of X?
23
116
 
24
117
  yajl is the only JSON library we've found that has error messages that
@@ -36,15 +129,6 @@ could support non-MRI rubies) and we also needed some bug fixes in
36
129
  yajl2, but the maintainer wasn't able to devote enough time to the
37
130
  project to make these updates in a timeframe that worked for us.
38
131
 
39
- ## Yajl Library Packaging
40
-
41
- This library prefers to use the embedded yajl 2.x C library packaged in the
42
- libyajl2 gem. In order to use the operating system yajl library (which must be
43
- yajl 2.x) the environment variable `USE_SYSTEM_LIBYAJL2` can be set before
44
- installing or bundling libyajl2. This will force the libyajl2 gem to skip
45
- compiling its embedded library and the ffi-yajl gem will fallback to using the
46
- system yajl library.
47
-
48
132
  ## Thanks
49
133
 
50
134
  This was initially going to be a clean rewrite of an ffi ruby wrapper around
@@ -7,9 +7,6 @@ static VALUE cYajl_Gen;
7
7
 
8
8
  /* FIXME: the json gem does a whole bunch of indirection around monkeypatching... not sure if we need to as well... */
9
9
 
10
- #define CHECK_STATUS(call) \
11
- if ((status = (call)) != yajl_gen_status_ok) { rb_funcall(mEncoder2, rb_intern("raise_error_for_status"), 1, INT2FIX(status)); }
12
-
13
10
  static VALUE mEncoder_do_yajl_encode(VALUE self, VALUE obj, VALUE yajl_gen_opts, VALUE json_opts) {
14
11
  ID sym_ffi_yajl = rb_intern("ffi_yajl");
15
12
  VALUE sym_yajl_gen_beautify = ID2SYM(rb_intern("yajl_gen_beautify"));
@@ -72,290 +69,285 @@ int rb_cHash_ffi_yajl_callback(VALUE key, VALUE val, VALUE extra) {
72
69
  return 0;
73
70
  }
74
71
 
75
- static VALUE rb_cHash_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
72
+ #define RB_FUNC0(call) rb_funcall(self, rb_intern(call), 0)
73
+
74
+ /*
75
+ * wrappers around yajl_gen_* functions
76
+ */
77
+
78
+ /* encode a c-string as a yajl string */
79
+ VALUE gen_cstring(VALUE rb_yajl_gen, char *cptr, int len) {
76
80
  yajl_gen_status status;
77
- VALUE extra;
78
81
  struct yajl_gen_t *yajl_gen;
79
82
  Data_Get_Struct(rb_yajl_gen, struct yajl_gen_t, yajl_gen);
80
83
 
81
- if ( rb_hash_aref(state, rb_str_new2("processing_key")) == Qtrue ) {
82
- ID sym_to_s = rb_intern("to_s");
83
- VALUE str = rb_funcall(self, sym_to_s, 0);
84
- char *cptr = RSTRING_PTR(str);
85
- int len = RSTRING_LEN(str);
86
-
87
- CHECK_STATUS(
88
- yajl_gen_string(yajl_gen, (unsigned char *)cptr, len)
89
- );
90
- } else {
91
- extra = rb_hash_new(); /* FIXME: reduce garbage */
84
+ if ((status = yajl_gen_string(yajl_gen, (unsigned char *)cptr, len)) != yajl_gen_status_ok) {
85
+ rb_funcall(mEncoder2, rb_intern("raise_error_for_status"), 2, INT2FIX(status), rb_str_new(cptr, len));
86
+ }
92
87
 
93
- rb_hash_aset(extra, rb_str_new2("yajl_gen"), rb_yajl_gen);
88
+ return Qnil;
89
+ }
94
90
 
95
- rb_hash_aset(extra, rb_str_new2("state"), state);
91
+ /* encode a ruby-sring as a yajl string */
92
+ VALUE gen_string(VALUE rb_yajl_gen, VALUE str) {
93
+ char *cptr = RSTRING_PTR(str);
94
+ int len = RSTRING_LEN(str);
96
95
 
97
- CHECK_STATUS(
98
- yajl_gen_map_open(yajl_gen)
99
- );
100
- rb_hash_foreach(self, rb_cHash_ffi_yajl_callback, extra);
101
- CHECK_STATUS(
102
- yajl_gen_map_close(yajl_gen)
103
- );
96
+ return gen_cstring(rb_yajl_gen, cptr, len);
97
+ }
98
+
99
+ /* calls #to_s on an object to encode it as a yajl string */
100
+ static VALUE gen_string_to_s(VALUE rb_yajl_gen, VALUE self) {
101
+ return gen_string(rb_yajl_gen, RB_FUNC0("to_s"));
102
+ }
103
+
104
+ /* encode a ruby string as a yajl number (also used to embed already-rendered json from #to_json) */
105
+ VALUE gen_number(VALUE rb_yajl_gen, VALUE str) {
106
+ yajl_gen_status status;
107
+ struct yajl_gen_t *yajl_gen;
108
+ Data_Get_Struct(rb_yajl_gen, struct yajl_gen_t, yajl_gen);
109
+ char *cptr = RSTRING_PTR(str);
110
+ int len = RSTRING_LEN(str);
111
+
112
+ if ((status = yajl_gen_number(yajl_gen, cptr, len)) != yajl_gen_status_ok) {
113
+ rb_funcall(mEncoder2, rb_intern("raise_error_for_status"), 2, INT2FIX(status), str);
104
114
  }
105
115
 
106
116
  return Qnil;
107
117
  }
108
118
 
109
- static VALUE rb_cArray_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
119
+ /* encode hash open */
120
+ VALUE gen_map_open(VALUE rb_yajl_gen) {
110
121
  yajl_gen_status status;
111
- ID sym_ffi_yajl = rb_intern("ffi_yajl");
112
- long i;
113
- VALUE val;
114
122
  struct yajl_gen_t *yajl_gen;
115
123
  Data_Get_Struct(rb_yajl_gen, struct yajl_gen_t, yajl_gen);
116
124
 
117
- if ( rb_hash_aref(state, rb_str_new2("processing_key")) == Qtrue ) {
118
- ID sym_to_s = rb_intern("to_s");
119
- VALUE str = rb_funcall(self, sym_to_s, 0);
120
- char *cptr = RSTRING_PTR(str);
121
- int len = RSTRING_LEN(str);
122
-
123
- CHECK_STATUS(
124
- yajl_gen_string(yajl_gen, (unsigned char *)cptr, len)
125
- );
126
- } else {
127
- CHECK_STATUS(
128
- yajl_gen_array_open(yajl_gen)
129
- );
130
- for(i=0; i<RARRAY_LEN(self); i++) {
131
- val = rb_ary_entry(self, i);
132
- rb_funcall(val, sym_ffi_yajl, 2, rb_yajl_gen, state);
133
- }
134
- CHECK_STATUS(
135
- yajl_gen_array_close(yajl_gen)
136
- );
125
+ if ((status = yajl_gen_map_open(yajl_gen)) != yajl_gen_status_ok) {
126
+ rb_funcall(mEncoder2, rb_intern("raise_error_for_status"), 2, INT2FIX(status), rb_str_new2("{"));
137
127
  }
138
128
 
139
129
  return Qnil;
140
130
  }
141
131
 
142
- static VALUE rb_cNilClass_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
132
+ /* encode a hash close */
133
+ VALUE gen_map_close(VALUE rb_yajl_gen) {
143
134
  yajl_gen_status status;
144
135
  struct yajl_gen_t *yajl_gen;
145
136
  Data_Get_Struct(rb_yajl_gen, struct yajl_gen_t, yajl_gen);
146
137
 
147
- if ( rb_hash_aref(state, rb_str_new2("processing_key")) == Qtrue ) {
148
- ID sym_to_s = rb_intern("to_s");
149
- VALUE str = rb_funcall(self, sym_to_s, 0);
150
- char *cptr = RSTRING_PTR(str);
151
- int len = RSTRING_LEN(str);
152
-
153
- CHECK_STATUS(
154
- yajl_gen_string(yajl_gen, (unsigned char *)cptr, len)
155
- );
156
- } else {
157
- CHECK_STATUS(
158
- yajl_gen_null(yajl_gen)
159
- );
138
+ if ((status = yajl_gen_map_close(yajl_gen)) != yajl_gen_status_ok) {
139
+ rb_funcall(mEncoder2, rb_intern("raise_error_for_status"), 2, INT2FIX(status), rb_str_new2("}"));
160
140
  }
161
141
 
162
142
  return Qnil;
163
143
  }
164
144
 
165
- static VALUE rb_cTrueClass_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
145
+ /* encode an array open */
146
+ VALUE gen_array_open(VALUE rb_yajl_gen) {
166
147
  yajl_gen_status status;
167
148
  struct yajl_gen_t *yajl_gen;
168
149
  Data_Get_Struct(rb_yajl_gen, struct yajl_gen_t, yajl_gen);
169
150
 
170
- if ( rb_hash_aref(state, rb_str_new2("processing_key")) == Qtrue ) {
171
- ID sym_to_s = rb_intern("to_s");
172
- VALUE str = rb_funcall(self, sym_to_s, 0);
173
- char *cptr = RSTRING_PTR(str);
174
- int len = RSTRING_LEN(str);
175
-
176
- CHECK_STATUS(
177
- yajl_gen_string(yajl_gen, (unsigned char *)cptr, len)
178
- );
179
- } else {
180
- CHECK_STATUS(
181
- yajl_gen_bool(yajl_gen, 1)
182
- );
151
+ if ((status = yajl_gen_array_open(yajl_gen)) != yajl_gen_status_ok) {
152
+ rb_funcall(mEncoder2, rb_intern("raise_error_for_status"), 2, INT2FIX(status), rb_str_new2("["));
183
153
  }
184
154
 
185
155
  return Qnil;
186
156
  }
187
157
 
188
- static VALUE rb_cFalseClass_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
158
+ /* encode an array close */
159
+ VALUE gen_array_close(VALUE rb_yajl_gen) {
189
160
  yajl_gen_status status;
190
161
  struct yajl_gen_t *yajl_gen;
191
162
  Data_Get_Struct(rb_yajl_gen, struct yajl_gen_t, yajl_gen);
192
163
 
193
- if ( rb_hash_aref(state, rb_str_new2("processing_key")) == Qtrue ) {
194
- ID sym_to_s = rb_intern("to_s");
195
- VALUE str = rb_funcall(self, sym_to_s, 0);
196
- char *cptr = RSTRING_PTR(str);
197
- int len = RSTRING_LEN(str);
198
-
199
- CHECK_STATUS(
200
- yajl_gen_string(yajl_gen, (unsigned char *)cptr, len)
201
- );
202
- } else {
203
- CHECK_STATUS(
204
- yajl_gen_bool(yajl_gen, 0)
205
- );
164
+ if ((status = yajl_gen_array_close(yajl_gen)) != yajl_gen_status_ok) {
165
+ rb_funcall(mEncoder2, rb_intern("raise_error_for_status"), 2, INT2FIX(status), rb_str_new2("]"));
206
166
  }
207
167
 
208
168
  return Qnil;
209
169
  }
210
170
 
211
- static VALUE rb_cFixnum_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
171
+ /* encode a json null */
172
+ VALUE gen_null(VALUE rb_yajl_gen) {
212
173
  yajl_gen_status status;
213
- ID sym_to_s = rb_intern("to_s");
214
- VALUE str = rb_funcall(self, sym_to_s, 0);
215
- char *cptr = RSTRING_PTR(str);
216
- int len = RSTRING_LEN(str);
217
174
  struct yajl_gen_t *yajl_gen;
218
175
  Data_Get_Struct(rb_yajl_gen, struct yajl_gen_t, yajl_gen);
219
176
 
220
- if (memcmp(cptr, "NaN", 3) == 0 || memcmp(cptr, "Infinity", 8) == 0 || memcmp(cptr, "-Infinity", 9) == 0) {
221
- rb_raise(cEncodeError, "'%s' is an invalid number", cptr);
177
+ if ((status = yajl_gen_null(yajl_gen)) != yajl_gen_status_ok) {
178
+ rb_funcall(mEncoder2, rb_intern("raise_error_for_status"), 2, INT2FIX(status), rb_str_new2("null"));
222
179
  }
223
- if ( rb_hash_aref(state, rb_str_new2("processing_key")) == Qtrue ) {
224
- CHECK_STATUS(
225
- yajl_gen_string(yajl_gen, (unsigned char *)cptr, len)
226
- );
227
- } else {
228
- CHECK_STATUS(
229
- yajl_gen_number(yajl_gen, cptr, len)
230
- );
180
+
181
+ return Qnil;
182
+ }
183
+
184
+ /* encode a true value */
185
+ VALUE gen_true(VALUE rb_yajl_gen) {
186
+ yajl_gen_status status;
187
+ struct yajl_gen_t *yajl_gen;
188
+ Data_Get_Struct(rb_yajl_gen, struct yajl_gen_t, yajl_gen);
189
+
190
+ if ((status = yajl_gen_bool(yajl_gen, 1)) != yajl_gen_status_ok) {
191
+ rb_funcall(mEncoder2, rb_intern("raise_error_for_status"), 2, INT2FIX(status), rb_str_new2("true"));
231
192
  }
232
193
 
233
194
  return Qnil;
234
195
  }
235
196
 
236
- static VALUE rb_cBignum_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
197
+ /* encode a false value */
198
+ VALUE gen_false(VALUE rb_yajl_gen) {
237
199
  yajl_gen_status status;
238
- ID sym_to_s = rb_intern("to_s");
239
- VALUE str = rb_funcall(self, sym_to_s, 0);
240
- char *cptr = RSTRING_PTR(str);
241
- int len = RSTRING_LEN(str);
242
200
  struct yajl_gen_t *yajl_gen;
243
201
  Data_Get_Struct(rb_yajl_gen, struct yajl_gen_t, yajl_gen);
244
202
 
245
- if (memcmp(cptr, "NaN", 3) == 0 || memcmp(cptr, "Infinity", 8) == 0 || memcmp(cptr, "-Infinity", 9) == 0) {
246
- rb_raise(cEncodeError, "'%s' is an invalid number", cptr);
203
+ if ((status = yajl_gen_bool(yajl_gen, 0)) != yajl_gen_status_ok) {
204
+ rb_funcall(mEncoder2, rb_intern("raise_error_for_status"), 2, INT2FIX(status), rb_str_new2("false"));
247
205
  }
206
+
207
+ return Qnil;
208
+ }
209
+
210
+ /*
211
+ * <Object>#to_ffi_yajl() method calls
212
+ */
213
+
214
+ static VALUE rb_cHash_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
248
215
  if ( rb_hash_aref(state, rb_str_new2("processing_key")) == Qtrue ) {
249
- CHECK_STATUS(
250
- yajl_gen_string(yajl_gen, (unsigned char *)cptr, len)
251
- );
216
+ gen_string(rb_yajl_gen, rb_funcall(self, rb_intern("to_s"), 0));
252
217
  } else {
253
- CHECK_STATUS(
254
- yajl_gen_number(yajl_gen, cptr, len)
255
- );
218
+
219
+ /* FIXME: i think this got refactored from something else and it is now pointless --
220
+ should just pass rb_yajl_gen directly instead of this 'extra' hash -- i don't
221
+ *think* this indirection is doing anything useful to mark memory against the GC */
222
+
223
+ VALUE extra = rb_hash_new();
224
+
225
+ rb_hash_aset(extra, rb_str_new2("yajl_gen"), rb_yajl_gen);
226
+
227
+ rb_hash_aset(extra, rb_str_new2("state"), state);
228
+
229
+ gen_map_open(rb_yajl_gen);
230
+
231
+ rb_hash_foreach(self, rb_cHash_ffi_yajl_callback, extra);
232
+
233
+ gen_map_close(rb_yajl_gen);
256
234
  }
257
235
 
258
236
  return Qnil;
259
237
  }
260
238
 
261
- static VALUE rb_cFloat_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
262
- yajl_gen_status status;
263
- ID sym_to_s = rb_intern("to_s");
264
- VALUE str = rb_funcall(self, sym_to_s, 0);
265
- char *cptr = RSTRING_PTR(str);
266
- int len = RSTRING_LEN(str);
267
- struct yajl_gen_t *yajl_gen;
268
- Data_Get_Struct(rb_yajl_gen, struct yajl_gen_t, yajl_gen);
239
+ static VALUE rb_cArray_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
240
+ if ( rb_hash_aref(state, rb_str_new2("processing_key")) == Qtrue ) {
241
+ gen_string(rb_yajl_gen, rb_funcall(self, rb_intern("to_s"), 0));
242
+ } else {
243
+ VALUE val;
244
+ long i;
245
+ ID sym_ffi_yajl = rb_intern("ffi_yajl");
269
246
 
270
- if (memcmp(cptr, "NaN", 3) == 0 || memcmp(cptr, "Infinity", 8) == 0 || memcmp(cptr, "-Infinity", 9) == 0) {
271
- rb_raise(cEncodeError, "'%s' is an invalid number", cptr);
247
+ gen_array_open(rb_yajl_gen);
248
+
249
+ for(i=0; i<RARRAY_LEN(self); i++) {
250
+ val = rb_ary_entry(self, i);
251
+ rb_funcall(val, sym_ffi_yajl, 2, rb_yajl_gen, state);
252
+ }
253
+
254
+ gen_array_close(rb_yajl_gen);
272
255
  }
256
+
257
+ return Qnil;
258
+ }
259
+
260
+ static VALUE rb_cNilClass_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
273
261
  if ( rb_hash_aref(state, rb_str_new2("processing_key")) == Qtrue ) {
274
- CHECK_STATUS(
275
- yajl_gen_string(yajl_gen, (unsigned char *)cptr, len)
276
- );
262
+ gen_cstring(rb_yajl_gen, "", sizeof("")-1);
277
263
  } else {
278
- CHECK_STATUS(
279
- yajl_gen_number(yajl_gen, cptr, len)
280
- );
264
+ gen_null(rb_yajl_gen);
281
265
  }
282
266
 
283
267
  return Qnil;
284
268
  }
285
269
 
286
- static VALUE rb_cString_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
287
- yajl_gen_status status;
288
- struct yajl_gen_t *yajl_gen;
289
- Data_Get_Struct(rb_yajl_gen, struct yajl_gen_t, yajl_gen);
270
+ static VALUE rb_cTrueClass_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
271
+ if ( rb_hash_aref(state, rb_str_new2("processing_key")) == Qtrue ) {
272
+ gen_cstring(rb_yajl_gen, "true", sizeof("true")-1);
273
+ } else {
274
+ gen_true(rb_yajl_gen);
275
+ }
290
276
 
291
- CHECK_STATUS(
292
- yajl_gen_string(yajl_gen, (unsigned char *)RSTRING_PTR(self), RSTRING_LEN(self))
293
- );
277
+ return Qnil;
278
+ }
279
+
280
+ static VALUE rb_cFalseClass_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
281
+ if ( rb_hash_aref(state, rb_str_new2("processing_key")) == Qtrue ) {
282
+ gen_cstring(rb_yajl_gen, "false", sizeof("false")-1);
283
+ } else {
284
+ gen_false(rb_yajl_gen);
285
+ }
294
286
 
295
287
  return Qnil;
296
288
  }
297
289
 
298
- static VALUE string_ffi_yajl(VALUE str, VALUE rb_yajl_gen, VALUE state) {
299
- yajl_gen_status status;
290
+ static VALUE rb_cFixnum_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
291
+ VALUE str = rb_funcall(self, rb_intern("to_s"), 0);
300
292
  char *cptr = RSTRING_PTR(str);
301
- int len = RSTRING_LEN(str);
302
- struct yajl_gen_t *yajl_gen;
303
- Data_Get_Struct(rb_yajl_gen, struct yajl_gen_t, yajl_gen);
304
293
 
305
- CHECK_STATUS(
306
- yajl_gen_string(yajl_gen, (unsigned char *)cptr, len)
307
- );
294
+ if (memcmp(cptr, "NaN", sizeof("NaN")) == 0 || memcmp(cptr, "Infinity", sizeof("Infinity")) == 0 || memcmp(cptr, "-Infinity", sizeof("-Infinity")) == 0) {
295
+ rb_raise(cEncodeError, "'%s' is an invalid number", cptr);
296
+ }
297
+
298
+ if ( rb_hash_aref(state, rb_str_new2("processing_key")) == Qtrue ) {
299
+ gen_string(rb_yajl_gen, str);
300
+ } else {
301
+ gen_number(rb_yajl_gen, str);
302
+ }
308
303
 
309
304
  return Qnil;
310
305
  }
311
306
 
312
- /* calls #to_s on an object to encode it */
313
- static VALUE object_to_s_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
314
- ID sym_to_s = rb_intern("to_s");
315
- VALUE str = rb_funcall(self, sym_to_s, 0);
316
- return string_ffi_yajl(str, rb_yajl_gen, state);
307
+ static VALUE rb_cBignum_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
308
+ return rb_cFixnum_ffi_yajl(self, rb_yajl_gen, state);
309
+ }
310
+
311
+ static VALUE rb_cFloat_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
312
+ return rb_cFixnum_ffi_yajl(self, rb_yajl_gen, state);
313
+ }
314
+
315
+ static VALUE rb_cString_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
316
+ return gen_string(rb_yajl_gen, self);
317
317
  }
318
318
 
319
319
  static VALUE rb_cSymbol_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
320
- return object_to_s_ffi_yajl(self, rb_yajl_gen, state);
320
+ return gen_string_to_s(rb_yajl_gen, self);
321
321
  }
322
322
 
323
323
  static VALUE rb_cDate_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
324
- return object_to_s_ffi_yajl(self, rb_yajl_gen, state);
324
+ return gen_string_to_s(rb_yajl_gen, self);
325
325
  }
326
326
 
327
327
  static VALUE rb_cTime_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
328
328
  ID sym_strftime = rb_intern("strftime");
329
329
  VALUE str = rb_funcall(self, sym_strftime, 1, rb_str_new2("%Y-%m-%d %H:%M:%S %z"));
330
- return string_ffi_yajl(str, rb_yajl_gen, state);
330
+
331
+ return gen_string(rb_yajl_gen, str);
331
332
  }
332
333
 
333
334
  static VALUE rb_cStringIO_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
334
- ID sym_read = rb_intern("read");
335
- VALUE str = rb_funcall(self, sym_read, 0);
336
- return string_ffi_yajl(str, rb_yajl_gen, state);
335
+ return gen_string(rb_yajl_gen, RB_FUNC0("read"));
337
336
  }
338
337
 
339
338
  static VALUE rb_cDateTime_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
340
- return object_to_s_ffi_yajl(self, rb_yajl_gen, state);
339
+ return gen_string_to_s(rb_yajl_gen, self);
341
340
  }
342
341
 
343
342
  static VALUE rb_cObject_ffi_yajl(VALUE self, VALUE rb_yajl_gen, VALUE state) {
344
- yajl_gen_status status;
345
343
  ID sym_to_json = rb_intern("to_json");
346
- VALUE str;
347
-
348
344
  if ( rb_hash_aref(state, rb_str_new2("processing_key")) != Qtrue && rb_respond_to(self, sym_to_json) ) {
349
- VALUE json_opts = rb_hash_aref(state, rb_str_new2("json_opts"));
350
- struct yajl_gen_t *yajl_gen;
351
- Data_Get_Struct(rb_yajl_gen, struct yajl_gen_t, yajl_gen);
352
-
353
- str = rb_funcall(self, sym_to_json, 1, json_opts);
354
- CHECK_STATUS(
355
- yajl_gen_number(yajl_gen, (char *)RSTRING_PTR(str), RSTRING_LEN(str))
356
- );
345
+ VALUE json_opts = rb_hash_aref(state, rb_str_new2("json_opts"));
346
+ VALUE str = rb_funcall(self, sym_to_json, 1, json_opts);
347
+
348
+ gen_number(rb_yajl_gen, str);
357
349
  } else {
358
- object_to_s_ffi_yajl(self, rb_yajl_gen, state);
350
+ gen_string_to_s(rb_yajl_gen, self);
359
351
  }
360
352
 
361
353
  return Qnil;
@@ -40,7 +40,8 @@ module FFI_Yajl
40
40
 
41
41
  # call either the ext or ffi hook
42
42
  str = do_yajl_encode(obj, yajl_gen_opts, opts)
43
- str.force_encoding('UTF-8') if defined? Encoding
43
+ # we can skip cleaning the whole string for utf-8 issues if we have yajl validate as we go
44
+ str.encode!("utf-8", "binary", :undef => :replace) unless yajl_gen_opts[:yajl_gen_validate_utf8]
44
45
  str
45
46
  end
46
47
 
@@ -53,7 +54,9 @@ module FFI_Yajl
53
54
  @opts ||= {}
54
55
  end
55
56
 
56
- def self.raise_error_for_status(status)
57
+ def self.raise_error_for_status(status, token=nil)
58
+ # scrub token to valid utf-8 since we may be issuing an exception on an invalid utf-8 token
59
+ token = token.to_s.encode("utf-8", "binary", :undef => :replace)
57
60
  case status
58
61
  when 1 # yajl_gen_keys_must_be_strings
59
62
  raise FFI_Yajl::EncodeError, "YAJL internal error: attempted use of non-string object as key"
@@ -68,7 +71,7 @@ module FFI_Yajl
68
71
  when 6 # yajl_gen_no_buf
69
72
  raise FFI_Yajl::EncodeError, "YAJL internal error: yajl_gen_get_buf was called, but a print callback was specified, so no internal buffer is available"
70
73
  when 7 # yajl_gen_invalid_string
71
- raise FFI_Yajl::EncodeError, "Invalid UTF-8 string: cannot encode to UTF-8"
74
+ raise FFI_Yajl::EncodeError, "Invalid UTF-8 string '#{token}': cannot encode to UTF-8"
72
75
  else
73
76
  raise FFI_Yajl::EncodeError, "Unknown YAJL Error (#{status}), please report this as a bug"
74
77
  end
@@ -70,11 +70,11 @@ class Hash
70
70
  if state[:processing_key]
71
71
  str = self.to_s
72
72
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize) ) != 0
73
- FFI_Yajl::Encoder.raise_error_for_status(status)
73
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
74
74
  end
75
75
  else
76
76
  if ( status = FFI_Yajl.yajl_gen_map_open(yajl_gen) ) != 0
77
- FFI_Yajl::Encoder.raise_error_for_status(status)
77
+ FFI_Yajl::Encoder.raise_error_for_status(status, '{')
78
78
  end
79
79
  self.each do |key, value|
80
80
  # Perf Fix: mutate state hash rather than creating new copy
@@ -84,7 +84,7 @@ class Hash
84
84
  value.ffi_yajl(yajl_gen, state)
85
85
  end
86
86
  if ( status = FFI_Yajl.yajl_gen_map_close(yajl_gen) ) != 0
87
- FFI_Yajl::Encoder.raise_error_for_status(status)
87
+ FFI_Yajl::Encoder.raise_error_for_status(status, '}')
88
88
  end
89
89
  end
90
90
  end
@@ -95,17 +95,17 @@ class Array
95
95
  if state[:processing_key]
96
96
  str = self.to_s
97
97
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize) ) != 0
98
- FFI_Yajl::Encoder.raise_error_for_status(status)
98
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
99
99
  end
100
100
  else
101
101
  if ( status = FFI_Yajl.yajl_gen_array_open(yajl_gen) ) != 0
102
- FFI_Yajl::Encoder.raise_error_for_status(status)
102
+ FFI_Yajl::Encoder.raise_error_for_status(status, '[')
103
103
  end
104
104
  self.each do |value|
105
105
  value.ffi_yajl(yajl_gen, state)
106
106
  end
107
107
  if ( status = FFI_Yajl.yajl_gen_array_close(yajl_gen) ) != 0
108
- FFI_Yajl::Encoder.raise_error_for_status(status)
108
+ FFI_Yajl::Encoder.raise_error_for_status(status, ']')
109
109
  end
110
110
  end
111
111
  end
@@ -113,14 +113,14 @@ end
113
113
 
114
114
  class NilClass
115
115
  def ffi_yajl(yajl_gen, state)
116
+ str = self.to_s
116
117
  if state[:processing_key]
117
- str = self.to_s
118
118
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize) ) != 0
119
- FFI_Yajl::Encoder.raise_error_for_status(status)
119
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
120
120
  end
121
121
  else
122
122
  if ( status = FFI_Yajl.yajl_gen_null(yajl_gen) ) != 0
123
- FFI_Yajl::Encoder.raise_error_for_status(status)
123
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
124
124
  end
125
125
  end
126
126
  end
@@ -128,14 +128,14 @@ end
128
128
 
129
129
  class TrueClass
130
130
  def ffi_yajl(yajl_gen, state)
131
+ str = self.to_s
131
132
  if state[:processing_key]
132
- str = self.to_s
133
133
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize) ) != 0
134
- FFI_Yajl::Encoder.raise_error_for_status(status)
134
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
135
135
  end
136
136
  else
137
137
  if ( status = FFI_Yajl.yajl_gen_bool(yajl_gen, 1) ) != 0
138
- FFI_Yajl::Encoder.raise_error_for_status(status)
138
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
139
139
  end
140
140
  end
141
141
  end
@@ -143,14 +143,14 @@ end
143
143
 
144
144
  class FalseClass
145
145
  def ffi_yajl(yajl_gen, state)
146
+ str = self.to_s
146
147
  if state[:processing_key]
147
- str = self.to_s
148
148
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize) ) != 0
149
- FFI_Yajl::Encoder.raise_error_for_status(status)
149
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
150
150
  end
151
151
  else
152
152
  if ( status = FFI_Yajl.yajl_gen_bool(yajl_gen, 0) ) != 0
153
- FFI_Yajl::Encoder.raise_error_for_status(status)
153
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
154
154
  end
155
155
  end
156
156
  end
@@ -164,11 +164,11 @@ class Fixnum
164
164
  end
165
165
  if state[:processing_key]
166
166
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize) ) != 0
167
- FFI_Yajl::Encoder.raise_error_for_status(status)
167
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
168
168
  end
169
169
  else
170
170
  if ( status = FFI_Yajl.yajl_gen_integer(yajl_gen, self) ) != 0
171
- FFI_Yajl::Encoder.raise_error_for_status(status)
171
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
172
172
  end
173
173
  end
174
174
  end
@@ -182,11 +182,11 @@ class Bignum
182
182
  end
183
183
  if state[:processing_key]
184
184
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize) ) != 0
185
- FFI_Yajl::Encoder.raise_error_for_status(status)
185
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
186
186
  end
187
187
  else
188
188
  if ( status = FFI_Yajl.yajl_gen_number(yajl_gen, str, str.bytesize) ) != 0
189
- FFI_Yajl::Encoder.raise_error_for_status(status)
189
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
190
190
  end
191
191
  end
192
192
  end
@@ -200,11 +200,11 @@ class Float
200
200
  end
201
201
  if state[:processing_key]
202
202
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize) ) != 0
203
- FFI_Yajl::Encoder.raise_error_for_status(status)
203
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
204
204
  end
205
205
  else
206
206
  if ( status = FFI_Yajl.yajl_gen_number(yajl_gen, str, str.bytesize) ) != 0
207
- FFI_Yajl::Encoder.raise_error_for_status(status)
207
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
208
208
  end
209
209
  end
210
210
  end
@@ -214,7 +214,7 @@ class Symbol
214
214
  def ffi_yajl(yajl_gen, state)
215
215
  str = self.to_s
216
216
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize) ) != 0
217
- FFI_Yajl::Encoder.raise_error_for_status(status)
217
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
218
218
  end
219
219
  end
220
220
  end
@@ -222,7 +222,7 @@ end
222
222
  class String
223
223
  def ffi_yajl(yajl_gen, state)
224
224
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, self, self.bytesize) ) != 0
225
- FFI_Yajl::Encoder.raise_error_for_status(status)
225
+ FFI_Yajl::Encoder.raise_error_for_status(status, self)
226
226
  end
227
227
  end
228
228
  end
@@ -231,7 +231,7 @@ class StringIO
231
231
  def ffi_yajl(yajl_gen, state)
232
232
  str = self.read
233
233
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize) ) != 0
234
- FFI_Yajl::Encoder.raise_error_for_status(status)
234
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
235
235
  end
236
236
  end
237
237
  end
@@ -240,7 +240,7 @@ class Date
240
240
  def ffi_yajl(yajl_gen, state)
241
241
  str = self.to_s
242
242
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize) ) != 0
243
- FFI_Yajl::Encoder.raise_error_for_status(status)
243
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
244
244
  end
245
245
  end
246
246
  end
@@ -249,16 +249,16 @@ class Time
249
249
  def ffi_yajl(yajl_gen, state)
250
250
  str = self.strftime "%Y-%m-%d %H:%M:%S %z"
251
251
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize) ) != 0
252
- FFI_Yajl::Encoder.raise_error_for_status(status)
252
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
253
253
  end
254
254
  end
255
255
  end
256
256
 
257
- class DateTime
257
+ class DateTime < Date
258
258
  def ffi_yajl(yajl_gen, state)
259
259
  str = self.to_s
260
260
  if ( status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize) ) != 0
261
- FFI_Yajl::Encoder.raise_error_for_status(status)
261
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
262
262
  end
263
263
  end
264
264
  end
@@ -267,15 +267,15 @@ end
267
267
  class Object
268
268
  def ffi_yajl(yajl_gen, state)
269
269
  if !state[:processing_key] && self.respond_to?(:to_json)
270
- json = self.to_json(state[:json_opts])
270
+ str = self.to_json(state[:json_opts])
271
271
  # #yajl_gen_number outputs a string without quotes around it
272
- status = FFI_Yajl.yajl_gen_number(yajl_gen, json, json.bytesize)
272
+ status = FFI_Yajl.yajl_gen_number(yajl_gen, str, str.bytesize)
273
273
  else
274
274
  str = self.to_s
275
275
  status = FFI_Yajl.yajl_gen_string(yajl_gen, str, str.bytesize)
276
276
  end
277
277
  if ( status ) != 0
278
- FFI_Yajl::Encoder.raise_error_for_status(status)
278
+ FFI_Yajl::Encoder.raise_error_for_status(status, str)
279
279
  end
280
280
  end
281
281
  end
@@ -21,5 +21,5 @@
21
21
  # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
22
22
 
23
23
  module FFI_Yajl
24
- VERSION = "2.1.0"
24
+ VERSION = "2.2.0"
25
25
  end
@@ -187,7 +187,7 @@ describe "FFI_Yajl::Encoder" do
187
187
  }
188
188
 
189
189
  it "raises an error on invalid json" do
190
- expect{ encoder.encode(ruby) }.to raise_error(FFI_Yajl::EncodeError, "Invalid UTF-8 string: cannot encode to UTF-8")
190
+ expect{ encoder.encode(ruby) }.to raise_error(FFI_Yajl::EncodeError, /Invalid UTF-8 string 'Elan Ruusam.e': cannot encode to UTF-8/)
191
191
  end
192
192
 
193
193
  context "when validate_utf8 is off" do
@@ -196,6 +196,14 @@ describe "FFI_Yajl::Encoder" do
196
196
  it "does not raise an error" do
197
197
  expect{ encoder.encode(ruby) }.not_to raise_error
198
198
  end
199
+
200
+ it "returns utf8" do
201
+ expect( encoder.encode(ruby).encoding ).to eq(Encoding::UTF_8)
202
+ end
203
+
204
+ it "returns valid utf8" do
205
+ expect( encoder.encode(ruby).valid_encoding? ).to be true
206
+ end
199
207
  end
200
208
  end
201
209
  end
data/spec/spec_helper.rb CHANGED
@@ -35,7 +35,6 @@ require 'ffi_yajl'
35
35
 
36
36
  RSpec.configure do |c|
37
37
  c.filter_run_excluding :unix_only => true unless RUBY_PLATFORM !~ /mswin|mingw|windows/
38
- c.filter_run_excluding :ruby_gte_19 => true unless RUBY_VERSION.to_f >= 1.9
39
38
  c.filter_run_excluding :ruby_gte_193 => true unless RUBY_VERSION.to_f >= 2.0 || RUBY_VERSION =~ /^1\.9\.3/
40
39
 
41
40
  c.order = 'random'
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: ffi-yajl
3
3
  version: !ruby/object:Gem::Version
4
- version: 2.1.0
4
+ version: 2.2.0
5
5
  platform: universal-java
6
6
  authors:
7
7
  - Lamont Granquist
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2015-04-20 00:00:00.000000000 Z
11
+ date: 2015-04-30 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rake
@@ -174,7 +174,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
174
174
  requirements:
175
175
  - - ">="
176
176
  - !ruby/object:Gem::Version
177
- version: '0'
177
+ version: 1.9.2
178
178
  required_rubygems_version: !ruby/object:Gem::Requirement
179
179
  requirements:
180
180
  - - ">="