oj 3.13.4 → 3.13.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +32 -0
- data/README.md +1 -1
- data/RELEASE_NOTES.md +61 -0
- data/ext/oj/custom.c +2 -2
- data/ext/oj/fast.c +6 -2
- data/ext/oj/intern.c +7 -1
- data/ext/oj/mimic_json.c +4 -5
- data/ext/oj/object.c +26 -45
- data/ext/oj/parse.c +9 -1
- data/ext/oj/parser.c +76 -28
- data/ext/oj/strict.c +2 -2
- data/ext/oj/usual.c +19 -17
- data/lib/oj/version.rb +1 -1
- data/test/bar.rb +16 -0
- data/test/baz.rb +16 -0
- data/test/bug.rb +16 -0
- data/test/foo.rb +7 -4
- data/test/prec.rb +23 -0
- data/test/test_fast.rb +18 -7
- data/test/test_parser_usual.rb +9 -5
- data/test/zoo.rb +13 -0
- metadata +19 -11
- data/test/benny.rb +0 -50
- data/test/big.rb +0 -15
- data/test/test_parser_memory.rb +0 -45
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 182e77505f61b9f1327d36552a04ac18a960831d01d675b02ec00ec6e33d3239
|
4
|
+
data.tar.gz: 4b56206268f665fe45a8cbd63b605bab35a0adf834e2ad1d603d4e99a1d8b118
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: cc57186d14210b95b7875948ceb2e1782f7a3223cfabcf7e45a5cad617eba67fe506bfe17820decf1881d341e6233c114f617c3ad8395033d75c7be8b72804b9
|
7
|
+
data.tar.gz: cc069dec094cde64f098fa65131d81f6552f2302af87730e19442ad027ad5722f7d6555de1e4a888196b994fcbb8f809f3892a10b05afd955b805851e7d53928
|
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,37 @@
|
|
1
1
|
# CHANGELOG
|
2
2
|
|
3
|
+
|
4
|
+
## 3.13.8 - 2021-09-27
|
5
|
+
|
6
|
+
- Fix `Oj::Doc` behaviour for inexisting path.
|
7
|
+
```ruby
|
8
|
+
Oj::Doc.open('{"foo":1}') do |doc|
|
9
|
+
doc.fetch('/foo/bar') # used to give `1`, now gives `nil`
|
10
|
+
doc.exists?('/foo/bar') # used to give `true`, now gives `false`
|
11
|
+
end
|
12
|
+
```
|
13
|
+
|
14
|
+
- Fix `Oj::Parser` handling of BigDecimal. `snprint()` does not handle `%Lg` correctly but `sprintf()` does.
|
15
|
+
|
16
|
+
## 3.13.7 - 2021-09-16
|
17
|
+
|
18
|
+
- The JSON gem allows invalid unicode so Oj, when mimicing JSON now
|
19
|
+
allows it as well. Use `:allow_invalid_unicode` to change that.
|
20
|
+
|
21
|
+
## 3.13.6 - 2021-09-11
|
22
|
+
|
23
|
+
- Fixed unicode UTF 8 parsing in string values.
|
24
|
+
|
25
|
+
- Fixed hash key allocation issue.
|
26
|
+
|
27
|
+
- The `Oj::Parser.new()` function now allows optional arguments that
|
28
|
+
set the allowed options for the mode. As an example
|
29
|
+
`Oj::Parser.new(:usual, cache_keys: true)`.
|
30
|
+
|
31
|
+
## 3.13.5 - 2021-09-08
|
32
|
+
|
33
|
+
- Assure value strings of zero length are not always cached.
|
34
|
+
|
3
35
|
## 3.13.4 - 2021-09-04
|
4
36
|
|
5
37
|
- Fixed concurrent GC issue in the cache.
|
data/README.md
CHANGED
data/RELEASE_NOTES.md
ADDED
@@ -0,0 +1,61 @@
|
|
1
|
+
# RELEASE NOTES
|
2
|
+
|
3
|
+
The release notes here are organized by release. For a list of changes
|
4
|
+
see the See [{file:CHANGELOG.md}](CHANGELOG.md) file. In this file are
|
5
|
+
the steps to take to aid in keeping things rolling after updating to
|
6
|
+
the latest version.
|
7
|
+
|
8
|
+
## 3.13.7
|
9
|
+
|
10
|
+
The default for JSON when mimicked by Oj is now to set
|
11
|
+
`:allow_invalid_unicode`. To change that behavior JSON.load, set that
|
12
|
+
option to false.
|
13
|
+
|
14
|
+
## 3.13.x
|
15
|
+
|
16
|
+
This release included a new cache that performs better than the
|
17
|
+
earlier cache and a new high performance parser.
|
18
|
+
|
19
|
+
### Cache
|
20
|
+
|
21
|
+
The new cache includes a least recently used expiration to reduce
|
22
|
+
memory use. The cache is also self adjusting and will expand as needed
|
23
|
+
for better performance. It also handles Hash keys and string values
|
24
|
+
with two options, `:cache_keys`, a boolean and `:cache_str` an
|
25
|
+
integer. The `:cache_str` if set to more than zero is the limit for
|
26
|
+
the length of string values to cache. The maximum value is 35 which
|
27
|
+
allows strings up to 34 bytes to be cached.
|
28
|
+
|
29
|
+
One interesting aspect of the cache is not so much the string caching
|
30
|
+
which performs similar to the Ruby intern functions but the caching of
|
31
|
+
symbols and object attribute names. There is a significant gain for
|
32
|
+
symbols and object attributes.
|
33
|
+
|
34
|
+
If the cache is not desired then setting the default options to turn
|
35
|
+
it off can be done with this line:
|
36
|
+
|
37
|
+
``` ruby
|
38
|
+
Oj.default_options = { cache_keys: false, cache_str: 0 }
|
39
|
+
```
|
40
|
+
|
41
|
+
### Oj::Parser
|
42
|
+
|
43
|
+
The new parser uses a different core that follows the approach taken
|
44
|
+
by [OjC](https://github.com/ohler55/ojc) and
|
45
|
+
[OjG](https://github.com/ohler55/ojg). It also takes advantage of the
|
46
|
+
bulk Array and Hash functions. Another issue the new parser addresses
|
47
|
+
is option management. Instead of a single global default_options each
|
48
|
+
parser instance maintains it's own options.
|
49
|
+
|
50
|
+
There is a price to be paid when using the Oj::Parser. The API is not
|
51
|
+
the same the older parser. A single parser can only be used in a
|
52
|
+
single thread. This allows reuse of internal buffers for additional
|
53
|
+
improvements in performance.
|
54
|
+
|
55
|
+
The performane advantage of the Oj::Parse is that it is more than 3
|
56
|
+
times faster than the Oj::compat_load call and 6 times faster than the
|
57
|
+
JSON gem.
|
58
|
+
|
59
|
+
### Dump Performance
|
60
|
+
|
61
|
+
Thanks to Watson1978 Oj.dump also received a speed boost.
|
data/ext/oj/custom.c
CHANGED
@@ -955,8 +955,8 @@ static void hash_set_cstr(ParseInfo pi, Val kval, const char *str, size_t len, c
|
|
955
955
|
}
|
956
956
|
}
|
957
957
|
} else {
|
958
|
-
|
959
|
-
volatile VALUE rstr = rb_utf8_str_new(str, len);
|
958
|
+
volatile VALUE rstr = oj_cstr_to_value(str, len, (size_t)pi->options.cache_str);
|
959
|
+
//volatile VALUE rstr = rb_utf8_str_new(str, len);
|
960
960
|
|
961
961
|
if (Qundef == rkey) {
|
962
962
|
if (Yes == pi->options.sym_key) {
|
data/ext/oj/fast.c
CHANGED
@@ -879,6 +879,10 @@ static Leaf get_leaf(Leaf *stack, Leaf *lp, const char *path) {
|
|
879
879
|
}
|
880
880
|
} else if (NULL == leaf->elements) {
|
881
881
|
leaf = NULL;
|
882
|
+
} else if (STR_VAL == leaf->value_type || RUBY_VAL == leaf->value_type) {
|
883
|
+
// We are trying to get a children of a leaf, which
|
884
|
+
// doesn't exist.
|
885
|
+
leaf = NULL;
|
882
886
|
} else if (COL_VAL == leaf->value_type) {
|
883
887
|
Leaf first = leaf->elements->next;
|
884
888
|
Leaf e = first;
|
@@ -1373,8 +1377,8 @@ static VALUE doc_fetch(int argc, VALUE *argv, VALUE self) {
|
|
1373
1377
|
* Returns true if the value at the location identified by the path exists.
|
1374
1378
|
* @param [String] path path to the location
|
1375
1379
|
* @example
|
1376
|
-
* Oj::Doc.open('[1,2]') { |doc| doc.exists('/1') } #=> true
|
1377
|
-
* Oj::Doc.open('[1,2]') { |doc| doc.exists('/3') } #=> false
|
1380
|
+
* Oj::Doc.open('[1,2]') { |doc| doc.exists?('/1') } #=> true
|
1381
|
+
* Oj::Doc.open('[1,2]') { |doc| doc.exists?('/3') } #=> false
|
1378
1382
|
*/
|
1379
1383
|
static VALUE doc_exists(VALUE self, VALUE str) {
|
1380
1384
|
Doc doc;
|
data/ext/oj/intern.c
CHANGED
@@ -186,6 +186,7 @@ static VALUE resolve_classpath(ParseInfo pi, const char *name, size_t len, int a
|
|
186
186
|
char * end = class_name + sizeof(class_name) - 1;
|
187
187
|
char * s;
|
188
188
|
const char *n = name;
|
189
|
+
size_t nlen = len;
|
189
190
|
|
190
191
|
clas = rb_cObject;
|
191
192
|
for (s = class_name; 0 < len; n++, len--) {
|
@@ -208,7 +209,12 @@ static VALUE resolve_classpath(ParseInfo pi, const char *name, size_t len, int a
|
|
208
209
|
}
|
209
210
|
*s = '\0';
|
210
211
|
if (Qundef == (clas = resolve_classname(clas, class_name, auto_define))) {
|
211
|
-
|
212
|
+
if (sizeof(class_name) <= nlen) {
|
213
|
+
nlen = sizeof(class_name) - 1;
|
214
|
+
}
|
215
|
+
strncpy(class_name, name, nlen);
|
216
|
+
class_name[nlen] = '\0';
|
217
|
+
oj_set_error_at(pi, error_class, __FILE__, __LINE__, "class '%s' is not defined", class_name);
|
212
218
|
if (Qnil != error_class) {
|
213
219
|
pi->err_class = error_class;
|
214
220
|
}
|
data/ext/oj/mimic_json.c
CHANGED
@@ -516,7 +516,7 @@ static VALUE mimic_parse_core(int argc, VALUE *argv, VALUE self, bool bang) {
|
|
516
516
|
pi.options = oj_default_options;
|
517
517
|
pi.options.auto_define = No;
|
518
518
|
pi.options.quirks_mode = Yes;
|
519
|
-
pi.options.allow_invalid =
|
519
|
+
pi.options.allow_invalid = Yes;
|
520
520
|
pi.options.empty_string = No;
|
521
521
|
pi.options.create_ok = No;
|
522
522
|
pi.options.allow_nan = (bang ? Yes : No);
|
@@ -573,8 +573,7 @@ static VALUE mimic_parse_core(int argc, VALUE *argv, VALUE self, bool bang) {
|
|
573
573
|
}
|
574
574
|
}
|
575
575
|
if (oj_hash_has_key(ropts, oj_decimal_class_sym)) {
|
576
|
-
pi.options.compat_bigdec = (oj_bigdecimal_class ==
|
577
|
-
rb_hash_lookup(ropts, oj_decimal_class_sym));
|
576
|
+
pi.options.compat_bigdec = (oj_bigdecimal_class == rb_hash_lookup(ropts, oj_decimal_class_sym));
|
578
577
|
}
|
579
578
|
v = rb_hash_lookup(ropts, oj_max_nesting_sym);
|
580
579
|
if (Qtrue == v) {
|
@@ -682,7 +681,7 @@ static VALUE mimic_set_create_id(VALUE self, VALUE id) {
|
|
682
681
|
*/
|
683
682
|
static VALUE mimic_create_id(VALUE self) {
|
684
683
|
if (NULL != oj_default_options.create_id) {
|
685
|
-
return
|
684
|
+
return rb_utf8_str_new(oj_default_options.create_id, oj_default_options.create_id_len);
|
686
685
|
}
|
687
686
|
return rb_str_new_cstr(oj_json_class);
|
688
687
|
}
|
@@ -706,7 +705,7 @@ static struct _options mimic_object_to_json_options = {0, // indent
|
|
706
705
|
No, // empty_string
|
707
706
|
Yes, // allow_gc
|
708
707
|
Yes, // quirks_mode
|
709
|
-
|
708
|
+
Yes, // allow_invalid
|
710
709
|
No, // create_ok
|
711
710
|
No, // allow_nan
|
712
711
|
No, // trace
|
data/ext/oj/object.c
CHANGED
@@ -35,7 +35,7 @@ static VALUE calc_hash_key(ParseInfo pi, Val kval, char k1) {
|
|
35
35
|
return ID2SYM(rb_intern3(kval->key + 1, kval->klen - 1, oj_utf8_encoding));
|
36
36
|
}
|
37
37
|
if (Yes == pi->options.sym_key) {
|
38
|
-
|
38
|
+
return ID2SYM(rb_intern3(kval->key, kval->klen, oj_utf8_encoding));
|
39
39
|
}
|
40
40
|
#if HAVE_RB_ENC_INTERNED_STR
|
41
41
|
rkey = rb_enc_interned_str(kval->key, kval->klen, oj_utf8_encoding);
|
@@ -60,21 +60,16 @@ static VALUE str_to_value(ParseInfo pi, const char *str, size_t len, const char
|
|
60
60
|
}
|
61
61
|
rstr = oj_circ_array_get(pi->circ_array, i);
|
62
62
|
} else {
|
63
|
-
|
63
|
+
rstr = rb_utf8_str_new(str, len);
|
64
64
|
}
|
65
65
|
return rstr;
|
66
66
|
}
|
67
67
|
|
68
|
-
#if (RUBY_VERSION_MAJOR == 1 && RUBY_VERSION_MINOR == 8)
|
69
|
-
static VALUE oj_parse_xml_time(const char *str, int len) {
|
70
|
-
return rb_funcall(rb_cTime, oj_parse_id, 1, rb_str_new(str, len));
|
71
|
-
}
|
72
|
-
#else
|
73
68
|
// The much faster approach (4x faster)
|
74
69
|
static int parse_num(const char *str, const char *end, int cnt) {
|
75
|
-
int
|
70
|
+
int n = 0;
|
76
71
|
char c;
|
77
|
-
int
|
72
|
+
int i;
|
78
73
|
|
79
74
|
for (i = cnt; 0 < i; i--, str++) {
|
80
75
|
c = *str;
|
@@ -88,9 +83,9 @@ static int parse_num(const char *str, const char *end, int cnt) {
|
|
88
83
|
|
89
84
|
VALUE
|
90
85
|
oj_parse_xml_time(const char *str, int len) {
|
91
|
-
VALUE
|
86
|
+
VALUE args[8];
|
92
87
|
const char *end = str + len;
|
93
|
-
int
|
88
|
+
int n;
|
94
89
|
|
95
90
|
// year
|
96
91
|
if (0 > (n = parse_num(str, end, 4))) {
|
@@ -201,7 +196,6 @@ oj_parse_xml_time(const char *str, int len) {
|
|
201
196
|
}
|
202
197
|
return rb_funcall2(rb_cTime, oj_new_id, 7, args);
|
203
198
|
}
|
204
|
-
#endif
|
205
199
|
|
206
200
|
static int hat_cstr(ParseInfo pi, Val parent, Val kval, const char *str, size_t len) {
|
207
201
|
const char *key = kval->key;
|
@@ -226,13 +220,10 @@ static int hat_cstr(ParseInfo pi, Val parent, Val kval, const char *str, size_t
|
|
226
220
|
}
|
227
221
|
parent->val = odd->clas;
|
228
222
|
parent->odd_args = oj_odd_alloc_args(odd);
|
229
|
-
} break;
|
230
|
-
case 'm':
|
231
|
-
parent->val = ID2SYM(rb_intern3(str + 1, len - 1, oj_utf8_encoding));
|
232
|
-
break;
|
233
|
-
case 's':
|
234
|
-
parent->val = rb_utf8_str_new(str, len);
|
235
223
|
break;
|
224
|
+
}
|
225
|
+
case 'm': parent->val = ID2SYM(rb_intern3(str + 1, len - 1, oj_utf8_encoding)); break;
|
226
|
+
case 's': parent->val = rb_utf8_str_new(str, len); break;
|
236
227
|
case 'c': // class
|
237
228
|
{
|
238
229
|
VALUE clas = oj_name2class(pi, str, len, Yes == pi->options.auto_define, rb_eArgError);
|
@@ -242,7 +233,8 @@ static int hat_cstr(ParseInfo pi, Val parent, Val kval, const char *str, size_t
|
|
242
233
|
} else {
|
243
234
|
parent->val = clas;
|
244
235
|
}
|
245
|
-
|
236
|
+
break;
|
237
|
+
}
|
246
238
|
case 't': // time
|
247
239
|
parent->val = oj_parse_xml_time(str, (int)len);
|
248
240
|
break;
|
@@ -282,22 +274,21 @@ static int hat_num(ParseInfo pi, Val parent, Val kval, NumInfo ni) {
|
|
282
274
|
VALUE args[8];
|
283
275
|
|
284
276
|
sec_as_time(t, &ti);
|
285
|
-
args[0]
|
286
|
-
args[1]
|
287
|
-
args[2]
|
288
|
-
args[3]
|
289
|
-
args[4]
|
290
|
-
args[5]
|
291
|
-
args[6]
|
277
|
+
args[0] = LONG2NUM((long)(ti.year));
|
278
|
+
args[1] = LONG2NUM(ti.mon);
|
279
|
+
args[2] = LONG2NUM(ti.day);
|
280
|
+
args[3] = LONG2NUM(ti.hour);
|
281
|
+
args[4] = LONG2NUM(ti.min);
|
282
|
+
args[5] = rb_float_new((double)ti.sec + ((double)nsec + 0.5) / 1000000000.0);
|
283
|
+
args[6] = LONG2NUM(ni->exp);
|
292
284
|
parent->val = rb_funcall2(rb_cTime, oj_new_id, 7, args);
|
293
285
|
} else {
|
294
286
|
parent->val = rb_time_nano_new(ni->i, (long)nsec);
|
295
287
|
}
|
296
288
|
}
|
297
289
|
break;
|
298
|
-
case 'i':
|
299
|
-
if (!ni->infinity && !ni->neg && 1 == ni->div && 0 == ni->exp &&
|
300
|
-
0 != pi->circ_array) { // fixnum
|
290
|
+
case 'i': // circular index
|
291
|
+
if (!ni->infinity && !ni->neg && 1 == ni->div && 0 == ni->exp && 0 != pi->circ_array) { // fixnum
|
301
292
|
if (Qnil == parent->val) {
|
302
293
|
parent->val = rb_hash_new();
|
303
294
|
}
|
@@ -402,9 +393,7 @@ WHICH_TYPE:
|
|
402
393
|
}
|
403
394
|
break;
|
404
395
|
case T_HASH:
|
405
|
-
rb_hash_aset(parent->val,
|
406
|
-
calc_hash_key(pi, kval, parent->k1),
|
407
|
-
str_to_value(pi, str, len, orig));
|
396
|
+
rb_hash_aset(parent->val, calc_hash_key(pi, kval, parent->k1), str_to_value(pi, str, len, orig));
|
408
397
|
break;
|
409
398
|
case T_STRING:
|
410
399
|
rval = str_to_value(pi, str, len, orig);
|
@@ -481,8 +470,8 @@ WHICH_TYPE:
|
|
481
470
|
rb_hash_aset(parent->val, calc_hash_key(pi, kval, parent->k1), rval);
|
482
471
|
break;
|
483
472
|
case T_OBJECT:
|
484
|
-
if (2 == klen && '^' == *key && 'i' == key[1] && !ni->infinity && !ni->neg &&
|
485
|
-
|
473
|
+
if (2 == klen && '^' == *key && 'i' == key[1] && !ni->infinity && !ni->neg && 1 == ni->div && 0 == ni->exp &&
|
474
|
+
0 != pi->circ_array) { // fixnum
|
486
475
|
oj_circ_array_set(pi->circ_array, parent->val, ni->i);
|
487
476
|
} else {
|
488
477
|
rval = oj_num_as_value(ni);
|
@@ -559,11 +548,7 @@ WHICH_TYPE:
|
|
559
548
|
volatile VALUE *a = RARRAY_PTR(value);
|
560
549
|
|
561
550
|
if (2 != len) {
|
562
|
-
oj_set_error_at(pi,
|
563
|
-
oj_parse_error_class,
|
564
|
-
__FILE__,
|
565
|
-
__LINE__,
|
566
|
-
"invalid hash pair");
|
551
|
+
oj_set_error_at(pi, oj_parse_error_class, __FILE__, __LINE__, "invalid hash pair");
|
567
552
|
return;
|
568
553
|
}
|
569
554
|
rb_hash_aset(parent->val, *a, a[1]);
|
@@ -637,10 +622,7 @@ static void end_hash(ParseInfo pi) {
|
|
637
622
|
} else if (NULL != parent->odd_args) {
|
638
623
|
OddArgs oa = parent->odd_args;
|
639
624
|
|
640
|
-
parent->val = rb_funcall2(oa->odd->create_obj,
|
641
|
-
oa->odd->create_op,
|
642
|
-
oa->odd->attr_cnt,
|
643
|
-
oa->args);
|
625
|
+
parent->val = rb_funcall2(oa->odd->create_obj, oa->odd->create_op, oa->odd->attr_cnt, oa->args);
|
644
626
|
oj_odd_free(oa);
|
645
627
|
parent->odd_args = NULL;
|
646
628
|
}
|
@@ -653,8 +635,7 @@ static void array_append_cstr(ParseInfo pi, const char *str, size_t len, const c
|
|
653
635
|
volatile VALUE rval = Qnil;
|
654
636
|
|
655
637
|
// orig lets us know whether the string was ^r1 or \u005er1
|
656
|
-
if (3 <= len && 0 != pi->circ_array && '^' == orig[0] &&
|
657
|
-
0 == rb_array_len(stack_peek(&pi->stack)->val)) {
|
638
|
+
if (3 <= len && 0 != pi->circ_array && '^' == orig[0] && 0 == rb_array_len(stack_peek(&pi->stack)->val)) {
|
658
639
|
if ('i' == str[1]) {
|
659
640
|
long i = read_long(str + 2, len - 2);
|
660
641
|
|
data/ext/oj/parse.c
CHANGED
@@ -904,9 +904,17 @@ void oj_set_error_at(ParseInfo pi,
|
|
904
904
|
char * end = p + sizeof(msg) - 2;
|
905
905
|
char * start;
|
906
906
|
Val vp;
|
907
|
+
int mlen;
|
907
908
|
|
908
909
|
va_start(ap, format);
|
909
|
-
|
910
|
+
mlen = vsnprintf(msg, sizeof(msg) - 1, format, ap);
|
911
|
+
if (0 < mlen) {
|
912
|
+
if (sizeof(msg) - 2 < (size_t)mlen) {
|
913
|
+
p = end - 2;
|
914
|
+
} else {
|
915
|
+
p += mlen;
|
916
|
+
}
|
917
|
+
}
|
910
918
|
va_end(ap);
|
911
919
|
pi->err.clas = err_clas;
|
912
920
|
if (p + 3 < end) {
|
data/ext/oj/parser.c
CHANGED
@@ -610,6 +610,9 @@ static void parse(ojParser p, const byte *json) {
|
|
610
610
|
printf("*** parse - mode: %c %s\n", p->map[256], (const char *)json);
|
611
611
|
#endif
|
612
612
|
for (; '\0' != *b; b++) {
|
613
|
+
#if DEBUG
|
614
|
+
printf("*** parse - mode: %c %02x %s => %c\n", p->map[256], *b, b, p->map[*b]);
|
615
|
+
#endif
|
613
616
|
switch (p->map[*b]) {
|
614
617
|
case SKIP_NEWLINE:
|
615
618
|
p->line++;
|
@@ -887,13 +890,17 @@ static void parse(ojParser p, const byte *json) {
|
|
887
890
|
buf_append_string(&p->buf, (const char *)start, b - start);
|
888
891
|
}
|
889
892
|
if ('"' == *b) {
|
893
|
+
p->funcs[p->stack[p->depth]].add_str(p);
|
890
894
|
p->map = p->next_map;
|
891
895
|
break;
|
892
896
|
}
|
893
897
|
b--;
|
894
898
|
break;
|
895
899
|
case STR_SLASH: p->map = esc_map; break;
|
896
|
-
case STR_QUOTE:
|
900
|
+
case STR_QUOTE:
|
901
|
+
p->funcs[p->stack[p->depth]].add_str(p);
|
902
|
+
p->map = p->next_map;
|
903
|
+
break;
|
897
904
|
case ESC_U:
|
898
905
|
p->map = u_map;
|
899
906
|
p->ri = 0;
|
@@ -1135,13 +1142,42 @@ extern void oj_set_parser_saj(ojParser p);
|
|
1135
1142
|
extern void oj_set_parser_usual(ojParser p);
|
1136
1143
|
extern void oj_set_parser_debug(ojParser p);
|
1137
1144
|
|
1145
|
+
static int opt_cb(VALUE rkey, VALUE value, VALUE ptr) {
|
1146
|
+
ojParser p = (ojParser)ptr;
|
1147
|
+
const char *key = NULL;
|
1148
|
+
char set_key[64];
|
1149
|
+
long klen;
|
1150
|
+
|
1151
|
+
switch (rb_type(rkey)) {
|
1152
|
+
case RUBY_T_SYMBOL:
|
1153
|
+
rkey = rb_sym2str(rkey);
|
1154
|
+
// fall through
|
1155
|
+
case RUBY_T_STRING:
|
1156
|
+
key = rb_string_value_ptr(&rkey);
|
1157
|
+
klen = RSTRING_LEN(rkey);
|
1158
|
+
break;
|
1159
|
+
default: rb_raise(rb_eArgError, "option keys must be a symbol or string");
|
1160
|
+
}
|
1161
|
+
if ((long)sizeof(set_key) - 1 <= klen) {
|
1162
|
+
return ST_CONTINUE;
|
1163
|
+
}
|
1164
|
+
memcpy(set_key, key, klen);
|
1165
|
+
set_key[klen] = '=';
|
1166
|
+
set_key[klen + 1] = '\0';
|
1167
|
+
p->option(p, set_key, value);
|
1168
|
+
|
1169
|
+
return ST_CONTINUE;
|
1170
|
+
}
|
1171
|
+
|
1138
1172
|
/* Document-method: new
|
1139
1173
|
* call-seq: new(mode=nil)
|
1140
1174
|
*
|
1141
1175
|
* Creates a new Parser with the specified mode. If no mode is provided
|
1142
|
-
* validation is assumed.
|
1176
|
+
* validation is assumed. Optional arguments can be provided that match the
|
1177
|
+
* mode. For example with the :usual mode the call might look like
|
1178
|
+
* Oj::Parser.new(:usual, cache_keys: true).
|
1143
1179
|
*/
|
1144
|
-
static VALUE parser_new(VALUE
|
1180
|
+
static VALUE parser_new(int argc, VALUE *argv, VALUE self) {
|
1145
1181
|
ojParser p = ALLOC(struct _ojParser);
|
1146
1182
|
|
1147
1183
|
#if HAVE_RB_EXT_RACTOR_SAFE
|
@@ -1151,33 +1187,45 @@ static VALUE parser_new(VALUE self, VALUE mode) {
|
|
1151
1187
|
memset(p, 0, sizeof(struct _ojParser));
|
1152
1188
|
buf_init(&p->key);
|
1153
1189
|
buf_init(&p->buf);
|
1154
|
-
|
1155
1190
|
p->map = value_map;
|
1156
|
-
|
1157
|
-
|
1191
|
+
|
1192
|
+
if (argc < 1) {
|
1193
|
+
oj_set_parser_validator(p);
|
1158
1194
|
} else {
|
1159
|
-
|
1160
|
-
|
1161
|
-
|
1162
|
-
case RUBY_T_SYMBOL:
|
1163
|
-
mode = rb_sym2str(mode);
|
1164
|
-
// fall through
|
1165
|
-
case RUBY_T_STRING: ms = RSTRING_PTR(mode); break;
|
1166
|
-
default: rb_raise(rb_eArgError, "mode must be :validate, :usual, :saj, or :object");
|
1167
|
-
}
|
1168
|
-
if (0 == strcmp("usual", ms) || 0 == strcmp("standard", ms) || 0 == strcmp("strict", ms) ||
|
1169
|
-
0 == strcmp("compat", ms)) {
|
1170
|
-
oj_set_parser_usual(p);
|
1171
|
-
} else if (0 == strcmp("object", ms)) {
|
1172
|
-
// TBD
|
1173
|
-
} else if (0 == strcmp("saj", ms)) {
|
1174
|
-
oj_set_parser_saj(p);
|
1175
|
-
} else if (0 == strcmp("validate", ms)) {
|
1195
|
+
VALUE mode = argv[0];
|
1196
|
+
|
1197
|
+
if (Qnil == mode) {
|
1176
1198
|
oj_set_parser_validator(p);
|
1177
|
-
} else if (0 == strcmp("debug", ms)) {
|
1178
|
-
oj_set_parser_debug(p);
|
1179
1199
|
} else {
|
1180
|
-
|
1200
|
+
const char *ms = NULL;
|
1201
|
+
|
1202
|
+
switch (rb_type(mode)) {
|
1203
|
+
case RUBY_T_SYMBOL:
|
1204
|
+
mode = rb_sym2str(mode);
|
1205
|
+
// fall through
|
1206
|
+
case RUBY_T_STRING: ms = RSTRING_PTR(mode); break;
|
1207
|
+
default: rb_raise(rb_eArgError, "mode must be :validate, :usual, :saj, or :object");
|
1208
|
+
}
|
1209
|
+
if (0 == strcmp("usual", ms) || 0 == strcmp("standard", ms) || 0 == strcmp("strict", ms) ||
|
1210
|
+
0 == strcmp("compat", ms)) {
|
1211
|
+
oj_set_parser_usual(p);
|
1212
|
+
} else if (0 == strcmp("object", ms)) {
|
1213
|
+
// TBD
|
1214
|
+
} else if (0 == strcmp("saj", ms)) {
|
1215
|
+
oj_set_parser_saj(p);
|
1216
|
+
} else if (0 == strcmp("validate", ms)) {
|
1217
|
+
oj_set_parser_validator(p);
|
1218
|
+
} else if (0 == strcmp("debug", ms)) {
|
1219
|
+
oj_set_parser_debug(p);
|
1220
|
+
} else {
|
1221
|
+
rb_raise(rb_eArgError, "mode must be :validate, :usual, :saj, or :object");
|
1222
|
+
}
|
1223
|
+
}
|
1224
|
+
if (1 < argc) {
|
1225
|
+
VALUE ropts = argv[1];
|
1226
|
+
|
1227
|
+
Check_Type(ropts, T_HASH);
|
1228
|
+
rb_hash_foreach(ropts, opt_cb, (VALUE)p);
|
1181
1229
|
}
|
1182
1230
|
}
|
1183
1231
|
return Data_Wrap_Struct(parser_class, parser_mark, parser_free, p);
|
@@ -1204,7 +1252,7 @@ static VALUE parser_new(VALUE self, VALUE mode) {
|
|
1204
1252
|
* - *:usual*
|
1205
1253
|
* - _cache_keys=_ sets the value of the _cache_keys_ flag.
|
1206
1254
|
* - _cache_keys_ returns the value of the _cache_keys_ flag.
|
1207
|
-
* - _cache_strings=_ sets the value of the _cache_strings_ to
|
1255
|
+
* - _cache_strings=_ sets the value of the _cache_strings_ to a positive integer less than 35. Strings shorter than
|
1208
1256
|
* that length are cached.
|
1209
1257
|
* - _cache_strings_ returns the value of the _cache_strings_ integer value.
|
1210
1258
|
* - _cache_expunge=_ sets the value of the _cache_expunge_ where 0 never expunges, 1 expunges slowly, 2 expunges
|
@@ -1469,7 +1517,7 @@ static VALUE parser_validate(VALUE self) {
|
|
1469
1517
|
*/
|
1470
1518
|
void oj_parser_init() {
|
1471
1519
|
parser_class = rb_define_class_under(Oj, "Parser", rb_cObject);
|
1472
|
-
rb_define_module_function(parser_class, "new", parser_new, 1);
|
1520
|
+
rb_define_module_function(parser_class, "new", parser_new, -1);
|
1473
1521
|
rb_define_method(parser_class, "parse", parser_parse, 1);
|
1474
1522
|
rb_define_method(parser_class, "load", parser_load, 1);
|
1475
1523
|
rb_define_method(parser_class, "file", parser_file, 1);
|
data/ext/oj/strict.c
CHANGED
@@ -16,7 +16,7 @@
|
|
16
16
|
VALUE oj_cstr_to_value(const char *str, size_t len, size_t cache_str) {
|
17
17
|
volatile VALUE rstr = Qnil;
|
18
18
|
|
19
|
-
if (len
|
19
|
+
if (len < cache_str) {
|
20
20
|
rstr = oj_str_intern(str, len);
|
21
21
|
} else {
|
22
22
|
rstr = rb_str_new(str, len);
|
@@ -37,7 +37,7 @@ VALUE oj_calc_hash_key(ParseInfo pi, Val parent) {
|
|
37
37
|
} else {
|
38
38
|
rkey = rb_str_new(parent->key, parent->klen);
|
39
39
|
rkey = oj_encode(rkey);
|
40
|
-
|
40
|
+
OBJ_FREEZE(rkey); // frozen when used as a Hash key anyway
|
41
41
|
}
|
42
42
|
return rkey;
|
43
43
|
}
|
data/ext/oj/usual.c
CHANGED
@@ -39,7 +39,7 @@ typedef struct _col {
|
|
39
39
|
typedef union _key {
|
40
40
|
struct {
|
41
41
|
int16_t len;
|
42
|
-
char buf[
|
42
|
+
char buf[30];
|
43
43
|
};
|
44
44
|
struct {
|
45
45
|
int16_t xlen; // should be the same as len
|
@@ -209,21 +209,21 @@ static void push(ojParser p, VALUE v) {
|
|
209
209
|
static VALUE cache_key(ojParser p, Key kp) {
|
210
210
|
Delegate d = (Delegate)p->ctx;
|
211
211
|
|
212
|
-
if ((size_t)kp->len < sizeof(kp->buf)
|
212
|
+
if ((size_t)kp->len < sizeof(kp->buf)) {
|
213
213
|
return cache_intern(d->key_cache, kp->buf, kp->len);
|
214
214
|
}
|
215
215
|
return cache_intern(d->key_cache, kp->key, kp->len);
|
216
216
|
}
|
217
217
|
|
218
218
|
static VALUE str_key(ojParser p, Key kp) {
|
219
|
-
if ((size_t)kp->len < sizeof(kp->buf)
|
219
|
+
if ((size_t)kp->len < sizeof(kp->buf)) {
|
220
220
|
return rb_str_freeze(rb_utf8_str_new(kp->buf, kp->len));
|
221
221
|
}
|
222
222
|
return rb_str_freeze(rb_utf8_str_new(kp->key, kp->len));
|
223
223
|
}
|
224
224
|
|
225
225
|
static VALUE sym_key(ojParser p, Key kp) {
|
226
|
-
if ((size_t)kp->len < sizeof(kp->buf)
|
226
|
+
if ((size_t)kp->len < sizeof(kp->buf)) {
|
227
227
|
return rb_str_freeze(rb_str_intern(rb_utf8_str_new(kp->buf, kp->len)));
|
228
228
|
}
|
229
229
|
return rb_str_freeze(rb_str_intern(rb_utf8_str_new(kp->key, kp->len)));
|
@@ -232,7 +232,7 @@ static VALUE sym_key(ojParser p, Key kp) {
|
|
232
232
|
static ID get_attr_id(ojParser p, Key kp) {
|
233
233
|
Delegate d = (Delegate)p->ctx;
|
234
234
|
|
235
|
-
if ((size_t)kp->len < sizeof(kp->buf)
|
235
|
+
if ((size_t)kp->len < sizeof(kp->buf)) {
|
236
236
|
return (ID)cache_intern(d->attr_cache, kp->buf, kp->len);
|
237
237
|
}
|
238
238
|
return (ID)cache_intern(d->attr_cache, kp->key, kp->len);
|
@@ -253,7 +253,7 @@ static void push_key(ojParser p) {
|
|
253
253
|
d->kend = d->khead + cap;
|
254
254
|
}
|
255
255
|
d->ktail->len = klen;
|
256
|
-
if (klen
|
256
|
+
if (klen < sizeof(d->ktail->buf)) {
|
257
257
|
memcpy(d->ktail->buf, key, klen);
|
258
258
|
d->ktail->buf[klen] = '\0';
|
259
259
|
} else {
|
@@ -336,7 +336,7 @@ static void close_object(ojParser p) {
|
|
336
336
|
#if HAVE_RB_HASH_BULK_INSERT
|
337
337
|
for (vp = head; kp < d->ktail; kp++, vp += 2) {
|
338
338
|
*vp = d->get_key(p, kp);
|
339
|
-
if (sizeof(kp->buf)
|
339
|
+
if (sizeof(kp->buf) <= (size_t)kp->len) {
|
340
340
|
xfree(kp->key);
|
341
341
|
}
|
342
342
|
}
|
@@ -344,7 +344,7 @@ static void close_object(ojParser p) {
|
|
344
344
|
#else
|
345
345
|
for (vp = head; kp < d->ktail; kp++, vp += 2) {
|
346
346
|
rb_hash_aset(obj, d->get_key(p, kp), *(vp + 1));
|
347
|
-
if (sizeof(kp->buf)
|
347
|
+
if (sizeof(kp->buf) <= (size_t)kp->len) {
|
348
348
|
xfree(kp->key);
|
349
349
|
}
|
350
350
|
}
|
@@ -368,7 +368,7 @@ static void close_object_class(ojParser p) {
|
|
368
368
|
|
369
369
|
for (vp = head; kp < d->ktail; kp++, vp += 2) {
|
370
370
|
rb_funcall(obj, hset_id, 2, d->get_key(p, kp), *(vp + 1));
|
371
|
-
if (sizeof(kp->buf)
|
371
|
+
if (sizeof(kp->buf) <= (size_t)kp->len) {
|
372
372
|
xfree(kp->key);
|
373
373
|
}
|
374
374
|
}
|
@@ -396,7 +396,7 @@ static void close_object_create(ojParser p) {
|
|
396
396
|
#if HAVE_RB_HASH_BULK_INSERT
|
397
397
|
for (vp = head; kp < d->ktail; kp++, vp += 2) {
|
398
398
|
*vp = d->get_key(p, kp);
|
399
|
-
if (sizeof(kp->buf)
|
399
|
+
if (sizeof(kp->buf) <= (size_t)kp->len) {
|
400
400
|
xfree(kp->key);
|
401
401
|
}
|
402
402
|
}
|
@@ -404,7 +404,7 @@ static void close_object_create(ojParser p) {
|
|
404
404
|
#else
|
405
405
|
for (vp = head; kp < d->ktail; kp++, vp += 2) {
|
406
406
|
rb_hash_aset(obj, d->get_key(p, kp), *(vp + 1));
|
407
|
-
if (sizeof(kp->buf)
|
407
|
+
if (sizeof(kp->buf) <= (size_t)kp->len) {
|
408
408
|
xfree(kp->key);
|
409
409
|
}
|
410
410
|
}
|
@@ -413,7 +413,7 @@ static void close_object_create(ojParser p) {
|
|
413
413
|
obj = rb_class_new_instance(0, NULL, d->hash_class);
|
414
414
|
for (vp = head; kp < d->ktail; kp++, vp += 2) {
|
415
415
|
rb_funcall(obj, hset_id, 2, d->get_key(p, kp), *(vp + 1));
|
416
|
-
if (sizeof(kp->buf)
|
416
|
+
if (sizeof(kp->buf) <= (size_t)kp->len) {
|
417
417
|
xfree(kp->key);
|
418
418
|
}
|
419
419
|
}
|
@@ -428,7 +428,7 @@ static void close_object_create(ojParser p) {
|
|
428
428
|
#if HAVE_RB_HASH_BULK_INSERT
|
429
429
|
for (vp = head; kp < d->ktail; kp++, vp += 2) {
|
430
430
|
*vp = d->get_key(p, kp);
|
431
|
-
if (sizeof(kp->buf)
|
431
|
+
if (sizeof(kp->buf) <= (size_t)kp->len) {
|
432
432
|
xfree(kp->key);
|
433
433
|
}
|
434
434
|
}
|
@@ -436,7 +436,7 @@ static void close_object_create(ojParser p) {
|
|
436
436
|
#else
|
437
437
|
for (vp = head; kp < d->ktail; kp++, vp += 2) {
|
438
438
|
rb_hash_aset(arg, d->get_key(p, kp), *(vp + 1));
|
439
|
-
if (sizeof(kp->buf)
|
439
|
+
if (sizeof(kp->buf) <= (size_t)kp->len) {
|
440
440
|
xfree(kp->key);
|
441
441
|
}
|
442
442
|
}
|
@@ -446,7 +446,7 @@ static void close_object_create(ojParser p) {
|
|
446
446
|
obj = rb_class_new_instance(0, NULL, clas);
|
447
447
|
for (vp = head; kp < d->ktail; kp++, vp += 2) {
|
448
448
|
rb_ivar_set(obj, get_attr_id(p, kp), *(vp + 1));
|
449
|
-
if (sizeof(kp->buf)
|
449
|
+
if (sizeof(kp->buf) <= (size_t)kp->len) {
|
450
450
|
xfree(kp->key);
|
451
451
|
}
|
452
452
|
}
|
@@ -537,7 +537,7 @@ static void add_float_key(ojParser p) {
|
|
537
537
|
static void add_float_as_big(ojParser p) {
|
538
538
|
char buf[64];
|
539
539
|
|
540
|
-
// fails on ubuntu
|
540
|
+
// snprintf fails on ubuntu and macOS for long double
|
541
541
|
// snprintf(buf, sizeof(buf), "%Lg", p->num.dub);
|
542
542
|
sprintf(buf, "%Lg", p->num.dub);
|
543
543
|
push(p, rb_funcall(rb_cObject, oj_bigdecimal_id, 1, rb_str_new2(buf)));
|
@@ -546,7 +546,9 @@ static void add_float_as_big(ojParser p) {
|
|
546
546
|
static void add_float_as_big_key(ojParser p) {
|
547
547
|
char buf[64];
|
548
548
|
|
549
|
-
snprintf
|
549
|
+
// snprintf fails on ubuntu and macOS for long double
|
550
|
+
// snprintf(buf, sizeof(buf), "%Lg", p->num.dub);
|
551
|
+
sprintf(buf, "%Lg", p->num.dub);
|
550
552
|
push_key(p);
|
551
553
|
push2(p, rb_funcall(rb_cObject, oj_bigdecimal_id, 1, rb_str_new2(buf)));
|
552
554
|
}
|
data/lib/oj/version.rb
CHANGED
data/test/bar.rb
ADDED
@@ -0,0 +1,16 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
$: << '.'
|
4
|
+
$: << File.join(File.dirname(__FILE__), "../lib")
|
5
|
+
$: << File.join(File.dirname(__FILE__), "../ext")
|
6
|
+
|
7
|
+
require 'oj'
|
8
|
+
|
9
|
+
json = %|[{"x12345678901234567890": true}]|
|
10
|
+
|
11
|
+
p = Oj::Parser.new(:usual)
|
12
|
+
p.cache_keys = false
|
13
|
+
p.symbol_keys = true
|
14
|
+
x = p.parse(json)
|
15
|
+
|
16
|
+
pp x
|
data/test/baz.rb
ADDED
@@ -0,0 +1,16 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
$: << File.dirname(__FILE__)
|
4
|
+
$oj_dir = File.dirname(File.expand_path(File.dirname(__FILE__)))
|
5
|
+
%w(lib ext).each do |dir|
|
6
|
+
$: << File.join($oj_dir, dir)
|
7
|
+
end
|
8
|
+
|
9
|
+
require 'oj'
|
10
|
+
Oj.mimic_JSON()
|
11
|
+
|
12
|
+
begin
|
13
|
+
::JSON.load('name=&email=&subject=&comment=&submit=Send+Message')
|
14
|
+
rescue ::JSON::ParserError
|
15
|
+
puts "*** Pass"
|
16
|
+
end
|
data/test/bug.rb
ADDED
@@ -0,0 +1,16 @@
|
|
1
|
+
$: << '.'
|
2
|
+
$: << File.join(File.dirname(__FILE__), "../lib")
|
3
|
+
$: << File.join(File.dirname(__FILE__), "../ext")
|
4
|
+
|
5
|
+
|
6
|
+
#require 'bundler/setup'
|
7
|
+
require 'oj'
|
8
|
+
require 'active_support'
|
9
|
+
require 'active_support/time_with_zone'
|
10
|
+
require 'tzinfo'
|
11
|
+
|
12
|
+
puts ActiveSupport::TimeWithZone
|
13
|
+
|
14
|
+
json = File.read('./bug.json')
|
15
|
+
|
16
|
+
Oj.load(json)
|
data/test/foo.rb
CHANGED
@@ -1,11 +1,14 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
2
|
|
3
3
|
$: << '.'
|
4
|
-
$: <<
|
5
|
-
$: <<
|
4
|
+
$: << File.join(File.dirname(__FILE__), "../lib")
|
5
|
+
$: << File.join(File.dirname(__FILE__), "../ext")
|
6
6
|
|
7
7
|
require 'oj'
|
8
8
|
|
9
|
-
|
9
|
+
parser = Oj::Parser.new(:usual, cache_keys: true, cache_strings: 30, decimal: :bigdecimal, symbol_keys: false)
|
10
|
+
raw = %({"amount":0.10})
|
11
|
+
parsed = parser.parse(raw)
|
10
12
|
|
11
|
-
|
13
|
+
pp parsed
|
14
|
+
puts parsed['amount'].class
|
data/test/prec.rb
ADDED
@@ -0,0 +1,23 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'oj'
|
4
|
+
|
5
|
+
extras = {"locationLng" => -97.14690769100295}
|
6
|
+
|
7
|
+
Oj.default_options = {float_precision: 17}
|
8
|
+
|
9
|
+
encoded = Oj.dump(extras)
|
10
|
+
puts encoded
|
11
|
+
puts Oj.load(encoded)
|
12
|
+
|
13
|
+
require "active_record"
|
14
|
+
|
15
|
+
Oj::Rails.set_encoder()
|
16
|
+
Oj::Rails.set_decoder()
|
17
|
+
|
18
|
+
Oj.default_options = {float_precision: 17}
|
19
|
+
# Using Oj rails encoder, gets the correct value: {"locationLng":-97.14690769100295}
|
20
|
+
encoded = ActiveSupport::JSON.encode(extras)
|
21
|
+
puts encoded
|
22
|
+
puts ActiveSupport::JSON.decode(encoded)
|
23
|
+
puts Oj.load(encoded)
|
data/test/test_fast.rb
CHANGED
@@ -1,5 +1,5 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
|
-
#
|
2
|
+
# frozen_string_literal: true
|
3
3
|
|
4
4
|
$: << File.dirname(__FILE__)
|
5
5
|
|
@@ -36,6 +36,17 @@ class DocTest < Minitest::Test
|
|
36
36
|
end
|
37
37
|
end
|
38
38
|
|
39
|
+
def test_leaf_of_existing_path
|
40
|
+
json = %{{"foo": 1, "fizz": true}}
|
41
|
+
Oj::Doc.open(json) do |doc|
|
42
|
+
%w(/foo/bar /fizz/bar).each do |path|
|
43
|
+
assert_nil(doc.fetch(path))
|
44
|
+
assert_equal(:default, doc.fetch(path, :default))
|
45
|
+
refute(doc.exists?(path))
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
49
|
+
|
39
50
|
def test_true
|
40
51
|
json = %{true}
|
41
52
|
Oj::Doc.open(json) do |doc|
|
@@ -282,11 +293,11 @@ class DocTest < Minitest::Test
|
|
282
293
|
['/nothing', nil],
|
283
294
|
['/array/10', nil],
|
284
295
|
].each do |path,val|
|
285
|
-
|
286
|
-
|
287
|
-
|
296
|
+
if val.nil?
|
297
|
+
assert_nil(doc.fetch(path))
|
298
|
+
else
|
288
299
|
assert_equal(val, doc.fetch(path))
|
289
|
-
|
300
|
+
end
|
290
301
|
end
|
291
302
|
end
|
292
303
|
# verify empty hash and arrays return nil when a member is requested
|
@@ -313,7 +324,7 @@ class DocTest < Minitest::Test
|
|
313
324
|
end
|
314
325
|
end
|
315
326
|
|
316
|
-
def
|
327
|
+
def test_exists
|
317
328
|
Oj::Doc.open(@json1) do |doc|
|
318
329
|
[['/array/1', true],
|
319
330
|
['/array/1', true],
|
@@ -322,7 +333,7 @@ class DocTest < Minitest::Test
|
|
322
333
|
['/array/3', false],
|
323
334
|
['/nothing', false],
|
324
335
|
].each do |path,val|
|
325
|
-
assert_equal(val, doc.exists?(path))
|
336
|
+
assert_equal(val, doc.exists?(path), "failed for #{path.inspect}")
|
326
337
|
end
|
327
338
|
end
|
328
339
|
end
|
data/test/test_parser_usual.rb
CHANGED
@@ -1,5 +1,5 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
|
-
# encoding:
|
2
|
+
# encoding: utf-8
|
3
3
|
|
4
4
|
$: << File.dirname(__FILE__)
|
5
5
|
|
@@ -73,9 +73,14 @@ class UsualTest < Minitest::Test
|
|
73
73
|
assert_equal({a: true, b: false}, doc)
|
74
74
|
end
|
75
75
|
|
76
|
-
def
|
76
|
+
def test_strings
|
77
77
|
p = Oj::Parser.new(:usual)
|
78
|
-
p.
|
78
|
+
doc = p.parse('{"ぴ": "", "ぴ ": "x", "c": "ぴーたー", "d": " ぴーたー "}')
|
79
|
+
assert_equal({'ぴ' => '', 'ぴ ' => 'x', 'c' => 'ぴーたー', 'd' => ' ぴーたー '}, doc)
|
80
|
+
end
|
81
|
+
|
82
|
+
def test_capacity
|
83
|
+
p = Oj::Parser.new(:usual, capacity: 1000)
|
79
84
|
assert_equal(4096, p.capacity)
|
80
85
|
p.capacity = 5000
|
81
86
|
assert_equal(5000, p.capacity)
|
@@ -181,8 +186,7 @@ class UsualTest < Minitest::Test
|
|
181
186
|
end
|
182
187
|
|
183
188
|
def test_missing_class
|
184
|
-
p = Oj::Parser.new(:usual)
|
185
|
-
p.create_id = '^'
|
189
|
+
p = Oj::Parser.new(:usual, create_id: '^')
|
186
190
|
json = '{"a":true,"^":"Auto","b":false}'
|
187
191
|
doc = p.parse(json)
|
188
192
|
assert_equal(Hash, doc.class)
|
data/test/zoo.rb
ADDED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: oj
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 3.13.
|
4
|
+
version: 3.13.8
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Peter Ohler
|
8
|
-
autorequire:
|
8
|
+
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-09-
|
11
|
+
date: 2021-09-27 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: rake-compiler
|
@@ -79,6 +79,9 @@ extensions:
|
|
79
79
|
- ext/oj/extconf.rb
|
80
80
|
extra_rdoc_files:
|
81
81
|
- README.md
|
82
|
+
- LICENSE
|
83
|
+
- CHANGELOG.md
|
84
|
+
- RELEASE_NOTES.md
|
82
85
|
- pages/Advanced.md
|
83
86
|
- pages/Compatibility.md
|
84
87
|
- pages/Custom.md
|
@@ -94,6 +97,7 @@ files:
|
|
94
97
|
- CHANGELOG.md
|
95
98
|
- LICENSE
|
96
99
|
- README.md
|
100
|
+
- RELEASE_NOTES.md
|
97
101
|
- ext/oj/buf.h
|
98
102
|
- ext/oj/cache.c
|
99
103
|
- ext/oj/cache.h
|
@@ -195,8 +199,9 @@ files:
|
|
195
199
|
- test/activesupport6/test_common.rb
|
196
200
|
- test/activesupport6/test_helper.rb
|
197
201
|
- test/activesupport6/time_zone_test_helpers.rb
|
198
|
-
- test/
|
199
|
-
- test/
|
202
|
+
- test/bar.rb
|
203
|
+
- test/baz.rb
|
204
|
+
- test/bug.rb
|
200
205
|
- test/files.rb
|
201
206
|
- test/foo.rb
|
202
207
|
- test/helper.rb
|
@@ -232,6 +237,7 @@ files:
|
|
232
237
|
- test/perf_simple.rb
|
233
238
|
- test/perf_strict.rb
|
234
239
|
- test/perf_wab.rb
|
240
|
+
- test/prec.rb
|
235
241
|
- test/sample.rb
|
236
242
|
- test/sample/change.rb
|
237
243
|
- test/sample/dir.rb
|
@@ -258,7 +264,6 @@ files:
|
|
258
264
|
- test/test_null.rb
|
259
265
|
- test/test_object.rb
|
260
266
|
- test/test_parser.rb
|
261
|
-
- test/test_parser_memory.rb
|
262
267
|
- test/test_parser_saj.rb
|
263
268
|
- test/test_parser_usual.rb
|
264
269
|
- test/test_rails.rb
|
@@ -271,6 +276,7 @@ files:
|
|
271
276
|
- test/tests.rb
|
272
277
|
- test/tests_mimic.rb
|
273
278
|
- test/tests_mimic_addition.rb
|
279
|
+
- test/zoo.rb
|
274
280
|
homepage: http://www.ohler.com/oj
|
275
281
|
licenses:
|
276
282
|
- MIT
|
@@ -281,7 +287,7 @@ metadata:
|
|
281
287
|
homepage_uri: http://www.ohler.com/oj/
|
282
288
|
source_code_uri: https://github.com/ohler55/oj
|
283
289
|
wiki_uri: https://github.com/ohler55/oj/wiki
|
284
|
-
post_install_message:
|
290
|
+
post_install_message:
|
285
291
|
rdoc_options:
|
286
292
|
- "--title"
|
287
293
|
- Oj
|
@@ -301,7 +307,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
301
307
|
version: '0'
|
302
308
|
requirements: []
|
303
309
|
rubygems_version: 3.2.22
|
304
|
-
signing_key:
|
310
|
+
signing_key:
|
305
311
|
specification_version: 4
|
306
312
|
summary: A fast JSON parser and serializer.
|
307
313
|
test_files:
|
@@ -325,8 +331,9 @@ test_files:
|
|
325
331
|
- test/activesupport6/test_common.rb
|
326
332
|
- test/activesupport6/test_helper.rb
|
327
333
|
- test/activesupport6/time_zone_test_helpers.rb
|
328
|
-
- test/
|
329
|
-
- test/
|
334
|
+
- test/bar.rb
|
335
|
+
- test/baz.rb
|
336
|
+
- test/bug.rb
|
330
337
|
- test/files.rb
|
331
338
|
- test/foo.rb
|
332
339
|
- test/helper.rb
|
@@ -362,6 +369,7 @@ test_files:
|
|
362
369
|
- test/perf_simple.rb
|
363
370
|
- test/perf_strict.rb
|
364
371
|
- test/perf_wab.rb
|
372
|
+
- test/prec.rb
|
365
373
|
- test/sample/change.rb
|
366
374
|
- test/sample/dir.rb
|
367
375
|
- test/sample/doc.rb
|
@@ -388,7 +396,6 @@ test_files:
|
|
388
396
|
- test/test_null.rb
|
389
397
|
- test/test_object.rb
|
390
398
|
- test/test_parser.rb
|
391
|
-
- test/test_parser_memory.rb
|
392
399
|
- test/test_parser_saj.rb
|
393
400
|
- test/test_parser_usual.rb
|
394
401
|
- test/test_rails.rb
|
@@ -401,3 +408,4 @@ test_files:
|
|
401
408
|
- test/tests.rb
|
402
409
|
- test/tests_mimic.rb
|
403
410
|
- test/tests_mimic_addition.rb
|
411
|
+
- test/zoo.rb
|
data/test/benny.rb
DELETED
@@ -1,50 +0,0 @@
|
|
1
|
-
#!/usr/bin/env ruby
|
2
|
-
# frozen_string_literal
|
3
|
-
|
4
|
-
require 'bundler/inline'
|
5
|
-
|
6
|
-
gemfile do
|
7
|
-
source 'https://rubygems.org'
|
8
|
-
|
9
|
-
gem 'oj'
|
10
|
-
gem 'benchmark-ips', require: 'benchmark/ips'
|
11
|
-
end
|
12
|
-
|
13
|
-
require 'json'
|
14
|
-
require 'open-uri'
|
15
|
-
|
16
|
-
CANADA_DATA_JSON = URI.parse('https://raw.githubusercontent.com/serde-rs/json-benchmark/master/data/canada.json').read
|
17
|
-
CANADA_DATA = JSON.parse(CANADA_DATA_JSON)
|
18
|
-
|
19
|
-
Benchmark.ips do |x|
|
20
|
-
x.config(:time => 10, :warmup => 5)
|
21
|
-
|
22
|
-
x.report("marshall Canada data with Oj") do
|
23
|
-
Oj.dump(CANADA_DATA)
|
24
|
-
end
|
25
|
-
|
26
|
-
x.report("marshall Canada data with JSON") do
|
27
|
-
JSON.dump(CANADA_DATA)
|
28
|
-
end
|
29
|
-
|
30
|
-
x.compare!
|
31
|
-
end
|
32
|
-
|
33
|
-
Oj.default_options = {
|
34
|
-
mode: :strict,
|
35
|
-
bigdecimal_load: :fast
|
36
|
-
}
|
37
|
-
|
38
|
-
Benchmark.ips do |x|
|
39
|
-
x.config(:time => 10, :warmup => 5)
|
40
|
-
|
41
|
-
x.report("unmarshall Canada data with Oj") do
|
42
|
-
Oj.load(CANADA_DATA_JSON)
|
43
|
-
end
|
44
|
-
|
45
|
-
x.report("unmarshall Canada data with JSON") do
|
46
|
-
JSON.parse(CANADA_DATA_JSON)
|
47
|
-
end
|
48
|
-
|
49
|
-
x.compare!
|
50
|
-
end
|
data/test/big.rb
DELETED
@@ -1,15 +0,0 @@
|
|
1
|
-
#require 'active_support'
|
2
|
-
#require 'active_support/core_ext'
|
3
|
-
#require 'active_support/json'
|
4
|
-
require 'oj'
|
5
|
-
|
6
|
-
#Oj.optimize_rails
|
7
|
-
Oj.mimic_JSON
|
8
|
-
|
9
|
-
h = {:type=>:record, :name=>:group, :namespace=>"com.salsify.identity", :fields=>[{:name=>"id", :type=>{:name=>:salsify_uuid, :type=>:fixed, :namespace=>"com.salsify", :size=>38}}, {:name=>"type", :type=>"string", :default=>"groups"}, {:name=>"external_id", :type=>[:null, "string"], :default=>nil}, {:name=>"created_at", :type=>{:type=>"long", :logicalType=>"timestamp-micros"}}, {:name=>"updated_at", :type=>{:type=>"long", :logicalType=>"timestamp-micros"}}, {:name=>"name", :type=>"string"}, {:name=>"policy", :type=>[:null, {:type=>:record, :name=>:policy, :namespace=>"com.salsify.security", :fields=>[{:name=>"created_at", :type=>{:type=>"long", :logicalType=>"timestamp-micros"}}, {:name=>"updated_at", :type=>{:type=>"long", :logicalType=>"timestamp-micros"}}, {:name=>"id", :type=>"com.salsify.salsify_uuid"}, {:name=>"type", :type=>"string", :default=>"policies"}, {:name=>"external_id", :type=>[:null, "string"], :default=>nil}, {:name=>"name", :type=>"string"}, {:name=>"statements", :type=>{:type=>:array, :items=>{:type=>:record, :name=>:statement, :namespace=>"com.salsify.security", :fields=>[{:name=>"created_at", :type=>{:type=>"long", :logicalType=>"timestamp-micros"}}, {:name=>"updated_at", :type=>{:type=>"long", :logicalType=>"timestamp-micros"}}, {:name=>"id", :type=>"com.salsify.salsify_uuid"}, {:name=>"type", :type=>"string", :default=>"statements"}, {:name=>"external_id", :type=>[:null, "string"], :default=>nil}, {:name=>"action", :type=>{:name=>"__statement_action_enum", :type=>:enum, :namespace=>"com.salsify.security", :symbols=>[:manage, :read]}}, {:name=>"resource", :type=>{:name=>"__statement_resource_enum", :type=>:enum, :namespace=>"com.salsify.security", :symbols=>[:product, :digital_asset]}}, {:name=>"conditions", :type=>{:type=>:array, :items=>{:type=>:record, :name=>:condition, :namespace=>"com.salsify.security", :fields=>[{:name=>"created_at", :type=>{:type=>"long", :logicalType=>"timestamp-micros"}}, {:name=>"updated_at", :type=>{:type=>"long", :logicalType=>"timestamp-micros"}}, {:name=>"id", :type=>"com.salsify.salsify_uuid"}, {:name=>"type", :type=>"string", :default=>"conditions"}, {:name=>"external_id", :type=>[:null, "string"], :default=>nil}, {:name=>"operator", :type=>{:name=>"__condition_operator_enum", :type=>:enum, :namespace=>"com.salsify.security", :symbols=>[:equals]}}, {:name=>"attribute_type", :type=>{:name=>"__condition_attribute_type_enum", :type=>:enum, :namespace=>"com.salsify.security", :symbols=>[:resource]}}, {:name=>"value", :type=>"string"}, {:name=>"attribute", :type=>[:null, {:type=>:record, :name=>:reference, :namespace=>"com.salsify", :fields=>[{:name=>"id", :type=>"com.salsify.salsify_uuid"}, {:name=>"type", :type=>"string", :doc=>"snake_case, plural name for the resource type"}, {:name=>"external_id", :type=>[:null, "string"], :default=>nil}]}], :default=>nil}, {:name=>"broken", :type=>[:null, "boolean"], :default=>nil}]}}}]}}}]}], :default=>nil}]}
|
10
|
-
|
11
|
-
#Oj.dump(h)
|
12
|
-
puts JSON.pretty_generate(h)
|
13
|
-
#puts JSON.fast_generate(h)
|
14
|
-
#puts JSON.generate(h)
|
15
|
-
|
data/test/test_parser_memory.rb
DELETED
@@ -1,45 +0,0 @@
|
|
1
|
-
#!/usr/bin/env ruby
|
2
|
-
# encoding: utf-8
|
3
|
-
|
4
|
-
%w(lib ext test).each do |dir|
|
5
|
-
$LOAD_PATH.unshift File.expand_path("../../#{dir}", __FILE__)
|
6
|
-
end
|
7
|
-
|
8
|
-
require 'oj'
|
9
|
-
|
10
|
-
json =<<-EOF
|
11
|
-
{
|
12
|
-
"$id": "https://example.com/person.schema.json",
|
13
|
-
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
14
|
-
"title": "Person",
|
15
|
-
"type": "object",
|
16
|
-
"properties": {
|
17
|
-
"firstName": {
|
18
|
-
"type": "string",
|
19
|
-
"description": "The person's first name."
|
20
|
-
},
|
21
|
-
"lastName": {
|
22
|
-
"type": "string",
|
23
|
-
"description": "The person's last name."
|
24
|
-
},
|
25
|
-
"age": {
|
26
|
-
"description": "Age in years which must be equal to or greater than zero.",
|
27
|
-
"type": "integer",
|
28
|
-
"minimum": 0
|
29
|
-
}
|
30
|
-
}
|
31
|
-
}
|
32
|
-
EOF
|
33
|
-
|
34
|
-
#json = '{"abc": true}'
|
35
|
-
#json = '[true,false]'
|
36
|
-
|
37
|
-
100_001.times do |i|
|
38
|
-
p = Oj::Parser.new(:usual).parse(json)
|
39
|
-
|
40
|
-
if i % 10_000 == 0
|
41
|
-
GC.start
|
42
|
-
rss = Integer(`ps -o rss= -p #{Process.pid}`) / 1024.0
|
43
|
-
puts "#{i},#{rss} MB"
|
44
|
-
end
|
45
|
-
end
|