nosj 0.2.0-x86_64-linux → 0.3.0-x86_64-linux

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 46cefa7803d25d779048e2c450ea1c5b5daa0c42508277432ef99df7918c783c
4
- data.tar.gz: b11bcb71b9334785acbc7c3f1c48570f212e77646cb3c613db7cc76261895250
3
+ metadata.gz: 734f363fe006f81b64302674599f01f78239515aba0111125b2db71a8ea70022
4
+ data.tar.gz: 2f13bcd853bd227de0420cc3c86c8452a2201705ea51192ac083c25fe0bf925b
5
5
  SHA512:
6
- metadata.gz: 1789804de8bff93994f515aba7d8fe5ac0f527e865bb8d917c8ed7d9a2cd08e7a3cddb2b0194653697eb62aafa466820c97b01c15a9afde9530a4f8b98bdedd5
7
- data.tar.gz: 5aab59746b4856fb2afb505d18da3c4d9b2c3f0f02e115c402a17e94c5669b911443899cbf0958771606ae95789764ced92b091a1d5917cb186c620570811978
6
+ metadata.gz: 35091b15c193843620049fbc191beaad540c0774b3e6da311bf51dfdbf72f462e517ff3b527a474b0b62686d3ed4d828a4e610c9f5e6707055d19d61e7532327
7
+ data.tar.gz: 909edb0644da114fb00eba22c4c3d9a4d4e09a853b6f4652b28f62bb986f9342f5c7d91803126f213c3af8c4cc3070230f3bba222de1a7087cd47ccafa48d21b
data/CHANGELOG.md CHANGED
@@ -1,3 +1,91 @@
1
+ ## [0.3.0] - 2026-07-17
2
+
3
+ - Reformat without parsing. `NOSJ.minify(json, opts)` and
4
+ `NOSJ.reformat(json, opts)` (plus `NOSJ.reformat_file`) pipe the
5
+ parser's events straight into the emission kernels: zero Ruby
6
+ objects are allocated for the document, and output is exactly
7
+ `generate(parse(json))`—canonical numbers, normalized escapes, the
8
+ full set of `generate` formatting and escape options, with
9
+ `pretty: true` as a `pretty_generate` shorthand—except duplicate
10
+ object keys pass through and lone-surrogate string values re-escape
11
+ as `\uXXXX` instead of raising (the output must always reparse).
12
+ Acceptance options apply per `parse` (`allow_trailing_comma`
13
+ normalizes the commas away). Measured on the 631 KB twitter.json:
14
+ 409µs, 3.4× faster than `NOSJ.generate(NOSJ.parse(x))`, 3.9× faster
15
+ than gem json's cycle, 5.3× faster than Oj's, and 1.4× the cost of
16
+ `NOSJ.valid?`.
17
+ - Byte-splicing edits and JSON Patch. `NOSJ.splice(json, pointer =>
18
+ value, ...)` replaces values directly in the text: every target
19
+ resolves in one forward pass and the result is rebuilt copying all
20
+ bytes outside the target spans untouched (formatting, key order, and
21
+ number spellings elsewhere survive exactly). Measured on
22
+ twitter.json: 10× faster than parse-mutate-generate for a late
23
+ field, 51× for an early one. Missing targets raise KeyError,
24
+ overlapping targets ArgumentError. `NOSJ.patch(json, ops)` applies
25
+ RFC 6902 JSON Patch (add/remove/replace/move/copy/test, String or
26
+ Symbol op keys) to the raw string the same way, with structural ops
27
+ walking only the parent container's span; application failures raise
28
+ the new `NOSJ::PatchError`, malformed patch documents ArgumentError.
29
+ `NOSJ.merge_patch(json, patch)` applies RFC 7386 JSON Merge Patch
30
+ (semantic form). Inserted values are byte-identical to
31
+ `NOSJ.generate` and accept its options; the RFC 6902 appendix-A
32
+ suite and the full RFC 7386 test table are in the specs.
33
+ - NDJSON / JSON Lines. `NOSJ.each_line(source, opts)` yields one
34
+ parsed value per line (Enumerator without a block, so
35
+ `.first(10)`/`.lazy` walk only what they consume), skipping blank
36
+ lines and enforcing one value per line; a malformed line raises the
37
+ rich `ParserError` whose `#line` is the physical line number in the
38
+ stream. `NOSJ.generate_lines(values, opts)` emits one compact
39
+ newline-terminated document per element in a single buffer pass
40
+ (measured 4.1× faster than the map-generate-join idiom on twitter
41
+ statuses; `each_line` is 1.6× faster than line-split +
42
+ `JSON.parse`), rejecting formatting options that would break line
43
+ framing. File forms: `NOSJ.each_line_file` streams over a read-only
44
+ memory map, `NOSJ.write_lines` generates straight to disk and
45
+ returns the byte count. Parse options apply per line; pass a frozen
46
+ string to `each_line` for zero-copy iteration.
47
+ - `NOSJ.stats(source, opts)` / `NOSJ.stats_file(path, opts)`: document
48
+ statistics from one counting pass through the null-sink machinery,
49
+ answering "what is this 40 MB blob" without building any Ruby values
50
+ for the document (measured ~1.3× faster than a full parse). Reports
51
+ `byte_size`, `root` kind, `max_depth`, value counts by type, key
52
+ totals, a key histogram sorted by count, largest container sizes,
53
+ and string byte totals. Nesting is unlimited by default (pass
54
+ `max_nesting` to enforce a limit); `allow_nan` and
55
+ `allow_trailing_comma` are honored; malformed documents raise the
56
+ rich `ParserError`. The file form memory-maps, so the document never
57
+ enters Ruby at all.
58
+ - Rich parse errors. Parse failures now raise `NOSJ::ParserError`
59
+ (previously a bare `RuntimeError`) carrying the failure position,
60
+ computed only when a parse fails: `#byte_offset`, 1-based `#line`,
61
+ character-based `#column`, and a caret `#snippet` showing the
62
+ offending line (windowed when the line is long, as minified JSON
63
+ usually is). Positions are absolute within the document you passed,
64
+ including through partial parsing (`dig`, `at_pointer`, batches),
65
+ lazy documents, and the file APIs. `#detailed_message` appends the
66
+ snippet, so unrescued errors print it. Failures with no position
67
+ (encoding refusals) leave the accessors nil. Exceeding `max_nesting`
68
+ during parsing now raises `NOSJ::NestingError`, matching the gem's
69
+ class (generation already did); rescues of the old `RuntimeError`
70
+ need updating to `NOSJ::ParserError`/`NOSJ::Error`.
71
+ - Rails mode: `require "nosj/rails"` accelerates a Rails application
72
+ in both directions. It installs a nosj-backed ActiveSupport JSON
73
+ encoder, so `obj.to_json`, `render json:`, and `ActiveSupport::JSON.encode` walk the object tree natively—values recurse through `as_json` exactly
74
+ like ActiveSupport's own encoder. It also loads the `nosj/json` drop-in, so
75
+ `ActiveSupport::JSON.decode` and JSON request-body parsing take the
76
+ fast path (including on Rails 7.x, whose `quirks_mode` option the
77
+ drop-in now accepts; the drop-in also accepts valid-UTF-8 BINARY
78
+ strings now, which is what Rack delivers request bodies as). The
79
+ HTML-safety escaping is fused into the SIMD string-emission kernels,
80
+ so escaped output costs the same single pass as unescaped. Measured
81
+ against stock ActiveSupport encoding: ×1.7 on small documents up to
82
+ ×5.2 on large trees and ×14 on HTML-heavy content
83
+ (`rake bench:rails`). In a Rails Gemfile:
84
+ `gem "nosj", require: "nosj/rails"`.
85
+ - `JSON::Fragment` values now splice their pre-rendered JSON
86
+ everywhere the `json` gem does: in default mode, under `strict:
87
+ true`, and through the Rails encoder.
88
+
1
89
  ## [0.2.0] - 2026-07-16
2
90
 
3
91
  - File APIs. `NOSJ.load_file(path, opts)` parses a file directly
data/README.md CHANGED
@@ -13,11 +13,14 @@ faster than Yajl—[see Benchmarks](#benchmarks).
13
13
  - It has **lazy documents**: `NOSJ.lazy` wraps a document and parses a value only when you touch it—repeated access costs nanoseconds, and everything you never read is never parsed.
14
14
  - It has a **partial parsing mode**: JSON Pointer lookups that pull single values out of big documents in microseconds, skipping everything else.
15
15
  - It has **file APIs**: parse, generate, dig, and lazy-wrap files directly—no throwaway file-sized Ruby String, and the partial modes memory-map the file so unread pages never even leave the disk.
16
+ - It does **byte-splicing edits**: `NOSJ.splice` changes a field inside a passing payload 10–51× faster than parse-mutate-generate, and RFC 6902 JSON Patch / RFC 7386 merge patch apply straight to the raw string.
17
+ - It is **great to debug with**: parse errors carry line, column, and a caret snippet pointing at the break, and `NOSJ.stats` X-rays a mystery blob (depth, value counts, key histogram) faster than parsing it.
18
+ - It accelerates a **Rails** application in both encoding and decoding.
16
19
  - It comes **precompiled** (platform gems built with per-platform optimizations,
17
20
  nothing to compile on install).
18
21
  - Otherwise, same API and option names as gem json.
19
22
 
20
- **And there's more**: validate documents without building a single Ruby object, resolve whole batches of paths in one pass, and accelerate an entire application with a one-line drop-in.
23
+ **And there's more**: validate documents without building a single Ruby object, minify and pretty-print without parsing, resolve whole batches of paths in one pass, stream NDJSON / JSON Lines in both directions, and accelerate an entire application with a one-line drop-in.
21
24
 
22
25
  - [Requirements](#requirements)
23
26
  - [Getting started](#getting-started)
@@ -26,6 +29,7 @@ nothing to compile on install).
26
29
  - [Switching from the json gem](#switching-from-the-json-gem)
27
30
  - [How it works](#how-it-works)
28
31
  - [Development](#development)
32
+ - [Acknowledgements](#acknowledgements)
29
33
  - [License](#license)
30
34
 
31
35
  ## Requirements
@@ -50,6 +54,8 @@ NOSJ.generate({"a" => [1, true]}) #=> '{"a":[1,true]}'
50
54
 
51
55
  That's it—if you know the `json` gem, you already know `nosj`.
52
56
 
57
+ ### gem json compatibility
58
+
53
59
  Want the speedup without touching your code? One line reroutes
54
60
  `JSON.parse`, `JSON.generate`, `JSON.pretty_generate`, and `JSON.dump`
55
61
  through nosj:
@@ -65,18 +71,39 @@ Gemfile you can do this:
65
71
  gem "nosj", require: "nosj/json"
66
72
  ```
67
73
 
68
- Options nosj supports take the fast path; anything exotic
69
- (`create_additions`, `object_class`, `JSON::State`, procs, IO
70
- arguments) falls back to the original implementation, so `JSON.load`,
71
- `JSON.parse!`, and `JSON.load_file` keep their exact behavior.
72
- Exceptions re-raise as the JSON classes, so your rescue clauses keep
73
- working. Measured through the patch: parse 1.11×, generate 1.05× over
74
- the original gem. (A MultiJson adapter ships too:
75
- `require "nosj/multi_json"`, then `MultiJson.use NOSJ::MultiJsonAdapter`.)
74
+ A MultiJson adapter ships too:
75
+
76
+ ```ruby
77
+ require "nosj/multi_json"
78
+ ```
79
+
80
+ And then `MultiJson.use NOSJ::MultiJsonAdapter`.
81
+
82
+ ### Ruby on Rails
83
+
84
+ In a Rails app, use this for "Rails mode":
85
+
86
+ ```ruby
87
+ gem "nosj", require: "nosj/rails"
88
+ ```
89
+
90
+ That installs a nosj-backed ActiveSupport JSON encoder, so `obj.to_json`, `render json:`, and `ActiveSupport::JSON.encode` walk the object tree natively:
91
+ values that aren't JSON-native recurse through `as_json` exactly like
92
+ ActiveSupport's own encoder, non-finite floats encode as `null`, and
93
+ HTML-safety escaping (`escape_html_entities_in_json`) behaves
94
+ identically—verified differentially against ActiveSupport's encoder.
95
+ It loads the drop-in too, so `ActiveSupport::JSON.decode` and JSON
96
+ request-body parsing ride the fast path.
97
+
98
+ Measured against ActiveSupport's own encoder: ×1.9 on small documents
99
+ up to ×5.2 on large trees and ×14 on HTML-heavy content—see
100
+ [Benchmarks → Rails mode](#rails-mode).
76
101
 
77
102
  ## What's in the box
78
103
 
79
- **The `json` gem API**, on the `NOSJ` module:
104
+ ### json API
105
+
106
+ The `json` gem API, on the `NOSJ` module:
80
107
 
81
108
  ```ruby
82
109
  NOSJ.parse(src, symbolize_names: true) # also: freeze, max_nesting,
@@ -85,7 +112,9 @@ NOSJ.generate(obj) # indent, space, object_nl, ...,
85
112
  NOSJ.pretty_generate(obj) # ascii_only, script_safe, strict
86
113
  ```
87
114
 
88
- **Lazy documents.** `NOSJ.lazy` wraps a document in a lazy view: read
115
+ ### Lazy documents
116
+
117
+ `NOSJ.lazy` wraps a document in a lazy view: read
89
118
  a field and only that path is parsed—containers stay lazy, scalars
90
119
  arrive as plain Ruby values, and repeated reads are cached:
91
120
 
@@ -101,8 +130,9 @@ Pass a frozen string and creating the view is practically free—
101
130
  nanoseconds, even on a megabyte document. Malformed content raises
102
131
  when it is first read, not at wrap time.
103
132
 
133
+ ### Partial parsing
104
134
 
105
- **Partial parsing.** Pull values out of a document without
135
+ Pull values out of a document without
106
136
  materializing the rest—skipped content is stepped over at SIMD block
107
137
  speed, so a lookup costs what it skips, not what the document weighs:
108
138
 
@@ -122,7 +152,9 @@ at the far end of a 570 KB document costs ~71µs, still 13× faster
122
152
  than parse-then-dig. Misses return nil; matched subtrees materialize
123
153
  with the same options as `parse` (`symbolize_names:`, `freeze:`).
124
154
 
125
- **Files.** Every mode has a file-native form, so a document never
155
+ ### Files API
156
+
157
+ Every mode has a file-native form, so a document never
126
158
  round-trips through a throwaway Ruby String:
127
159
 
128
160
  ```ruby
@@ -138,10 +170,89 @@ read are never loaded from disk. Missing files raise the usual
138
170
  `Errno` exceptions. Measured numbers live in
139
171
  [Benchmarks → File APIs](#file-apis).
140
172
 
141
- **Validation without parsing.** `NOSJ.valid?` runs the full
142
- parser—tokenizers, string decode, number validation—into a null sink
143
- and allocates no Ruby objects at all. It is 2-4× faster than
144
- `NOSJ.parse`, which already leads every parser above:
173
+ ### Reformat without parsing
174
+
175
+ `NOSJ.minify` and `NOSJ.reformat` pipe the parser's events straight
176
+ into the emission kernels: **zero Ruby objects** are built for the
177
+ document (the specs assert it), SIMD in and SIMD out. Minifying the
178
+ 631 KB twitter.json takes 409µs—3.4× faster than a
179
+ `generate(parse(x))` round-trip, 3.9× faster than gem json's cycle,
180
+ and only 1.4× the cost of `valid?`:
181
+
182
+ ```ruby
183
+ NOSJ.minify(json) # compact
184
+ NOSJ.reformat(json, pretty: true) # pretty_generate layout
185
+ NOSJ.reformat(json, ascii_only: true) # escape-transcode, no parse
186
+ NOSJ.reformat_file("big.json") # straight off a memory map
187
+ ```
188
+
189
+ Output is exactly `generate(parse(json))`—canonical number spellings,
190
+ normalized escapes, same formatting options—except duplicate keys pass
191
+ through (a reformatter must not silently drop data) and lone-surrogate
192
+ strings re-escape instead of raising. Acceptance options apply too:
193
+ `minify(src, allow_trailing_comma: true)` normalizes the commas away.
194
+
195
+ ### Byte-splicing edits and JSON Patch
196
+
197
+ Change a field in a passing payload without the parse → mutate →
198
+ generate cycle: `NOSJ.splice` skips to the target spans (one pass for
199
+ the whole batch) and rebuilds the string around them—every byte
200
+ outside the targets is copied untouched, so formatting, key order,
201
+ and number spellings survive exactly. On twitter.json this is
202
+ **10×–51× faster** than parse-mutate-generate (129µs for a late
203
+ field, 26µs for an early one, vs ~1.3ms):
204
+
205
+ ```ruby
206
+ NOSJ.splice(json, "/config/timeout" => 30)
207
+ NOSJ.splice(json, "/a" => 1, "/b/c" => [true]) # batch, one pass
208
+ ```
209
+
210
+ On top of the same machinery: **RFC 6902 JSON Patch** applied to the
211
+ raw string (`add`, `remove`, `replace`, `move`, `copy`, `test`—the
212
+ whole appendix-A suite is in the specs), and **RFC 7386 JSON Merge
213
+ Patch**:
214
+
215
+ ```ruby
216
+ NOSJ.patch(json, [
217
+ {"op" => "test", "path" => "/a", "value" => 1},
218
+ {"op" => "replace", "path" => "/a", "value" => 2},
219
+ {"op" => "add", "path" => "/list/-", "value" => "x"}
220
+ ])
221
+ NOSJ.merge_patch(json, {"config" => {"legacy" => nil, "timeout" => 30}})
222
+ ```
223
+
224
+ Missing splice targets raise `KeyError` (use `patch` `add` to
225
+ insert); failed patches raise `NOSJ::PatchError`; inserted values are
226
+ byte-identical to `NOSJ.generate`'s output.
227
+
228
+ ### NDJSON / JSON Lines
229
+
230
+ Logs, data pipelines, LLM batch files—one JSON document per line.
231
+ `NOSJ.each_line` streams values out (1.6× faster than the
232
+ line-split-and-`JSON.parse` idiom on twitter statuses), and
233
+ `NOSJ.generate_lines` builds the whole stream in one buffer pass
234
+ (4.1× faster than map-generate-join):
235
+
236
+ ```ruby
237
+ NOSJ.each_line(log) { |event| ingest(event) } # or an Enumerator:
238
+ NOSJ.each_line(log).first(10) # walks only 10 lines
239
+
240
+ NOSJ.generate_lines(events) #=> %({"a":1}\n{"b":2}\n...)
241
+ NOSJ.each_line_file("events.ndjson") { |e| } # over a memory map
242
+ NOSJ.write_lines("out.ndjson", events) # straight to disk
243
+ ```
244
+
245
+ Blank lines are skipped, one value per line is enforced, and a
246
+ malformed line raises a [rich `ParserError`](#rich-parse-errors) whose
247
+ `#line` is the physical line in the stream. Parse options apply per
248
+ line—`NOSJ.each_line(log, freeze: true)` yields Ractor-shareable
249
+ values.
250
+
251
+ ### Validation without parsing
252
+
253
+ `NOSJ.valid?` runs the full parser—tokenizers, string decode,
254
+ number validation—into a null sink and allocates no Ruby objects
255
+ at all. It is 2-4× faster than `NOSJ.parse`:
145
256
 
146
257
  ```ruby
147
258
  NOSJ.valid?('{"a":1}') #=> true
@@ -149,6 +260,46 @@ NOSJ.valid?('{"a":}') #=> false
149
260
  NOSJ.valid?(src, max_nesting: false) # same options as parse
150
261
  ```
151
262
 
263
+ ### Document statistics
264
+
265
+ `NOSJ.stats` answers "what is this 40 MB blob": one counting pass
266
+ through the same null-sink machinery—no Ruby value is built for
267
+ the document, and it costs _less_ than a parse (~1.3× faster on `twitter.json`):
268
+
269
+ ```ruby
270
+ s = NOSJ.stats(blob) # or NOSJ.stats_file("huge.json")
271
+ s[:byte_size] #=> 631514
272
+ s[:root] #=> :object
273
+ s[:max_depth] #=> 10
274
+ s[:values] #=> {total: 13914, objects: 1264, arrays: 1050,
275
+ # strings: 4754, integers: 2108, ...}
276
+ s[:keys] #=> {total: 13345, unique: 94}
277
+ s[:key_histogram].first(3) #=> [["id", 447], ["id_str", 447], ["urls", 364]]
278
+ s[:containers] #=> {max_object_entries: 40, max_array_length: 100}
279
+ s[:strings] #=> {bytes: 200716, max_bytes: 463}
280
+ ```
281
+
282
+ Nesting is unlimited by default (a deep blob is exactly what a
283
+ diagnostic should describe); malformed documents raise the usual rich
284
+ `ParserError`.
285
+
286
+ ### Rich parse errors
287
+
288
+ A failed parse raises `NOSJ::ParserError`
289
+ carrying where the document broke—`#byte_offset`, `#line`, `#column`,
290
+ and a caret `#snippet`—computed only when a parse fails, so success
291
+ pays nothing. Positions stay absolute through partial parsing, lazy
292
+ documents, and the file APIs; unrescued errors print the snippet:
293
+
294
+ ```ruby
295
+ NOSJ.parse(%({\n "a": 1,\n "b": }))
296
+ # NOSJ::ParserError: unexpected character at byte 19
297
+ # e.line #=> 3
298
+ # e.column #=> 8
299
+ # e.snippet #=> "b": }
300
+ # ^
301
+ ```
302
+
152
303
  ## Benchmarks
153
304
 
154
305
  Every installed JSON gem, benchmark-ips: AWS EC2 c7a.2xlarge (AMD EPYC 9R14, Zen 4), Ruby 4.0.6 + YJIT, json 2.21.1, Oj 3.17.4, RapidJSON 0.4.0, FastJsonparser 0.6.0, Yajl 1.4.3, PGO build, 2026-07-16. `×N` = times slower than nosj.
@@ -209,6 +360,27 @@ Silicon dev box, Ruby 4.0.6 + YJIT, PGO build, 2026-07-16):
209
360
  too much for an honest multiplier; what it saves is the intermediate
210
361
  file-sized Ruby String.
211
362
 
363
+ ### Rails mode
364
+
365
+ `ActiveSupport::JSON.encode` with the nosj encoder installed, against
366
+ stock ActiveSupport (Apple Silicon dev box, Ruby 4.0.6 + YJIT,
367
+ activesupport 8.1, medians of 5 interleaved per-process rounds,
368
+ outputs verified byte-identical first, 2026-07-17;
369
+ `rake bench:rails`):
370
+
371
+ | workload | nosj (i/s) | vs ActiveSupport |
372
+ |---|---:|---|
373
+ | twitter tree (570 KB) | 4.5k | ×5.2 |
374
+ | 100-record index (with timestamps) | 33.7k | ×1.7 |
375
+ | HTML-heavy user content | 177.7k | ×14.2 |
376
+ | Time/Date/BigDecimal hash | 1.1M | ×3.0 |
377
+ | small API hash | 4.2M | ×1.9 |
378
+ | small hash `to_json` | 4.1M | ×1.9 |
379
+
380
+ The HTML-safety escaping that dominates stock encodes of
381
+ user-generated content is fused into the SIMD string-emission kernels
382
+ here: escaped output costs the same single pass as unescaped.
383
+
212
384
  Reproduce with `rake bench` (the parity-gated comparison, after a PGO retrain—the shipping configuration) or `rake bench:ips` (the multi-gem shoot-out).
213
385
 
214
386
  ## Switching from the json gem
@@ -222,8 +394,10 @@ You mostly don't have to do anything. Some differences:
222
394
  UTF-8) follow the strict semantics instead.
223
395
  - Unlike `Array#dig`, negative indices in `NOSJ.dig` return nil (JSON
224
396
  Pointer has no equivalent).
225
- - Parse error *messages* use byte offsets rather than the gem's
226
- phrasing (classes match).
397
+ - Parse errors raise `NOSJ::ParserError` (`NOSJ::NestingError` past
398
+ `max_nesting`, like the gem); messages use byte offsets rather than
399
+ the gem's phrasing, and the exception carries `#line`, `#column`,
400
+ and a caret `#snippet`.
227
401
 
228
402
  Everything else—including the gem's exact float formatting, which is
229
403
  not the shortest-round-trip form most libraries emit—matches
@@ -260,6 +434,28 @@ bundle exec rake bench:fast # the sweep without retraining
260
434
  bundle exec rake "bench:ips[twitter]" # multi-gem shoot-out (benchmark-ips); no args = full corpus
261
435
  ```
262
436
 
437
+ ## Acknowledgements
438
+
439
+ Thanks to [Jean Boussier](https://github.com/byroot), [Florian Frank](https://github.com/flori), [Hiroshi Shibata](https://github.com/hsbt), [Nobuyoshi Nakada](https://github.com/nobu), [Étienne Barrié](https://github.com/etiennebarrie), and the other authors and maintainers of the [json gem](https://github.com/ruby/json)—for their work on the gem itself, for optimization ideas, and for some of the JSON documents in the benchmark corpus.
440
+
441
+ Thanks to [Mat Sadler](https://github.com/matsadler) for [magnus](https://github.com/matsadler/magnus).
442
+
443
+ Thanks to the authors of the projects credited in [NOTICE](NOTICE) and the
444
+ crate's NOTICE: [fpconv](https://github.com/night-shift/fpconv) and
445
+ Florian Loitsch's Grisu2,
446
+ [fast_float](https://github.com/fastfloat/fast_float) and
447
+ [fast_double_parser](https://github.com/lemire/fast_double_parser)
448
+ (Daniel Lemire et al.),
449
+ [simdjson](https://github.com/simdjson/simdjson) (Daniel Lemire, Geoff
450
+ Langdale, and contributors), and the architectural lineage of
451
+ [yyjson](https://github.com/ibireme/yyjson),
452
+ [simd-json](https://github.com/simd-lite/simd-json), and
453
+ [sonic-rs](https://github.com/cloudwego/sonic-rs).
454
+
455
+ ## Assisted by
456
+
457
+ Claude Fable 5, Claude Opus 4.8.
458
+
263
459
  ## License
264
460
 
265
461
  MIT. The underlying Rust crate is `MIT AND BSL-1.0 AND Apache-2.0`; its
data/lib/nosj/3.3/nosj.so CHANGED
Binary file
data/lib/nosj/3.4/nosj.so CHANGED
Binary file
data/lib/nosj/4.0/nosj.so CHANGED
Binary file
data/lib/nosj/json.rb CHANGED
@@ -6,7 +6,7 @@
6
6
  #
7
7
  # reroutes JSON.parse, JSON.generate, JSON.pretty_generate and JSON.dump
8
8
  # through NOSJ whenever the requested options fall within NOSJ's
9
- # supported set, and falls back to the original json implementation for
9
+ # supported set, and falls back to gem json's own implementation for
10
10
  # everything else (create_additions, object_class/array_class,
11
11
  # decimal_class, on_load procs, JSON::State instances, IO arguments).
12
12
  # Entry points built on JSON.parse (JSON.load, JSON.parse!,
@@ -30,8 +30,11 @@ module NOSJ
30
30
  # Implementation detail of `require "nosj/json"`.
31
31
  # @private
32
32
  module JSONDropIn
33
+ # quirks_mode rides the fast path because NOSJ.parse is always
34
+ # quirks-mode (top-level scalars parse) and ignores the key; Rails
35
+ # 7.x passes it from ActiveSupport::JSON.decode.
33
36
  PARSE_OPTS = %i[symbolize_names freeze max_nesting allow_nan
34
- allow_trailing_comma].freeze
37
+ allow_trailing_comma quirks_mode].freeze
35
38
  GENERATE_OPTS = %i[indent space space_before object_nl array_nl
36
39
  max_nesting allow_nan ascii_only script_safe
37
40
  escape_slash strict depth
@@ -41,7 +44,7 @@ module NOSJ
41
44
 
42
45
  # The fast path handles nil or a plain Hash whose every key NOSJ
43
46
  # implements; anything else (JSON::State, exotic options, string
44
- # keys) belongs to the original implementation.
47
+ # keys) belongs to gem json.
45
48
  def supported?(opts, allowed)
46
49
  return true if opts.nil?
47
50
  return false unless opts.instance_of?(Hash)
@@ -50,8 +53,29 @@ module NOSJ
50
53
  end
51
54
 
52
55
  def parse(source, opts)
56
+ # NOSJ.parse is deliberately strict about encodings (json-3.0
57
+ # semantics), but the drop-in must match the installed gem, which
58
+ # accepts more. BINARY strings holding valid UTF-8 are the big
59
+ # real-world case: Rack delivers request bodies as BINARY, so
60
+ # Rails JSON params come through here. Retagging a dup is cheap
61
+ # (copy-on-write bytes), and the validity scan is memoized
62
+ # coderange the parse would compute anyway. Anything else
63
+ # non-UTF-8 (UTF-16, ...) belongs to gem json, which transcodes.
64
+ if source.is_a?(String)
65
+ case source.encoding
66
+ when Encoding::UTF_8, Encoding::US_ASCII
67
+ # the fast path as-is
68
+ when Encoding::BINARY
69
+ utf8 = source.dup.force_encoding(Encoding::UTF_8)
70
+ source = utf8 if utf8.valid_encoding?
71
+ else
72
+ return ::JSON.nosj_original_parse(source, **(opts || {}))
73
+ end
74
+ end
53
75
  NOSJ.parse(source, opts)
54
- rescue RuntimeError => e
76
+ rescue NOSJ::NestingError => e
77
+ raise ::JSON::NestingError, e.message
78
+ rescue NOSJ::ParserError => e
55
79
  raise ::JSON::ParserError, e.message
56
80
  end
57
81
 
@@ -103,9 +127,9 @@ module JSON
103
127
 
104
128
  def dump(obj, an_io = nil, limit = nil, kwargs = nil)
105
129
  # Fast path for the common shapes, dump(obj) and dump(obj, opts
106
- # hash), mirroring the gem: dump defaults merged under the
130
+ # hash), mirroring gem json: dump defaults merged under the
107
131
  # user's options, NestingError surfaced as ArgumentError. IO and
108
- # limit arguments take the original implementation.
132
+ # limit arguments take gem json's own dump.
109
133
  if limit.nil? && kwargs.nil? && (an_io.nil? || an_io.instance_of?(Hash))
110
134
  opts = _dump_default_options
111
135
  opts = opts.merge(an_io) if an_io
@@ -36,7 +36,7 @@ module NOSJ
36
36
  # @raise [JSON::ParserError] when the document is malformed
37
37
  def load(string, options = {})
38
38
  ::NOSJ.parse(string, options[:symbolize_names] ? SYMBOLIZE : nil)
39
- rescue RuntimeError => e
39
+ rescue ::NOSJ::ParserError, ::NOSJ::NestingError => e
40
40
  raise ParseError, e.message
41
41
  end
42
42
 
data/lib/nosj/rails.rb ADDED
@@ -0,0 +1,76 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Rails mode: `require "nosj/rails"` accelerates a Rails application in
4
+ # both directions.
5
+ #
6
+ # - It installs {NOSJ::RailsEncoder} as ActiveSupport's JSON encoder
7
+ # (the official +ActiveSupport::JSON::Encoding.json_encoder+ seam, the
8
+ # same one Oj's Rails mode uses). That captures every encode a Rails
9
+ # app performs through +obj.to_json+, +render json:+, and
10
+ # +ActiveSupport::JSON.encode+: the object tree is walked natively,
11
+ # values that are not JSON-native recurse through +as_json+ exactly
12
+ # like ActiveSupport's own encoder, and non-finite floats encode as
13
+ # +null+ (+Float#as_json+ parity).
14
+ # - It loads the `nosj/json` drop-in, so +JSON.parse+ (and with it
15
+ # +ActiveSupport::JSON.decode+ and JSON request-body parsing) takes
16
+ # the fast path.
17
+ #
18
+ # In a Rails Gemfile:
19
+ #
20
+ # gem "nosj", require: "nosj/rails"
21
+ #
22
+ # +JSON::Fragment+ values splice their pre-rendered JSON, like current
23
+ # ActiveSupport. On older ActiveSupport (before its encoder learned
24
+ # about fragments) stock encoding dumps the fragment's instance
25
+ # variables instead; there this encoder deliberately diverges in favor
26
+ # of real splicing.
27
+
28
+ require "nosj/json"
29
+ require "active_support"
30
+ require "active_support/json"
31
+
32
+ module NOSJ
33
+ # ActiveSupport JSON encoder backed by nosj: the interface of
34
+ # ActiveSupport's +JSONGemEncoder+ (accepts the options hash, encodes
35
+ # one value), with the tree walk, generation, AND the HTML/JS-safety
36
+ # escape pass running natively (a byte scan instead of ActiveSupport's
37
+ # Ruby regex post-pass), honoring +escape_html_entities_in_json+ (and,
38
+ # where present, +escape_js_separators_in_json+ and the per-call
39
+ # +escape:+ / +escape_html_entities:+ options).
40
+ class RailsEncoder
41
+ # The escape_js_separators_in_json knob is newer than the Rails
42
+ # versions we support; its presence is fixed at load time (only its
43
+ # value can change at runtime). Absent, ActiveSupport always
44
+ # escapes the JS separators.
45
+ HAS_JS_SEPARATORS_KNOB =
46
+ ActiveSupport::JSON::Encoding.respond_to?(:escape_js_separators_in_json)
47
+ private_constant :HAS_JS_SEPARATORS_KNOB
48
+
49
+ attr_reader :options
50
+
51
+ def initialize(options = nil)
52
+ @options = options || {}
53
+ end
54
+
55
+ def encode(value)
56
+ value = value.as_json(@options.dup) unless @options.empty?
57
+ if @options.fetch(:escape, true)
58
+ escape_html = @options.fetch(:escape_html_entities) do
59
+ ActiveSupport::JSON::Encoding.escape_html_entities_in_json
60
+ end
61
+ NOSJ.generate_rails_native(value, !!escape_html, escape_js_separators?)
62
+ else
63
+ NOSJ.generate_rails_native(value, false, false)
64
+ end
65
+ end
66
+
67
+ private
68
+
69
+ def escape_js_separators?
70
+ !HAS_JS_SEPARATORS_KNOB ||
71
+ ActiveSupport::JSON::Encoding.escape_js_separators_in_json
72
+ end
73
+ end
74
+
75
+ ActiveSupport::JSON::Encoding.json_encoder = RailsEncoder
76
+ end
data/lib/nosj/version.rb CHANGED
@@ -2,5 +2,5 @@
2
2
 
3
3
  module NOSJ
4
4
  # The gem version.
5
- VERSION = "0.2.0"
5
+ VERSION = "0.3.0"
6
6
  end
data/lib/nosj.rb CHANGED
@@ -26,15 +26,58 @@ module NOSJ
26
26
  # Base class for nosj errors.
27
27
  class Error < StandardError; end
28
28
 
29
+ # Raised when a document cannot be parsed. Carries the failure
30
+ # position, computed once when the parse fails (successful parses
31
+ # never pay for it): {#byte_offset}, 1-based {#line}, character-based
32
+ # {#column}, and a caret {#snippet} pointing at the offending spot.
33
+ # Positions are absolute within the document you passed, including
34
+ # through partial parsing ({NOSJ.dig}, {NOSJ.at_pointer}, lazy
35
+ # documents) and the file APIs. All four are +nil+ for failures
36
+ # without a position (encoding refusals).
37
+ #
38
+ # @example
39
+ # NOSJ.parse(%({\n "a": 1,\n "b": }))
40
+ # # => NOSJ::ParserError, with
41
+ # # e.line #=> 3
42
+ # # e.column #=> 8
43
+ # # e.snippet #=> " \"b\": }\n ^"
44
+ class ParserError < Error
45
+ # @return [Integer, nil] byte offset of the failure in the source
46
+ attr_reader :byte_offset
47
+ # @return [Integer, nil] 1-based line of the failure
48
+ attr_reader :line
49
+ # @return [Integer, nil] 1-based character (not byte) column within
50
+ # {#line}
51
+ attr_reader :column
52
+ # @return [String, nil] the offending line (windowed when long)
53
+ # with a caret line underneath
54
+ attr_reader :snippet
55
+
56
+ # The default message plus {#snippet}: Ruby prints
57
+ # +detailed_message+ when an exception reaches the top level, so an
58
+ # unrescued parse error shows where the document broke.
59
+ def detailed_message(highlight: false, **opts)
60
+ base = super
61
+ snippet ? "#{base}\n#{snippet}" : base
62
+ end
63
+ end
64
+
29
65
  # Raised when a value cannot be generated (non-finite floats without
30
66
  # +allow_nan+, unsupported objects under +strict+, broken encodings).
31
67
  # Message-compatible with +JSON::GeneratorError+.
32
68
  class GeneratorError < Error; end
33
69
 
34
- # Raised when generation exceeds +max_nesting+. Message-compatible
35
- # with +JSON::NestingError+.
70
+ # Raised when parsing or generation exceeds +max_nesting+.
71
+ # Message-compatible with +JSON::NestingError+.
36
72
  class NestingError < Error; end
37
73
 
74
+ # Raised when an RFC 6902 patch cannot be applied: a +test+ operation
75
+ # failed, a target or source path does not exist, an array index is
76
+ # out of range, or a +move+ targets its own child. Structurally
77
+ # malformed patch documents (not an op Hash, unknown op, missing
78
+ # fields) raise ArgumentError instead.
79
+ class PatchError < Error; end
80
+
38
81
  PRETTY_GENERATE_OPTS = {
39
82
  indent: " ", space: " ", object_nl: "\n", array_nl: "\n"
40
83
  }.freeze
@@ -57,7 +100,9 @@ module NOSJ
57
100
  # (Integer or +false+ for unlimited), +allow_nan+,
58
101
  # +allow_trailing_comma+
59
102
  # @return [Object] the parsed value tree
60
- # @raise [RuntimeError] when the document is malformed or not UTF-8
103
+ # @raise [ParserError] when the document is malformed or not UTF-8;
104
+ # carries the failure position ({ParserError#line} and friends)
105
+ # @raise [NestingError] when nesting exceeds +max_nesting+
61
106
  # @raise [ArgumentError] for unsupported options
62
107
  def self.parse(source, opts = nil)
63
108
  parse_native(source, opts)
@@ -204,7 +249,7 @@ module NOSJ
204
249
  # @param opts [Hash, nil] the same options as {.parse}
205
250
  # @return [Object] the parsed value tree
206
251
  # @raise [SystemCallError] +Errno::ENOENT+ and friends, like File.read
207
- # @raise [RuntimeError] when the document is malformed or not UTF-8
252
+ # @raise [ParserError] when the document is malformed or not UTF-8
208
253
  def self.load_file(path, opts = nil)
209
254
  load_file_native(path, opts)
210
255
  end
@@ -240,7 +285,7 @@ module NOSJ
240
285
  # @param opts [Hash, nil] {.parse} options applied on materialization
241
286
  # @return [NOSJ::Lazy, Object]
242
287
  # @raise [SystemCallError] +Errno::ENOENT+ and friends
243
- # @raise [RuntimeError] when the file is not UTF-8 or the root is malformed
288
+ # @raise [ParserError] when the file is not UTF-8 or the root is malformed
244
289
  def self.load_lazy_file(path, opts = nil)
245
290
  load_lazy_file_native(path, opts)
246
291
  end
@@ -273,4 +318,305 @@ module NOSJ
273
318
  def self.dig_file(path, *path_elements)
274
319
  dig_file_native(path, path_elements)
275
320
  end
321
+
322
+ # Minifies a document without building any Ruby values: the parser's
323
+ # events pipe straight into the emission kernels, SIMD in and SIMD
324
+ # out. Output is exactly what <code>generate(parse(json))</code>
325
+ # would produce, except duplicate object keys pass through instead of
326
+ # being collapsed (a reformatter must not silently drop data).
327
+ # Numbers come out in the canonical spelling (+1.50+ becomes +1.5+)
328
+ # and string escapes are normalized.
329
+ #
330
+ # @example
331
+ # NOSJ.minify(%({ "a": [1, 2],\n "b": "x" })) #=> '{"a":[1,2],"b":"x"}'
332
+ #
333
+ # @param json [String] the document (UTF-8 or US-ASCII)
334
+ # @param opts [Hash, nil] acceptance options (+allow_nan+,
335
+ # +allow_trailing_comma+, +max_nesting+); trailing commas are
336
+ # normalized away when accepted
337
+ # @return [String] the minified document
338
+ # @raise [ParserError] when the document is malformed
339
+ # @raise [NestingError] past +max_nesting+
340
+ def self.minify(json, opts = nil)
341
+ reformat_native(json, opts)
342
+ end
343
+
344
+ # Reformats a document without building any Ruby values: {.minify}'s
345
+ # pipe with formatting. <code>pretty: true</code> is a shorthand for
346
+ # {.pretty_generate}'s layout; the individual {.generate} formatting
347
+ # and escape options (+indent+, +space+, +object_nl+, +ascii_only+,
348
+ # +script_safe+, ...) compose with it and win over it.
349
+ #
350
+ # @example
351
+ # NOSJ.reformat(json, pretty: true)
352
+ # NOSJ.reformat(json, ascii_only: true) # escape-transcode, compact
353
+ #
354
+ # @param json [String] the document (UTF-8 or US-ASCII)
355
+ # @param opts [Hash, nil] +pretty+, {.generate} formatting/escape
356
+ # options, and {.minify}'s acceptance options
357
+ # @return [String] the reformatted document
358
+ # @raise [ParserError] when the document is malformed
359
+ # @raise [NestingError] past +max_nesting+
360
+ # @raise [GeneratorError] when +ascii_only+ meets a lone-surrogate
361
+ # string it cannot represent
362
+ def self.reformat(json, opts = nil)
363
+ if opts&.key?(:pretty)
364
+ pretty = opts[:pretty]
365
+ opts = opts.except(:pretty)
366
+ opts = PRETTY_GENERATE_OPTS.merge(opts) if pretty
367
+ end
368
+ reformat_native(json, opts)
369
+ end
370
+
371
+ # {.reformat} against a file: the pipe runs over a read-only memory
372
+ # map, so the input document never becomes a Ruby String; only the
373
+ # result does.
374
+ #
375
+ # @example
376
+ # compact = NOSJ.reformat_file("big.json") # minify
377
+ # pretty = NOSJ.reformat_file("big.json", pretty: true)
378
+ #
379
+ # @param path [String] the file to reformat (UTF-8)
380
+ # @param opts [Hash, nil] same options as {.reformat}
381
+ # @return [String] the reformatted document
382
+ # @raise [SystemCallError] +Errno::ENOENT+ and friends
383
+ # @raise [ParserError] when the file is malformed or not UTF-8
384
+ def self.reformat_file(path, opts = nil)
385
+ if opts&.key?(:pretty)
386
+ pretty = opts[:pretty]
387
+ opts = opts.except(:pretty)
388
+ opts = PRETTY_GENERATE_OPTS.merge(opts) if pretty
389
+ end
390
+ reformat_file_native(path, opts)
391
+ end
392
+
393
+ # Byte-splicing edits: replaces the values at the given JSON Pointers
394
+ # directly in the text. Every target resolves in ONE forward pass
395
+ # (skipping, not parsing), and the result is built in one sweep:
396
+ # every byte outside the target spans is copied untouched, so
397
+ # formatting, key order, and number spellings elsewhere are preserved
398
+ # exactly. For tweaking a field in a passing payload this replaces
399
+ # the whole parse → mutate → generate cycle.
400
+ #
401
+ # @example
402
+ # NOSJ.splice(json, "/config/timeout" => 30)
403
+ # NOSJ.splice(json, "/a" => 1, "/b/c" => [true]) # batch, one pass
404
+ #
405
+ # @param json [String] the document (UTF-8 or US-ASCII)
406
+ # @param edits [Hash{String => Object}] JSON Pointer => replacement
407
+ # value (generated compactly, byte-identical to {.generate})
408
+ # @param opts [Hash, nil] {.generate} options for the inserted values
409
+ # @return [String] the edited document
410
+ # @raise [KeyError] when a pointer does not resolve (splice replaces;
411
+ # use {.patch} +add+ to insert)
412
+ # @raise [ArgumentError] for malformed pointers or overlapping targets
413
+ # @raise [ParserError] when the document is malformed
414
+ def self.splice(json, edits, opts = nil)
415
+ splice_native(json, edits, opts)
416
+ end
417
+
418
+ # Applies an RFC 6902 JSON Patch to the raw document: +add+,
419
+ # +remove+, +replace+, +move+, +copy+, and +test+, applied
420
+ # sequentially, each as a byte-splice (structural ops walk only the
421
+ # parent container's span). Op hashes accept String or Symbol keys.
422
+ #
423
+ # @example
424
+ # NOSJ.patch(json, [
425
+ # {"op" => "test", "path" => "/a", "value" => 1},
426
+ # {"op" => "replace", "path" => "/a", "value" => 2},
427
+ # {"op" => "add", "path" => "/list/-", "value" => "x"},
428
+ # {"op" => "move", "from" => "/tmp", "path" => "/kept"}
429
+ # ])
430
+ #
431
+ # @param json [String] the document (UTF-8 or US-ASCII)
432
+ # @param ops [Array<Hash>] RFC 6902 operations
433
+ # @param opts [Hash, nil] {.generate} options for inserted values
434
+ # @return [String] the patched document
435
+ # @raise [PatchError] when application fails (failed +test+, missing
436
+ # target, index out of range, move into own child)
437
+ # @raise [ArgumentError] for structurally malformed patch documents
438
+ # @raise [ParserError] when the document is malformed
439
+ def self.patch(json, ops, opts = nil)
440
+ patch_native(json, ops, opts)
441
+ end
442
+
443
+ # Applies an RFC 7386 JSON Merge Patch: +nil+ values remove keys,
444
+ # nested Hashes merge recursively, everything else replaces. This is
445
+ # the semantic form (parse, merge, generate); Symbol keys in +patch+
446
+ # match String keys in the document.
447
+ #
448
+ # @example
449
+ # NOSJ.merge_patch(%({"a":{"b":1,"c":2}}), {a: {b: nil, d: 3}})
450
+ # #=> '{"a":{"c":2,"d":3}}'
451
+ #
452
+ # @param json [String] the document (UTF-8 or US-ASCII)
453
+ # @param patch [Object] the merge patch (a non-Hash replaces the
454
+ # whole document)
455
+ # @param opts [Hash, nil] {.generate} options for the result
456
+ # @return [String] the merged document
457
+ # @raise [ParserError] when the document is malformed
458
+ def self.merge_patch(json, patch, opts = nil)
459
+ return generate(patch, opts) unless patch.is_a?(Hash)
460
+ generate(merge_patch_value(parse(json), patch), opts)
461
+ end
462
+
463
+ # RFC 7386, applied to parsed values.
464
+ def self.merge_patch_value(target, patch)
465
+ return patch unless patch.is_a?(Hash)
466
+ target = {} unless target.is_a?(Hash)
467
+ out = target.dup
468
+ patch.each do |key, value|
469
+ key = key.to_s
470
+ if value.nil?
471
+ out.delete(key)
472
+ else
473
+ out[key] = merge_patch_value(out[key], value)
474
+ end
475
+ end
476
+ out
477
+ end
478
+ private_class_method :merge_patch_value
479
+
480
+ # NDJSON / JSON Lines: yields one parsed value per line of +source+.
481
+ # Framing is exact because a raw newline can never occur inside a
482
+ # JSON value; blank lines are skipped (the NDJSON convention). One
483
+ # value per line is enforced: a second value on a line raises, and a
484
+ # malformed line raises {ParserError} whose {ParserError#line} is the
485
+ # physical line number in +source+.
486
+ #
487
+ # Pass a frozen string for zero-copy iteration (an unfrozen source is
488
+ # walked over a private copy, like {.lazy}). The block may itself
489
+ # call any NOSJ method.
490
+ #
491
+ # @example
492
+ # NOSJ.each_line(log) { |event| ingest(event) }
493
+ # NOSJ.each_line(log).first(10) # Enumerator when blockless
494
+ #
495
+ # @param source [String] newline-delimited JSON (UTF-8 or US-ASCII)
496
+ # @param opts [Hash, nil] the same options as {.parse}, applied per line
497
+ # @yieldparam value [Object] one parsed document per non-blank line
498
+ # @return [Enumerator] when no block is given, else +nil+
499
+ # @raise [ParserError] on the first malformed line
500
+ def self.each_line(source, opts = nil, &block)
501
+ return enum_for(:each_line, source, opts) unless block
502
+ each_line_native(source, opts, &block)
503
+ end
504
+
505
+ # {.each_line} against a file: the NDJSON stream is walked over a
506
+ # read-only memory map, so the file never becomes a Ruby String.
507
+ #
508
+ # @example
509
+ # NOSJ.each_line_file("events.ndjson") { |event| ingest(event) }
510
+ #
511
+ # @param path [String] the NDJSON file (UTF-8)
512
+ # @param opts [Hash, nil] the same options as {.parse}, applied per line
513
+ # @yieldparam value [Object] one parsed document per non-blank line
514
+ # @return [Enumerator] when no block is given, else +nil+
515
+ # @raise [SystemCallError] +Errno::ENOENT+ and friends
516
+ # @raise [ParserError] on the first malformed line
517
+ def self.each_line_file(path, opts = nil, &block)
518
+ return enum_for(:each_line_file, path, opts) unless block
519
+ each_line_file_native(path, opts, &block)
520
+ end
521
+
522
+ # Generates NDJSON / JSON Lines: one compact document per element,
523
+ # each terminated with a newline, built in a single pass into one
524
+ # buffer. Formatting options containing newlines raise ArgumentError
525
+ # (they would break the line framing); everything else from
526
+ # {.generate} applies.
527
+ #
528
+ # @example
529
+ # NOSJ.generate_lines([{a: 1}, {b: 2}]) #=> %({"a":1}\n{"b":2}\n)
530
+ #
531
+ # @param values [Array, Enumerable] one document per element
532
+ # @param opts [Hash, nil] the same options as {.generate}
533
+ # @return [String] the NDJSON document (empty when +values+ is empty)
534
+ # @raise [ArgumentError] for formatting options that contain newlines
535
+ # @raise [GeneratorError] (see {.generate})
536
+ def self.generate_lines(values, opts = nil)
537
+ generate_lines_native(lines_array(values), opts)
538
+ end
539
+
540
+ # {.generate_lines} straight to a file, streaming the generator's
541
+ # buffer to disk like {.write_file}.
542
+ #
543
+ # @example
544
+ # NOSJ.write_lines("out.ndjson", events) #=> bytes written
545
+ #
546
+ # @param path [String] the file to (over)write
547
+ # @param values [Array, Enumerable] one document per line
548
+ # @param opts [Hash, nil] the same options as {.generate}
549
+ # @return [Integer] the number of bytes written, like File.write
550
+ # @raise [SystemCallError] +Errno::ENOENT+ and friends
551
+ # @raise [ArgumentError] for formatting options that contain newlines
552
+ # @raise [GeneratorError] (see {.generate})
553
+ def self.write_lines(path, values, opts = nil)
554
+ write_lines_native(path, lines_array(values), opts)
555
+ end
556
+
557
+ # One document per element: Arrays pass through, other Enumerables
558
+ # convert, anything else (nil included, whose to_a would silently
559
+ # yield []) is a TypeError.
560
+ def self.lines_array(values)
561
+ return values if values.is_a?(Array)
562
+ unless values.is_a?(Enumerable)
563
+ raise TypeError, "no implicit conversion of #{values.class} into Array"
564
+ end
565
+ values.to_a
566
+ end
567
+ private_class_method :lines_array
568
+
569
+ # Document statistics from one full-parser pass into a counting sink:
570
+ # no Ruby value is built for the document itself, only the small
571
+ # result Hash. A debugging endpoint for "what is this 40 MB blob".
572
+ #
573
+ # The result (Symbol keys):
574
+ #
575
+ # {
576
+ # byte_size: 631, # of the document
577
+ # root: :object, # :object/:array/:string/:integer/
578
+ # # :float/:boolean/:null
579
+ # max_depth: 4, # container nesting, counted like
580
+ # # max_nesting (root container = 1)
581
+ # values: {total:, objects:, arrays:, strings:, integers:,
582
+ # floats:, booleans:, nulls:},
583
+ # keys: {total:, unique:},
584
+ # key_histogram: {"name" => 128, ...}, # sorted by count desc,
585
+ # # so .first(10) = top 10
586
+ # containers: {max_object_entries:, max_array_length:},
587
+ # strings: {bytes:, max_bytes:} # decoded UTF-8 bytes
588
+ # }
589
+ #
590
+ # Unlike {.parse}, nesting is UNLIMITED by default (a deep blob is
591
+ # exactly what a diagnostic should describe); pass +max_nesting+ to
592
+ # enforce a limit. Histogram memory is proportional to the number of
593
+ # unique keys.
594
+ #
595
+ # @example Top ten keys of a mystery blob
596
+ # NOSJ.stats(blob)[:key_histogram].first(10)
597
+ #
598
+ # @param source [String] the JSON document (UTF-8 or US-ASCII)
599
+ # @param opts [Hash, nil] +max_nesting+, +allow_nan+,
600
+ # +allow_trailing_comma+ (acceptance options only)
601
+ # @return [Hash] the statistics described above
602
+ # @raise [ParserError] when the document is malformed or not UTF-8
603
+ def self.stats(source, opts = nil)
604
+ stats_native(source, opts)
605
+ end
606
+
607
+ # {.stats} against a file: memory-maps it and runs the counting pass
608
+ # without reading the document into Ruby. +byte_size+ is the file
609
+ # size.
610
+ #
611
+ # @example
612
+ # NOSJ.stats_file("huge.json") => {byte_size: 41_943_040, ...}
613
+ #
614
+ # @param path [String] the file to inspect (UTF-8)
615
+ # @param opts [Hash, nil] same options as {.stats}
616
+ # @return [Hash] the statistics described on {.stats}
617
+ # @raise [SystemCallError] +Errno::ENOENT+ and friends
618
+ # @raise [ParserError] when the file is malformed or not UTF-8
619
+ def self.stats_file(path, opts = nil)
620
+ stats_file_native(path, opts)
621
+ end
276
622
  end
data/sig/nosj.rbs CHANGED
@@ -25,12 +25,32 @@ module NOSJ
25
25
  class Error < StandardError
26
26
  end
27
27
 
28
+ # Parse failures. Position accessors are nil for failures without
29
+ # one (encoding refusals); positions are byte-absolute within the
30
+ # document, lines and (character) columns 1-based.
31
+ class ParserError < Error
32
+ def byte_offset: () -> Integer?
33
+
34
+ def line: () -> Integer?
35
+
36
+ def column: () -> Integer?
37
+
38
+ def snippet: () -> String?
39
+
40
+ def detailed_message: (?highlight: bool, **untyped) -> String
41
+ end
42
+
28
43
  class GeneratorError < Error
29
44
  end
30
45
 
31
46
  class NestingError < Error
32
47
  end
33
48
 
49
+ # RFC 6902 application failures (failed test, missing target, index
50
+ # out of range, move into own child).
51
+ class PatchError < Error
52
+ end
53
+
34
54
  def self.parse: (String source, ?opts opts) -> value
35
55
 
36
56
  def self.generate: (untyped obj, ?opts opts) -> String
@@ -39,6 +59,12 @@ module NOSJ
39
59
 
40
60
  def self.valid?: (String source, ?opts opts) -> bool
41
61
 
62
+ # Document statistics (byte_size, root, max_depth, values, keys,
63
+ # key_histogram, containers, strings) from one counting-sink pass.
64
+ def self.stats: (String source, ?opts opts) -> Hash[Symbol, untyped]
65
+
66
+ def self.stats_file: (String path, ?opts opts) -> Hash[Symbol, untyped]
67
+
42
68
  def self.dig: (String source, *path_element path) -> value
43
69
 
44
70
  def self.dig_many: (String source, Array[Array[path_element]] paths, ?opts opts) -> Array[value]
@@ -49,6 +75,36 @@ module NOSJ
49
75
 
50
76
  def self.lazy: (String source, ?opts opts) -> (Lazy | value)
51
77
 
78
+ # Reformat without building values: parser events pipe straight into
79
+ # the emission kernels. minify is compact; reformat takes pretty: and
80
+ # the generate formatting/escape options.
81
+ def self.minify: (String json, ?opts opts) -> String
82
+
83
+ def self.reformat: (String json, ?opts opts) -> String
84
+
85
+ def self.reformat_file: (String path, ?opts opts) -> String
86
+
87
+ # Byte-splicing edits: JSON Pointer => replacement value, resolved
88
+ # in one pass; bytes outside the target spans are copied untouched.
89
+ def self.splice: (String json, Hash[String, untyped] edits, ?opts opts) -> String
90
+
91
+ # RFC 6902 JSON Patch over the raw document.
92
+ def self.patch: (String json, Array[Hash[untyped, untyped]] ops, ?opts opts) -> String
93
+
94
+ # RFC 7386 JSON Merge Patch (semantic: parse, merge, generate).
95
+ def self.merge_patch: (String json, untyped patch, ?opts opts) -> String
96
+
97
+ # NDJSON / JSON Lines: one parsed value per non-blank line.
98
+ def self.each_line: (String source, ?opts opts) { (value) -> void } -> nil
99
+ | (String source, ?opts opts) -> Enumerator[value, nil]
100
+
101
+ def self.each_line_file: (String path, ?opts opts) { (value) -> void } -> nil
102
+ | (String path, ?opts opts) -> Enumerator[value, nil]
103
+
104
+ def self.generate_lines: (_Each[untyped] values, ?opts opts) -> String
105
+
106
+ def self.write_lines: (String path, _Each[untyped] values, ?opts opts) -> Integer
107
+
52
108
  def self.load_file: (String path, ?opts opts) -> value
53
109
 
54
110
  def self.write_file: (String path, untyped obj, ?opts opts) -> Integer
@@ -101,6 +157,16 @@ module NOSJ
101
157
  alias to_s inspect
102
158
  end
103
159
 
160
+ # Defined by `require "nosj/rails"`, which also installs it as
161
+ # ActiveSupport::JSON::Encoding.json_encoder.
162
+ class RailsEncoder
163
+ attr_reader options: Hash[Symbol, untyped]
164
+
165
+ def initialize: (?Hash[Symbol, untyped]? options) -> void
166
+
167
+ def encode: (untyped value) -> String
168
+ end
169
+
104
170
  # Defined by `require "nosj/multi_json"`. The runtime superclass is
105
171
  # multi_json's Adapter (whose namespace differs across multi_json
106
172
  # versions), so it is not declared here.
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: nosj
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.0
4
+ version: 0.3.0
5
5
  platform: x86_64-linux
6
6
  authors:
7
7
  - Yaroslav Markin
@@ -14,7 +14,9 @@ description: 'gem nosj is an extremely fast, json-gem-compatible JSON parser and
14
14
  with per-platform PGO, partial parsing (JSON Pointer, single and batch), lazy documents
15
15
  that parse a value only when you touch it, file APIs that parse, generate, and query
16
16
  files natively (memory-mapped, so unread pages never leave the disk), allocation-free
17
- validation, and a one-line JSON module drop-in.'
17
+ validation, document statistics from one counting pass, parse errors that carry
18
+ line, column, and a caret snippet, a one-line JSON module drop-in, and a Rails mode
19
+ that plugs into ActiveSupport''s encoder seam.'
18
20
  email:
19
21
  - yaroslav@markin.net
20
22
  executables: []
@@ -33,6 +35,7 @@ files:
33
35
  - lib/nosj/lazy.rb
34
36
  - lib/nosj/multi_json.rb
35
37
  - lib/nosj/native.rb
38
+ - lib/nosj/rails.rb
36
39
  - lib/nosj/version.rb
37
40
  - sig/nosj.rbs
38
41
  homepage: https://github.com/yaroslav/nosj-ruby