nosj 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 78314f13c659fff6d0e146ac0b951564e66b7b10deeb439e317703754a0100ef
-  data.tar.gz: 993f8a38c4e04dcd68a6ec8ee2dc62c611d2a04c1bc0ff23fc31d67ad69dd585
+  metadata.gz: e602e52eed45bcb8699e1742827fabc72f8c3579ea04d47a4a4a10349542d201
+  data.tar.gz: a257dfe77c69f02330cb48be35dc08492eb6e09fc2c11cc177f88ca969f31ab0
 SHA512:
-  metadata.gz: 6b5f09e43ed85c2e345a35d0fd681900e205d4743161542d82ba5c3d1c80879c62bff3acf88f12dac5836334c67d933cc6241ead7dbe6a7a820d5f672bccd3d4
-  data.tar.gz: 0b48922585bb4863f9ed71fb9500a9a60b4c1f857baeaaf79136ed4c9e752aa71c9c23235e85d3bab38b1f0967b7b7212ed6d771be57d62d87fcd39e91666cd8
+  metadata.gz: 5c012acd8c48dde81e6c46ec45a2e8c18c4d4b790866f506610c878353ca16e98bc313a0725cc1cabd5a3765325f9e3930a436c1d8ef574513df5abf4345c798
+  data.tar.gz: e4557199e32432b96e9d9f499fca2b9bf4f35cae723640bc02461625bb41c05efd8d9b8423f8695b3b8fca448658bfdcc12c0cb3780ead253b212098fc31f2a4

data/CHANGELOG.md

@@ -1,3 +1,91 @@
+## [0.3.0] - 2026-07-17
+
+- Reformat without parsing. `NOSJ.minify(json, opts)` and
+  `NOSJ.reformat(json, opts)` (plus `NOSJ.reformat_file`) pipe the
+  parser's events straight into the emission kernels: zero Ruby
+  objects are allocated for the document, and output is exactly
+  `generate(parse(json))`—canonical numbers, normalized escapes, the
+  full set of `generate` formatting and escape options, with
+  `pretty: true` as a `pretty_generate` shorthand—except duplicate
+  object keys pass through and lone-surrogate string values re-escape
+  as `\uXXXX` instead of raising (the output must always reparse).
+  Acceptance options apply per `parse` (`allow_trailing_comma`
+  normalizes the commas away). Measured on the 631 KB twitter.json:
+  409µs, 3.4× faster than `NOSJ.generate(NOSJ.parse(x))`, 3.9× faster
+  than gem json's cycle, 5.3× faster than Oj's, and 1.4× the cost of
+  `NOSJ.valid?`.
+- Byte-splicing edits and JSON Patch. `NOSJ.splice(json, pointer =>
+  value, ...)` replaces values directly in the text: every target
+  resolves in one forward pass and the result is rebuilt copying all
+  bytes outside the target spans untouched (formatting, key order, and
+  number spellings elsewhere survive exactly). Measured on
+  twitter.json: 10× faster than parse-mutate-generate for a late
+  field, 51× for an early one. Missing targets raise KeyError,
+  overlapping targets ArgumentError. `NOSJ.patch(json, ops)` applies
+  RFC 6902 JSON Patch (add/remove/replace/move/copy/test, String or
+  Symbol op keys) to the raw string the same way, with structural ops
+  walking only the parent container's span; application failures raise
+  the new `NOSJ::PatchError`, malformed patch documents ArgumentError.
+  `NOSJ.merge_patch(json, patch)` applies RFC 7386 JSON Merge Patch
+  (semantic form). Inserted values are byte-identical to
+  `NOSJ.generate` and accept its options; the RFC 6902 appendix-A
+  suite and the full RFC 7386 test table are in the specs.
+- NDJSON / JSON Lines. `NOSJ.each_line(source, opts)` yields one
+  parsed value per line (Enumerator without a block, so
+  `.first(10)`/`.lazy` walk only what they consume), skipping blank
+  lines and enforcing one value per line; a malformed line raises the
+  rich `ParserError` whose `#line` is the physical line number in the
+  stream. `NOSJ.generate_lines(values, opts)` emits one compact
+  newline-terminated document per element in a single buffer pass
+  (measured 4.1× faster than the map-generate-join idiom on twitter
+  statuses; `each_line` is 1.6× faster than line-split +
+  `JSON.parse`), rejecting formatting options that would break line
+  framing. File forms: `NOSJ.each_line_file` streams over a read-only
+  memory map, `NOSJ.write_lines` generates straight to disk and
+  returns the byte count. Parse options apply per line; pass a frozen
+  string to `each_line` for zero-copy iteration.
+- `NOSJ.stats(source, opts)` / `NOSJ.stats_file(path, opts)`: document
+  statistics from one counting pass through the null-sink machinery,
+  answering "what is this 40 MB blob" without building any Ruby values
+  for the document (measured ~1.3× faster than a full parse). Reports
+  `byte_size`, `root` kind, `max_depth`, value counts by type, key
+  totals, a key histogram sorted by count, largest container sizes,
+  and string byte totals. Nesting is unlimited by default (pass
+  `max_nesting` to enforce a limit); `allow_nan` and
+  `allow_trailing_comma` are honored; malformed documents raise the
+  rich `ParserError`. The file form memory-maps, so the document never
+  enters Ruby at all.
+- Rich parse errors. Parse failures now raise `NOSJ::ParserError`
+  (previously a bare `RuntimeError`) carrying the failure position,
+  computed only when a parse fails: `#byte_offset`, 1-based `#line`,
+  character-based `#column`, and a caret `#snippet` showing the
+  offending line (windowed when the line is long, as minified JSON
+  usually is). Positions are absolute within the document you passed,
+  including through partial parsing (`dig`, `at_pointer`, batches),
+  lazy documents, and the file APIs. `#detailed_message` appends the
+  snippet, so unrescued errors print it. Failures with no position
+  (encoding refusals) leave the accessors nil. Exceeding `max_nesting`
+  during parsing now raises `NOSJ::NestingError`, matching the gem's
+  class (generation already did); rescues of the old `RuntimeError`
+  need updating to `NOSJ::ParserError`/`NOSJ::Error`.
+- Rails mode: `require "nosj/rails"` accelerates a Rails application
+  in both directions. It installs a nosj-backed ActiveSupport JSON
+  encoder, so `obj.to_json`, `render json:`, and `ActiveSupport::JSON.encode` walk the object tree natively—values recurse through `as_json` exactly
+  like ActiveSupport's own encoder. It also loads the `nosj/json` drop-in, so
+  `ActiveSupport::JSON.decode` and JSON request-body parsing take the
+  fast path (including on Rails 7.x, whose `quirks_mode` option the
+  drop-in now accepts; the drop-in also accepts valid-UTF-8 BINARY
+  strings now, which is what Rack delivers request bodies as). The
+  HTML-safety escaping is fused into the SIMD string-emission kernels,
+  so escaped output costs the same single pass as unescaped. Measured
+  against stock ActiveSupport encoding: ×1.7 on small documents up to
+  ×5.2 on large trees and ×14 on HTML-heavy content
+  (`rake bench:rails`). In a Rails Gemfile:
+  `gem "nosj", require: "nosj/rails"`.
+- `JSON::Fragment` values now splice their pre-rendered JSON
+  everywhere the `json` gem does: in default mode, under `strict:
+  true`, and through the Rails encoder.
+
 ## [0.2.0] - 2026-07-16
 
 - File APIs. `NOSJ.load_file(path, opts)` parses a file directly

data/Cargo.lock

@@ -189,16 +189,16 @@ dependencies = [
 
 [[package]]
 name = "nosj"
-version = "0.1.1"
+version = "0.2.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "c6ecf0db23b7e9944ecee15ab3dd01b0bde0ab2a195442f363847739f667ac8d"
+checksum = "d25913644b7871b36095cc3da27daf95ebf9508189c5d8ccfb16da3bdb9f802f"
 dependencies = [
  "fast-float2",
 ]
 
 [[package]]
 name = "nosj_native"
-version = "0.2.0"
+version = "0.3.0"
 dependencies = [
  "ahash",
  "magnus",

data/README.md

@@ -13,11 +13,14 @@ faster than Yajl—[see Benchmarks](#benchmarks).
 - It has **lazy documents**: `NOSJ.lazy` wraps a document and parses a value only when you touch it—repeated access costs nanoseconds, and everything you never read is never parsed.
 - It has a **partial parsing mode**: JSON Pointer lookups that pull single values out of big documents in microseconds, skipping everything else.
 - It has **file APIs**: parse, generate, dig, and lazy-wrap files directly—no throwaway file-sized Ruby String, and the partial modes memory-map the file so unread pages never even leave the disk.
+- It does **byte-splicing edits**: `NOSJ.splice` changes a field inside a passing payload 10–51× faster than parse-mutate-generate, and RFC 6902 JSON Patch / RFC 7386 merge patch apply straight to the raw string.
+- It is **great to debug with**: parse errors carry line, column, and a caret snippet pointing at the break, and `NOSJ.stats` X-rays a mystery blob (depth, value counts, key histogram) faster than parsing it.
+- It accelerates a **Rails** application in both encoding and decoding.
 - It comes **precompiled** (platform gems built with per-platform optimizations,
 nothing to compile on install).
 - Otherwise, same API and option names as gem json.
 
-**And there's more**: validate documents without building a single Ruby object, resolve whole batches of paths in one pass, and accelerate an entire application with a one-line drop-in.
+**And there's more**: validate documents without building a single Ruby object, minify and pretty-print without parsing, resolve whole batches of paths in one pass, stream NDJSON / JSON Lines in both directions, and accelerate an entire application with a one-line drop-in.
 
 - [Requirements](#requirements)
 - [Getting started](#getting-started)
@@ -26,6 +29,7 @@ nothing to compile on install).
 - [Switching from the json gem](#switching-from-the-json-gem)
 - [How it works](#how-it-works)
 - [Development](#development)
+- [Acknowledgements](#acknowledgements)
 - [License](#license)
 
 ## Requirements
@@ -50,6 +54,8 @@ NOSJ.generate({"a" => [1, true]})  #=> '{"a":[1,true]}'
 
 That's it—if you know the `json` gem, you already know `nosj`.
 
+### gem json compatibility
+
 Want the speedup without touching your code? One line reroutes
 `JSON.parse`, `JSON.generate`, `JSON.pretty_generate`, and `JSON.dump`
 through nosj:
@@ -65,18 +71,39 @@ Gemfile you can do this:
 gem "nosj", require: "nosj/json"
 ```
 
-Options nosj supports take the fast path; anything exotic
-(`create_additions`, `object_class`, `JSON::State`, procs, IO
-arguments) falls back to the original implementation, so `JSON.load`,
-`JSON.parse!`, and `JSON.load_file` keep their exact behavior.
-Exceptions re-raise as the JSON classes, so your rescue clauses keep
-working. Measured through the patch: parse 1.11×, generate 1.05× over
-the original gem. (A MultiJson adapter ships too:
-`require "nosj/multi_json"`, then `MultiJson.use NOSJ::MultiJsonAdapter`.)
+A MultiJson adapter ships too: 
+
+```ruby
+require "nosj/multi_json"
+```
+
+And then `MultiJson.use NOSJ::MultiJsonAdapter`.
+
+### Ruby on Rails
+
+In a Rails app, use this for "Rails mode":
+
+```ruby
+gem "nosj", require: "nosj/rails"
+```
+
+That installs a nosj-backed ActiveSupport JSON encoder, so `obj.to_json`, `render json:`, and `ActiveSupport::JSON.encode` walk the object tree natively:
+values that aren't JSON-native recurse through `as_json` exactly like
+ActiveSupport's own encoder, non-finite floats encode as `null`, and
+HTML-safety escaping (`escape_html_entities_in_json`) behaves
+identically—verified differentially against ActiveSupport's encoder.
+It loads the drop-in too, so `ActiveSupport::JSON.decode` and JSON
+request-body parsing ride the fast path. 
+
+Measured against ActiveSupport's own encoder: ×1.9 on small documents
+up to ×5.2 on large trees and ×14 on HTML-heavy content—see
+[Benchmarks → Rails mode](#rails-mode).
 
 ## What's in the box
 
-**The `json` gem API**, on the `NOSJ` module:
+### json API
+
+The `json` gem API, on the `NOSJ` module:
 
 ```ruby
 NOSJ.parse(src, symbolize_names: true)   # also: freeze, max_nesting,
@@ -85,7 +112,9 @@ NOSJ.generate(obj)                       # indent, space, object_nl, ...,
 NOSJ.pretty_generate(obj)                # ascii_only, script_safe, strict
 ```
 
-**Lazy documents.** `NOSJ.lazy` wraps a document in a lazy view: read
+### Lazy documents
+
+`NOSJ.lazy` wraps a document in a lazy view: read
 a field and only that path is parsed—containers stay lazy, scalars
 arrive as plain Ruby values, and repeated reads are cached:
 
@@ -101,8 +130,9 @@ Pass a frozen string and creating the view is practically free—
 nanoseconds, even on a megabyte document. Malformed content raises
 when it is first read, not at wrap time.
 
+### Partial parsing
 
-**Partial parsing.** Pull values out of a document without
+Pull values out of a document without
 materializing the rest—skipped content is stepped over at SIMD block
 speed, so a lookup costs what it skips, not what the document weighs:
 
@@ -122,7 +152,9 @@ at the far end of a 570 KB document costs ~71µs, still 13× faster
 than parse-then-dig. Misses return nil; matched subtrees materialize
 with the same options as `parse` (`symbolize_names:`, `freeze:`).
 
-**Files.** Every mode has a file-native form, so a document never
+### Files API
+
+Every mode has a file-native form, so a document never
 round-trips through a throwaway Ruby String:
 
 ```ruby
@@ -138,10 +170,89 @@ read are never loaded from disk. Missing files raise the usual
 `Errno` exceptions. Measured numbers live in
 [Benchmarks → File APIs](#file-apis).
 
-**Validation without parsing.** `NOSJ.valid?` runs the full
-parser—tokenizers, string decode, number validation—into a null sink
-and allocates no Ruby objects at all. It is 2-4× faster than
-`NOSJ.parse`, which already leads every parser above:
+### Reformat without parsing
+
+`NOSJ.minify` and `NOSJ.reformat` pipe the parser's events straight
+into the emission kernels: **zero Ruby objects** are built for the
+document (the specs assert it), SIMD in and SIMD out. Minifying the
+631 KB twitter.json takes 409µs—3.4× faster than a
+`generate(parse(x))` round-trip, 3.9× faster than gem json's cycle,
+and only 1.4× the cost of `valid?`:
+
+```ruby
+NOSJ.minify(json)                        # compact
+NOSJ.reformat(json, pretty: true)        # pretty_generate layout
+NOSJ.reformat(json, ascii_only: true)    # escape-transcode, no parse
+NOSJ.reformat_file("big.json")           # straight off a memory map
+```
+
+Output is exactly `generate(parse(json))`—canonical number spellings,
+normalized escapes, same formatting options—except duplicate keys pass
+through (a reformatter must not silently drop data) and lone-surrogate
+strings re-escape instead of raising. Acceptance options apply too:
+`minify(src, allow_trailing_comma: true)` normalizes the commas away.
+
+### Byte-splicing edits and JSON Patch
+
+Change a field in a passing payload without the parse → mutate →
+generate cycle: `NOSJ.splice` skips to the target spans (one pass for
+the whole batch) and rebuilds the string around them—every byte
+outside the targets is copied untouched, so formatting, key order,
+and number spellings survive exactly. On twitter.json this is
+**10×–51× faster** than parse-mutate-generate (129µs for a late
+field, 26µs for an early one, vs ~1.3ms):
+
+```ruby
+NOSJ.splice(json, "/config/timeout" => 30)
+NOSJ.splice(json, "/a" => 1, "/b/c" => [true])   # batch, one pass
+```
+
+On top of the same machinery: **RFC 6902 JSON Patch** applied to the
+raw string (`add`, `remove`, `replace`, `move`, `copy`, `test`—the
+whole appendix-A suite is in the specs), and **RFC 7386 JSON Merge
+Patch**:
+
+```ruby
+NOSJ.patch(json, [
+  {"op" => "test", "path" => "/a", "value" => 1},
+  {"op" => "replace", "path" => "/a", "value" => 2},
+  {"op" => "add", "path" => "/list/-", "value" => "x"}
+])
+NOSJ.merge_patch(json, {"config" => {"legacy" => nil, "timeout" => 30}})
+```
+
+Missing splice targets raise `KeyError` (use `patch` `add` to
+insert); failed patches raise `NOSJ::PatchError`; inserted values are
+byte-identical to `NOSJ.generate`'s output.
+
+### NDJSON / JSON Lines
+
+Logs, data pipelines, LLM batch files—one JSON document per line.
+`NOSJ.each_line` streams values out (1.6× faster than the
+line-split-and-`JSON.parse` idiom on twitter statuses), and
+`NOSJ.generate_lines` builds the whole stream in one buffer pass
+(4.1× faster than map-generate-join):
+
+```ruby
+NOSJ.each_line(log) { |event| ingest(event) }   # or an Enumerator:
+NOSJ.each_line(log).first(10)                   # walks only 10 lines
+
+NOSJ.generate_lines(events)          #=> %({"a":1}\n{"b":2}\n...)
+NOSJ.each_line_file("events.ndjson") { |e| }    # over a memory map
+NOSJ.write_lines("out.ndjson", events)          # straight to disk
+```
+
+Blank lines are skipped, one value per line is enforced, and a
+malformed line raises a [rich `ParserError`](#rich-parse-errors) whose
+`#line` is the physical line in the stream. Parse options apply per
+line—`NOSJ.each_line(log, freeze: true)` yields Ractor-shareable
+values.
+
+### Validation without parsing
+
+`NOSJ.valid?` runs the full parser—tokenizers, string decode, 
+number validation—into a null sink and allocates no Ruby objects
+at all. It is 2-4× faster than `NOSJ.parse`:
 
 ```ruby
 NOSJ.valid?('{"a":1}')                #=> true
@@ -149,6 +260,46 @@ NOSJ.valid?('{"a":}')                 #=> false
 NOSJ.valid?(src, max_nesting: false)  # same options as parse
 ```
 
+### Document statistics
+
+`NOSJ.stats` answers "what is this 40 MB blob": one counting pass 
+through the same null-sink machinery—no Ruby value is built for 
+the document, and it costs _less_ than a parse (~1.3× faster on `twitter.json`):
+
+```ruby
+s = NOSJ.stats(blob)          # or NOSJ.stats_file("huge.json")
+s[:byte_size]                 #=> 631514
+s[:root]                      #=> :object
+s[:max_depth]                 #=> 10
+s[:values]                    #=> {total: 13914, objects: 1264, arrays: 1050,
+                              #    strings: 4754, integers: 2108, ...}
+s[:keys]                      #=> {total: 13345, unique: 94}
+s[:key_histogram].first(3)    #=> [["id", 447], ["id_str", 447], ["urls", 364]]
+s[:containers]                #=> {max_object_entries: 40, max_array_length: 100}
+s[:strings]                   #=> {bytes: 200716, max_bytes: 463}
+```
+
+Nesting is unlimited by default (a deep blob is exactly what a
+diagnostic should describe); malformed documents raise the usual rich
+`ParserError`.
+
+### Rich parse errors
+
+A failed parse raises `NOSJ::ParserError`
+carrying where the document broke—`#byte_offset`, `#line`, `#column`,
+and a caret `#snippet`—computed only when a parse fails, so success
+pays nothing. Positions stay absolute through partial parsing, lazy
+documents, and the file APIs; unrescued errors print the snippet:
+
+```ruby
+NOSJ.parse(%({\n  "a": 1,\n  "b": }))
+# NOSJ::ParserError: unexpected character at byte 19
+#   e.line     #=> 3
+#   e.column   #=> 8
+#   e.snippet  #=>   "b": }
+#                         ^
+```
+
 ## Benchmarks
 
 Every installed JSON gem, benchmark-ips: AWS EC2 c7a.2xlarge (AMD EPYC 9R14, Zen 4), Ruby 4.0.6 + YJIT, json 2.21.1, Oj 3.17.4, RapidJSON 0.4.0, FastJsonparser 0.6.0, Yajl 1.4.3, PGO build, 2026-07-16. `×N` = times slower than nosj. 
@@ -209,6 +360,27 @@ Silicon dev box, Ruby 4.0.6 + YJIT, PGO build, 2026-07-16):
 too much for an honest multiplier; what it saves is the intermediate
 file-sized Ruby String.
 
+### Rails mode
+
+`ActiveSupport::JSON.encode` with the nosj encoder installed, against
+stock ActiveSupport (Apple Silicon dev box, Ruby 4.0.6 + YJIT,
+activesupport 8.1, medians of 5 interleaved per-process rounds,
+outputs verified byte-identical first, 2026-07-17;
+`rake bench:rails`):
+
+| workload | nosj (i/s) | vs ActiveSupport |
+|---|---:|---|
+| twitter tree (570 KB) | 4.5k | ×5.2 |
+| 100-record index (with timestamps) | 33.7k | ×1.7 |
+| HTML-heavy user content | 177.7k | ×14.2 |
+| Time/Date/BigDecimal hash | 1.1M | ×3.0 |
+| small API hash | 4.2M | ×1.9 |
+| small hash `to_json` | 4.1M | ×1.9 |
+
+The HTML-safety escaping that dominates stock encodes of
+user-generated content is fused into the SIMD string-emission kernels
+here: escaped output costs the same single pass as unescaped.
+
 Reproduce with `rake bench` (the parity-gated comparison, after a PGO retrain—the shipping configuration) or `rake bench:ips` (the multi-gem shoot-out).
 
 ## Switching from the json gem
@@ -222,8 +394,10 @@ You mostly don't have to do anything. Some differences:
   UTF-8) follow the strict semantics instead.
 - Unlike `Array#dig`, negative indices in `NOSJ.dig` return nil (JSON
   Pointer has no equivalent).
-- Parse error *messages* use byte offsets rather than the gem's
-  phrasing (classes match).
+- Parse errors raise `NOSJ::ParserError` (`NOSJ::NestingError` past
+  `max_nesting`, like the gem); messages use byte offsets rather than
+  the gem's phrasing, and the exception carries `#line`, `#column`,
+  and a caret `#snippet`.
 
 Everything else—including the gem's exact float formatting, which is
 not the shortest-round-trip form most libraries emit—matches
@@ -260,6 +434,28 @@ bundle exec rake bench:fast             # the sweep without retraining
 bundle exec rake "bench:ips[twitter]"   # multi-gem shoot-out (benchmark-ips); no args = full corpus
 ```
 
+## Acknowledgements
+
+Thanks to [Jean Boussier](https://github.com/byroot), [Florian Frank](https://github.com/flori), [Hiroshi Shibata](https://github.com/hsbt), [Nobuyoshi Nakada](https://github.com/nobu), [Étienne Barrié](https://github.com/etiennebarrie), and the other authors and maintainers of the [json gem](https://github.com/ruby/json)—for their work on the gem itself, for optimization ideas, and for some of the JSON documents in the benchmark corpus.
+
+Thanks to [Mat Sadler](https://github.com/matsadler) for [magnus](https://github.com/matsadler/magnus).
+
+Thanks to the authors of the projects credited in [NOTICE](NOTICE) and the
+crate's NOTICE: [fpconv](https://github.com/night-shift/fpconv) and
+Florian Loitsch's Grisu2,
+[fast_float](https://github.com/fastfloat/fast_float) and
+[fast_double_parser](https://github.com/lemire/fast_double_parser)
+(Daniel Lemire et al.),
+[simdjson](https://github.com/simdjson/simdjson) (Daniel Lemire, Geoff
+Langdale, and contributors), and the architectural lineage of
+[yyjson](https://github.com/ibireme/yyjson),
+[simd-json](https://github.com/simd-lite/simd-json), and
+[sonic-rs](https://github.com/cloudwego/sonic-rs).
+
+## Assisted by
+
+Claude Fable 5, Claude Opus 4.8.
+
 ## License
 
 MIT. The underlying Rust crate is `MIT AND BSL-1.0 AND Apache-2.0`; its

data/ext/nosj/Cargo.toml

@@ -5,7 +5,7 @@
 # required by lib/nosj.rb as "nosj/nosj" (Init_nosj set explicitly in
 # lib.rs). Gem name, module, and API are plain nosj.
 name = "nosj_native"
-version = "0.2.0"
+version = "0.3.0"
 edition = "2021"
 authors = ["Yaroslav Markin <yaroslav@markin.net>"]
 license = "MIT"
@@ -25,4 +25,4 @@ memmap2 = "0.9"
 # First-party SIMD JSON parse/generate library (github.com/yaroslav/nosj).
 # For coordinated crate+gem work, temporarily flip to
 # { path = "../../../nosj" } and restore before any commit or release.
-nosj = "0.1.1"
+nosj = "0.2.0"

data/ext/nosj/src/errors.rs

@@ -0,0 +1,171 @@
+//! Parse-side exception raising. Malformed documents raise
+//! `NOSJ::ParserError` carrying the failure position (`#byte_offset`,
+//! `#line`, `#column`) and a caret snippet (`#snippet`); too-deep
+//! nesting raises `NOSJ::NestingError`, matching gem json's classes.
+//!
+//! Everything here runs only when a parse has already failed: the
+//! position pass over the source and the exception construction cost
+//! nothing on the happy path, and the accessors on the Ruby side are
+//! plain ivar reads.
+
+use magnus::value::ReprValue;
+use magnus::{Class, Error, ExceptionClass, Module, Object, RModule, RObject, Ruby};
+
+/// Look up an exception class under NOSJ, falling back to RuntimeError
+/// if the Ruby layer has not defined it (only possible when the native
+/// extension is loaded without lib/nosj.rb).
+pub(crate) fn nosj_exception(ruby: &Ruby, name: &str) -> ExceptionClass {
+    let lookup = || -> Result<ExceptionClass, Error> {
+        let m: RModule = ruby.define_module("NOSJ")?;
+        m.const_get(name)
+    };
+    lookup().unwrap_or_else(|_| ruby.exception_runtime_error())
+}
+
+/// NOSJ::ParserError without position info (encoding refusals and
+/// other failures that have no meaningful offset).
+#[cold]
+pub(crate) fn parser_error(ruby: &Ruby, msg: String) -> Error {
+    Error::new(nosj_exception(ruby, "ParserError"), msg)
+}
+
+/// NOSJ::NestingError (parse side: message parity with gem json, which
+/// raises JSON::NestingError when max_nesting is exceeded).
+#[cold]
+pub(crate) fn nesting_error(ruby: &Ruby, msg: String) -> Error {
+    Error::new(nosj_exception(ruby, "NestingError"), msg)
+}
+
+/// Plain RuntimeError, for failures that are not parse errors (the
+/// unmappable-I/O fallback).
+#[cold]
+pub(crate) fn runtime_error(ruby: &Ruby, msg: String) -> Error {
+    Error::new(ruby.exception_runtime_error(), msg)
+}
+
+/// NOSJ::ParserError enriched with the failure position: `offset` is a
+/// byte offset into `source` (the full document, so positions stay
+/// absolute even when the failing parse ran over a subtree slice).
+#[cold]
+pub(crate) fn parser_error_at(ruby: &Ruby, source: &[u8], offset: usize, msg: String) -> Error {
+    let class = nosj_exception(ruby, "ParserError");
+    let loc = locate(source, offset);
+    let Ok(exc) = class.new_instance((msg.as_str(),)) else {
+        return Error::new(class, msg);
+    };
+    // Exception instances are plain T_OBJECTs; magnus exposes ivar_set
+    // through RObject, not through its Exception wrapper.
+    if let Some(obj) = RObject::from_value(exc.as_value()) {
+        let set = |name: &str, v: usize| {
+            let _ = obj.ivar_set(name, v);
+        };
+        set("@byte_offset", offset.min(source.len()));
+        set("@line", loc.line);
+        set("@column", loc.column);
+        if let Some(snippet) = &loc.snippet {
+            let _ = obj.ivar_set("@snippet", snippet.as_str());
+        }
+    }
+    Error::from(exc)
+}
+
+/// A resolved source position: 1-based line, 1-based character column
+/// within that line, and a two-line caret snippet (`None` when the
+/// offending line is empty or not valid UTF-8).
+struct Location {
+    line: usize,
+    column: usize,
+    snippet: Option<String>,
+}
+
+/// UTF-8 continuation bytes are 0b10xxxxxx; every other byte starts a
+/// character, so counting non-continuation bytes counts characters.
+const UTF8_CONTINUATION_MASK: u8 = 0b1100_0000;
+const UTF8_CONTINUATION_BITS: u8 = 0b1000_0000;
+
+fn count_chars(bytes: &[u8]) -> usize {
+    bytes
+        .iter()
+        .filter(|&&b| b & UTF8_CONTINUATION_MASK != UTF8_CONTINUATION_BITS)
+        .count()
+}
+
+fn locate(source: &[u8], offset: usize) -> Location {
+    let off = offset.min(source.len());
+    let line = 1 + count_newlines(&source[..off]);
+    let line_start = source[..off]
+        .iter()
+        .rposition(|&b| b == b'\n')
+        .map_or(0, |p| p + 1);
+    let mut line_end = source[off..]
+        .iter()
+        .position(|&b| b == b'\n')
+        .map_or(source.len(), |p| off + p);
+    if line_end > line_start && source[line_end - 1] == b'\r' {
+        line_end -= 1;
+    }
+    // An offset sitting on the line terminator itself carets one past
+    // the last character of the line's content.
+    let caret = off.clamp(line_start, line_end);
+    let column = 1 + count_chars(&source[line_start..caret]);
+    let snippet = std::str::from_utf8(&source[line_start..line_end])
+        .ok()
+        .and_then(|l| build_snippet(l, caret - line_start));
+    Location {
+        line,
+        column,
+        snippet,
+    }
+}
+
+fn count_newlines(bytes: &[u8]) -> usize {
+    bytes.iter().filter(|&&b| b == b'\n').count()
+}
+
+/// Characters kept around the caret. Minified JSON is routinely one
+/// multi-kilobyte line, so the snippet shows a window, not the line.
+const SNIPPET_CHARS_BEFORE: usize = 50;
+const SNIPPET_CHARS_AFTER: usize = 30;
+const SNIPPET_ELLIPSIS: char = '…';
+
+/// Render the offending line plus a caret line underneath, windowed
+/// around `caret_byte` (a byte offset into `line`).
+fn build_snippet(line: &str, caret_byte: usize) -> Option<String> {
+    if line.is_empty() {
+        return None;
+    }
+    let mut caret_byte = caret_byte.min(line.len());
+    while !line.is_char_boundary(caret_byte) {
+        caret_byte -= 1;
+    }
+    let caret_chars = line[..caret_byte].chars().count();
+    let total_chars = caret_chars + line[caret_byte..].chars().count();
+
+    let window_first = caret_chars.saturating_sub(SNIPPET_CHARS_BEFORE);
+    let window_last = (caret_chars + SNIPPET_CHARS_AFTER).min(total_chars);
+    let mut caret_column = caret_chars - window_first;
+
+    let mut out = String::new();
+    if window_first > 0 {
+        out.push(SNIPPET_ELLIPSIS);
+        caret_column += 1;
+    }
+    for ch in line
+        .chars()
+        .skip(window_first)
+        .take(window_last - window_first)
+    {
+        // Control characters (tabs included) render as one space so
+        // the caret column stays aligned with what is printed.
+        out.push(if ch.is_control() { ' ' } else { ch });
+    }
+    if window_last < total_chars {
+        out.push(SNIPPET_ELLIPSIS);
+    }
+    out.push('\n');
+    for _ in 0..caret_column {
+        out.push(' ');
+    }
+    out.push('^');
+    Some(out)
+}