smarter_json 0.9.9 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fbc93b4afea26fc4d30c241ceb7823451cb7b777454441a9871f066b719f8c07
-  data.tar.gz: a9a5801cab6a604e4d166d6d86aa48f41f5e1bb74fb58f39f283a0477ac78db4
+  metadata.gz: 84a73a6cf0785c67eb2dfaf87dc663b860d5afed6bf0816f861b6430d1f55475
+  data.tar.gz: 953aebf65ab855450a7d3b41c826c169242dd19a190f089da3f52df2da0b0a44
 SHA512:
-  metadata.gz: c8749bd358973f0d284966a6c5fb42a4e71954c8884c337c5859f5f3e566b302d1a4c1dbc9961ed5ae46aaf5a9e019935fa46916bb858877a2ef34ecc99575c9
-  data.tar.gz: 2f2efe9c89ae08bcf807e061ca734d13056c6fafc590d072d7870a705cf25e3d4932fff567c1b09320f5cbd6b42e02255180fded05c84b08f1032abb0594b8bf
+  metadata.gz: 8fe6e07fd99f1557a716fc37370dfc3c5cdb34e054d07fa812983d6cefea1e74108bb0e708fdb42f47a31ff8435aa0d6b8c80deeebab09a221bbd9992691be28
+  data.tar.gz: ec166df8863b136abc38df844bba2286b542de03f24d410b6e4a5914bfcf2165337add25c5731b92540328c5496ad4ec658243e21e4830687de9f44dcaaf4d54

data/.gitignore

@@ -45,3 +45,4 @@ overage/
 .claude/
 CLAUDE.md
 INTERNAL_DEV_LOG.md
+research

data/CHANGELOG.md

@@ -1,20 +1,31 @@
 
 # SmarterJSON Change Log
 
-> 🚧 Getting ready for the 1.0.0 release - sorry for the interface changes - thank you for your patience! 🚧
-
-> ⚠️ **Interface change (since 0.9.7):** 
+> ⚠️ **New Interface (since 0.9.7):**
+>
+> SmarterJSON **always returns an `Array`** of documents.
 > 
-> `SmarterJSON.process` / `SmarterJSON.process_file` now **always return an `Array`** of documents:
->  — `[]` for no doc
->  - `[doc]` for one doc
->  - `[d1, d2, …]` for several docs (NDJSON / JSONL / concatenated docs). 
-
-Going forward this will be the supported interface.
+> `SmarterJSON.process` / `SmarterJSON.process_file` return:
+>
+> — `[]` for no doc
+> - `[doc]` for one doc
+> - `[d1, d2, …]` for several docs (NDJSON / JSONL / concatenated docs)
 
-> ⚠️ We discourage the use of `process(input).first` / `[0]` because it silently drops potential additional documents
+> ⚠️ We discourage the use of `process(input).first` / `process(input)[0]` because it silently drops potential additional documents
 >    Please use `process_one` if you are expecting only one JSON doc, e.g. in API payloads.
 
+## 1.0.0 (2026-06-08)
+
+RSpec tests: 1,034
+
+- **The public interface is now stable** — `process`, `process_one`, `process_file`, `generate`, and the documented options; semantic versioning from here on.
+- Unknown or wrongly-typed options now raise `ArgumentError` instead of being silently ignored, so a typo (e.g. `symbolize_names:` instead of `symbolize_keys:`) is caught immediately.
+- Input tagged `ASCII-8BIT` whose bytes are valid UTF-8 (e.g. a `Net::HTTP` `response.body`) is now read as UTF-8, so its string values compare equal to UTF-8 literals; ASCII-8BIT input that is not valid UTF-8 raises `SmarterJSON::EncodingError` (pass an explicit `encoding:` for legacy encodings).
+- Object keys may now use smart/curly quotes too (e.g. JSON pasted from a word processor), not just string values.
+- `SmarterJSON.generate` accepts `allow_nan: true` to emit `NaN` / `Infinity` / `-Infinity` (JSON5-style) instead of raising, so non-finite numbers round-trip; the default still raises.
+- A numeric literal that overflows `Float` range (e.g. `1e400`) now reports a `:number_overflow` warning via `on_warning` instead of silently becoming `Infinity`.
+- `SmarterJSON.generate` is now iterative (like the parser), so serializing a deeply nested structure no longer risks `SystemStackError` — reading and writing are both depth-safe.
+
 ## 0.9.9 (2026-06-07)
 - Much faster pure-Ruby parsing (the path used without the C extension) — roughly 3× on string-heavy data, ~2× on number-heavy, ~1.7× on object-heavy (on a YJIT-enabled Ruby). Parsed values are unchanged.
 

data/README.md

@@ -62,7 +62,7 @@ Three things set it apart:
 - Trailing commas; unquoted keys (`{host: localhost}`); single-quoted, triple-quoted (`'''…'''`), and quoteless string values
 - Implicit root object — a config file that starts with `key: value`, no outer `{}`
 - `NaN`, `Infinity`, hex (`0xFF`), leading `+` / `.`, underscores in numbers (`1_000_000`)
-- UTF-8 BOM, smart/curly quotes, Python literals (`True` / `False` / `None`), JavaScript `undefined`
+- UTF-8 BOM, smart/curly quotes (in keys and values), Python literals (`True` / `False` / `None`), JavaScript `undefined`
 - Mixed CR / LF / CRLF line endings, and any Ruby-supported input encoding (via `encoding:`)
 - Duplicate keys (last value wins by default; configurable)
 
@@ -165,26 +165,26 @@ SmarterJSON is a C extension (with a pure-Ruby fallback that runs everywhere). B
 - **None of the others parse JSON5, HJSON-style config, or LLM-wrapped output.** Comments, trailing commas, unquoted keys, quoteless values, `'single quotes'`, markdown code fences, prose wrappers — all raise in Oj / `json` / Yajl; SmarterJSON parses them.
 - **`json` and Yajl produce `Float` only — lossy on high-precision numbers.** On coordinate / scientific data (>16 significant digits) they silently round to `Float`, so they aren't a like-for-like comparison there. SmarterJSON (and Oj) keep full precision as `BigDecimal` by default.
 
-Where a like-for-like comparison exists, here is SmarterJSON's C path against each parser. **Apple M4, Ruby 3.4.7, p10 of 40 runs.** Each cell is **SmarterJSON vs that parser** — "faster" means SmarterJSON wins. Ratios shift with hardware; run `rake report` in `json_benchmarks/` to reproduce.
+Where a like-for-like comparison exists, here is SmarterJSON's C path against each parser. **Apple M4, Ruby 3.4.7, p10 of 40 runs (2026-06-07); the same picture holds on an Apple M1 Max.** Each cell is **SmarterJSON vs that parser** — "faster" means SmarterJSON wins. Ratios shift with hardware; run `rake report` in `json_benchmarks/` to reproduce.
 
 | File                          | vs Oj/strict    | vs `json`                    | vs Yajl         |
 | ----------------------------- | --------------- | ---------------------------- | --------------- |
-| big_decimals <sup>≠</sup>     | **1.8× faster** | **1.1× faster**              | **1.3× faster** |
-| canada <sup>≠</sup>           | **8× faster**   | 1.1× slower                  | **2.2× faster** |
-| citm_catalog                  | **1.6× faster** | 1.2× slower                  | **4.8× faster** |
-| citylots <sup>≠</sup>         | **3.6× faster** | **2.0× faster**              | **2.3× faster** |
-| config.jsonc                  | **1.1× faster** | 1.5× slower                  | **3.7× faster** |
-| deeply_nested                 | **1.4× faster** | **can't parse** <sup>‡</sup> | **5.1× faster** |
-| github_events                 | **1.2× faster** | ≈ tied                       | **3.1× faster** |
+| big_decimals <sup>≠</sup>     | **1.7× faster** | ≈ tied                       | **1.2× faster** |
+| canada <sup>≠</sup>           | **7× faster**   | ≈ tied                       | **2.1× faster** |
+| citm_catalog                  | **1.3× faster** | 1.2× slower                  | **3.2× faster** |
+| citylots <sup>≠</sup>         | **3.7× faster** | **2.0× faster**              | **2.3× faster** |
+| config.jsonc                  | **1.1× faster** | 1.2× slower                  | **3.6× faster** |
+| deeply_nested                 | **1.2× faster** | **can't parse** <sup>‡</sup> | **4.1× faster** |
+| github_events                 | ≈ tied          | 1.1× slower                  | **2.7× faster** |
 | string_array                  | ≈ tied          | ≈ tied                       | **1.6× faster** |
-| twitter                       | **1.4× faster** | 1.3× slower                  | **3.5× faster** |
-| usgs_earthquakes <sup>≠</sup> | **1.3× faster** | 1.5× slower                  | **3.6× faster** |
-| weather_berlin                | **1.9× faster** | 1.1× slower                  | **3.5× faster** |
+| twitter                       | **1.3× faster** | 1.2× slower                  | **3.2× faster** |
+| usgs_earthquakes <sup>≠</sup> | **1.4× faster** | 1.1× slower                  | **3.4× faster** |
+| weather_berlin                | **1.8× faster** | **1.1× faster**              | **3.2× faster** |
 
 <sup>≠</sup> High-precision file. The row uses `decimal_precision: :float` (Float, like-for-like) for `canada` / `citylots` / `big_decimals` / `usgs`. SmarterJSON's **default** `:auto` keeps these decimals as `BigDecimal` (no precision loss, like Oj's default) — intrinsically slower than `Float`, so default-vs-`Float` would be apples-to-oranges. Against Oj's matching `BigDecimal` default, SmarterJSON is faster there too.
 <sup>‡</sup> Not a measurement gap — `json` raises by default: it errors on multi-document / NDJSON input without a block, and caps nesting at 100 levels. SmarterJSON has neither limit.
 
-In short: **matches or beats Oj/strict on every file** — `string_array` is the one wash (within ~10%, and hardware-dependent: SmarterJSON edges ahead on an M1, Oj edges ahead on an M4) — **far faster than Yajl everywhere, and level-to-ahead of stdlib `json` on a like-for-like basis**, while parsing input `json` and Oj reject outright. Floats are decoded with the **Eisel-Lemire** algorithm (fast_float), correctly rounded and **bit-for-bit identical to `JSON.parse`** — fast *and* exact, even at full double precision.
+In short: **SmarterJSON's C path matches or beats Oj/strict on every file** (apples-to-apples — for the high-precision <sup>≠</sup> files that means `decimal_precision: :float`, where Oj/strict also produces `Float`; with `:float`, float-heavy data like `canada` is **~7× faster**). It is **far faster than Yajl everywhere**, and **level-to-ahead of stdlib `json`** — `json` edges ahead only on a few object-heavy files (`citm`, `twitter`, `config.jsonc`, `github_events`, all within ~1.25×) and **can't parse `deeply_nested` at all**. Floats are decoded with the **Eisel-Lemire** algorithm (fast_float), correctly rounded and **bit-for-bit identical to `JSON.parse`** — fast *and* exact, even at full double precision.
 
 **Two notes on fair comparison:**
 
@@ -200,8 +200,8 @@ In short: **matches or beats Oj/strict on every file** — `string_array` is the
 | `duplicate_key`   | `:last_wins` | `:last_wins` / `:first_wins` for a key repeated in one object (every repeat is also reported via `on_warning`) |
 | `decimal_precision` | `:auto`      | `:auto` keeps high-precision decimals as `BigDecimal`; `:float` forces `Float`; `:bigdecimal` forces `BigDecimal` |
 | `acceleration`    | `true`       | `true` uses the C extension when compiled and loadable; `false` forces pure Ruby (identical results) |
-| `encoding`        | `"UTF-8"`    | labels the input's encoding (no transcoding pass; see below)            |
-| `on_warning`      | `nil`        | a callable invoked once per lenient fix applied (`:empty_slot`, `:empty_value`, `:duplicate_key`), passed a `SmarterJSON::Warning`; the return value is never changed. See below. |
+| `encoding`        | `nil`        | labels the input's encoding; `nil` keeps the input's own (no transcoding pass; see below) |
+| `on_warning`      | `nil`        | a callable invoked once per lenient fix applied (`:empty_slot`, `:empty_value`, `:duplicate_key`, `:number_overflow`), passed a `SmarterJSON::Warning`; the return value is never changed. See below. |
 
 ## Examples
 
@@ -295,11 +295,11 @@ TEXT
 
 ## Encoding
 
-`encoding:` (default `"UTF-8"`) labels what the input is — it does **not** trigger a transcoding pass. SmarterJSON works on the bytes in their native encoding and emits string values with the same encoding tag, the same way `smarter_csv` handles encodings. Bytes that are invalid for the claimed encoding raise `SmarterJSON::EncodingError` (a kind of `SmarterJSON::ParseError`).
+`encoding:` (default `nil`) labels what the input is — it does **not** transcode. With `nil`, SmarterJSON keeps the input's own encoding tag and emits string values with that same tag, the way `smarter_csv` does — **with one smart default:** input tagged `ASCII-8BIT` (BINARY) whose bytes are valid UTF-8 is treated as UTF-8. That is exactly how `Net::HTTP` and many HTTP libraries hand you a `response.body` (correct UTF-8 bytes, BINARY tag); without this, string values would come back tagged `ASCII-8BIT` and compare unequal to UTF-8 literals. If such `ASCII-8BIT` input is *not* valid UTF-8, it raises `SmarterJSON::EncodingError` rather than guess a legacy encoding — pass an explicit `encoding:` (e.g. `"ISO-8859-1"`) for that. Bytes invalid for an explicitly claimed encoding also raise `SmarterJSON::EncodingError` (a kind of `SmarterJSON::ParseError`).
 
 ## Nesting & untrusted input
 
-Both the C extension and the pure-Ruby engine are **iterative, not recursive** — they track nesting on an explicit, heap-allocated stack rather than the call stack. So deeply nested input **cannot overflow the call stack or segfault**: nesting is bounded only by available memory, the same posture as Oj (which also ships no nesting limit; the stdlib `json` caps at 100). The `deeply_nested.json` benchmark (212 MB of nesting) is handled without issue.
+Both the C extension and the pure-Ruby engine are **iterative, not recursive** — they track nesting on an explicit, heap-allocated stack rather than the call stack. So deeply nested input **cannot overflow the call stack or segfault**: nesting is bounded only by available memory, the same posture as Oj (which also ships no nesting limit; the stdlib `json` caps at 100). The `deeply_nested.json` benchmark (212 MB of nesting) is handled without issue. **`generate` is iterative too**, so serializing a deeply nested Ruby structure can't overflow the stack either — reading *and* writing are both depth-safe.
 
 The trade-off: there is currently **no fixed nesting or input-size limit**, so extremely large or adversarially-nested untrusted input is bounded by memory (it can exhaust RAM), not by a crash. If you process untrusted input and want a hard cap, that's a planned opt-in guard — for now, size-limit upstream.
 

data/docs/_introduction.md

@@ -23,7 +23,7 @@ Most JSON parsers reject anything that isn't perfectly strict JSON, and they mak
 
 * **It reads multi-document input automatically — a distinguishing feature.** `SmarterJSON.process` handles NDJSON / JSONL / concatenated JSON with **no block and no special method**: it always returns an `Array` of the documents found (`[]` / `[doc]` / `[d1, d2, …]`). For the common single-document case, `SmarterJSON.process_one` returns the one value directly (and warns, never raises, if there was more than one). The same rule applies when wrapper noise is stripped and several payloads are recovered from one blob. **Only SmarterJSON reads multi-document input via plain `process` — Oj and the stdlib `json` library raise without a block.** For input larger than memory, pass a block to stream one document at a time. See [The Basic Read API](./basic_read_api.md).
 
-* **It's fast.** A C extension (with a pure-Ruby fallback that runs everywhere) matches or beats Oj on every file we benchmark, and is competitive with the stdlib `json` C parser. Floats are decoded with the **Eisel-Lemire** algorithm (fast_float), correctly rounded and bit-for-bit identical to `JSON.parse`, so number-heavy data is fast and exact.
+* **It's fast.** The C extension (with a pure-Ruby fallback that runs everywhere) is **faster than Oj/strict on every file** in our benchmark suite — up to **~7× faster on float-heavy data** with `decimal_precision: :float` — **far faster than Yajl**, and **level-to-ahead of the stdlib `json` C parser**, which can't even parse deeply-nested input. Floats are decoded with the **Eisel-Lemire** algorithm (fast_float), correctly rounded and bit-for-bit identical to `JSON.parse`, so number-heavy data is fast and exact. Full per-file numbers (Apple M4 / M1, relative ratios) are in the [README Performance section](../README.md#performance).
 
 * **It writes JSON too.** `SmarterJSON.generate` turns Ruby values into strict, interoperable JSON — or into NDJSON, one element per line, the exact inverse of reading NDJSON back into an Array. See [The Basic Write API](./basic_write_api.md).
 

data/docs/basic_write_api.md

@@ -58,7 +58,7 @@ SmarterJSON.generate(Float::INFINITY)   # raises SmarterJSON::GenerateError —
 SmarterJSON.generate(Float::NAN)        # raises SmarterJSON::GenerateError — non-finite Float
 ```
 
-(`GenerateError` is a kind of `SmarterJSON::Error`, so `rescue SmarterJSON::Error` catches it. `Infinity` and `NaN` are accepted on the *read* side as a leniency, but they are not valid JSON to *write*.)
+(`GenerateError` is a kind of `SmarterJSON::Error`, so `rescue SmarterJSON::Error` catches it. `Infinity` and `NaN` are accepted on the *read* side as a leniency; to *write* them, pass `allow_nan: true` and they're emitted as `NaN` / `Infinity` / `-Infinity` (JSON5-style, so SmarterJSON reads them back) — otherwise non-finite values raise, since they aren't valid strict JSON.)
 
 By default `generate` is strict: it only writes the types above and raises on anything else. To serialize `Time`, `Date`, or your own objects, pass `coerce: true` — an unsupported value is then converted by its own `as_json` (whose result is re-emitted, so escaping/`indent`/`sort_keys` still apply) or, failing that, `to_json` (spliced verbatim):
 

data/docs/options.md

@@ -13,7 +13,7 @@
 
 ## Reading
 
-These options are passed to [`SmarterJSON.process`](./basic_read_api.md), `SmarterJSON.process_one`, and `SmarterJSON.process_file` as the second argument; anything you set overrides the defaults below.
+These options are passed to [`SmarterJSON.process`](./basic_read_api.md), `SmarterJSON.process_one`, and `SmarterJSON.process_file` as the second argument; anything you set overrides the defaults below. Configuration is strict: an unknown option key, or a known key given the wrong type or value, raises `ArgumentError` immediately — leniency applies to the JSON *input* you parse, not to the options you pass, so a typo (e.g. `symbolize_names:` instead of `symbolize_keys:`) is caught right away.
 
 | Option            | Default      | Explanation                                                                                                            |
 |-------------------|--------------|------------------------------------------------------------------------------------------------------------------------|
@@ -43,11 +43,11 @@ warns.map(&:type)              # => [:empty_slot]
 warns.first.to_s               # => "extra comma, collapsed an empty slot at line 1, col 4"
 ```
 
-The warning types are `:empty_slot` (a collapsed empty comma slot, e.g. `[1,,2]`), `:empty_value` (a key with no value, read as `null`, e.g. `{a:}`), and `:duplicate_key` (a repeated key that was dropped), plus wrapper-recovery warnings such as `:code_fence_stripped`, `:prefix_text_ignored`, `:suffix_text_ignored`, and `:wrapper_tag_stripped`. Clean input never invokes the handler. Warnings work on both the C and pure-Ruby paths, so `acceleration:` doesn't change them.
+The warning types are `:empty_slot` (a collapsed empty comma slot, e.g. `[1,,2]`), `:empty_value` (a key with no value, read as `null`, e.g. `{a:}`), `:duplicate_key` (a repeated key that was dropped), and `:number_overflow` (a numeric literal too large for `Float`, e.g. `1e400`, collapsed to `Infinity`), plus wrapper-recovery warnings such as `:code_fence_stripped`, `:prefix_text_ignored`, `:suffix_text_ignored`, and `:wrapper_tag_stripped`. Clean input never invokes the handler. Warnings work on both the C and pure-Ruby paths, so `acceleration:` doesn't change them.
 
 ### A note on `:encoding`
 
-`:encoding` labels what the input *is* — it does not transcode. SmarterJSON works on the bytes in their native encoding and emits string values with the same encoding tag, the same way `smarter_csv` handles encodings. Bytes that are invalid for the claimed encoding raise `SmarterJSON::EncodingError` (a kind of `SmarterJSON::ParseError`). A UTF-8 BOM is handled automatically; UTF-16 / UTF-32 input is out of scope.
+`:encoding` labels what the input *is* — it does not transcode. With the default `nil`, SmarterJSON keeps the input's own encoding tag and emits string values with that tag, the same way `smarter_csv` handles encodings — **with one smart default:** input tagged `ASCII-8BIT` (BINARY) that is valid UTF-8 is treated as UTF-8. This is how `Net::HTTP` returns a `response.body`; without it, those string values would compare unequal to UTF-8 literals. `ASCII-8BIT` input that is *not* valid UTF-8 raises `SmarterJSON::EncodingError` — pass an explicit `:encoding` (e.g. `"ISO-8859-1"`) for genuinely-legacy bytes. Bytes invalid for an explicitly claimed encoding also raise `SmarterJSON::EncodingError` (a kind of `SmarterJSON::ParseError`). A UTF-8 BOM is handled automatically; UTF-16 / UTF-32 input is out of scope.
 
 ### A note on `:decimal_precision`
 
@@ -59,14 +59,15 @@ These options are passed to [`SmarterJSON.generate`](./basic_write_api.md) as th
 
 | Option     | Default | Explanation                                                                                                                |
 |------------|---------|-----------------------------------------------------------------------------------------------------------------------------|
+| `:allow_nan`    | `false` | When `true`, non-finite `Float`/`BigDecimal` values emit the JSON5 barewords `NaN` / `Infinity` / `-Infinity` (which SmarterJSON reads back, so they round-trip). When `false` (the default), a non-finite number raises `SmarterJSON::GenerateError` — they aren't valid strict JSON. |
+| `:ascii_only`   | `false` | Escape every non-ASCII character as `\uXXXX` (astral characters as a UTF-16 surrogate pair). The default emits raw UTF-8. |
+| `:coerce`       | `false` | When `true`, a value that isn't natively supported is converted by its own `as_json` (the result is re-emitted, so the other options still apply) or, failing that, `to_json` (spliced verbatim). When `false` (the default), such a value raises `SmarterJSON::GenerateError`. |
 | `:format`       | `:json` | `:json` writes standard JSON (Hash → object, Array → array, scalar → scalar). `:ndjson` writes newline-delimited JSON: an Array becomes one element per line, any other value becomes a single line. |
 | `:indent`       | `0`     | Spaces per nesting level for pretty-printing. `0` (the default) is compact output. Empty objects/arrays stay inline. Not allowed with `:ndjson` (a record must be a single line). |
-| `:sort_keys`    | `false` | Emit object keys in sorted order (Symbol keys sorted by their string form). Useful for canonical, diff-friendly output. |
-| `:ascii_only`   | `false` | Escape every non-ASCII character as `\uXXXX` (astral characters as a UTF-16 surrogate pair). The default emits raw UTF-8. |
 | `:script_safe`  | `false` | Escape the `/` in `</` and the JS line separators U+2028 / U+2029, so output is safe to embed in an HTML `<script>` tag. |
-| `:coerce`       | `false` | When `true`, a value that isn't natively supported is converted by its own `as_json` (the result is re-emitted, so the other options still apply) or, failing that, `to_json` (spliced verbatim). When `false` (the default), such a value raises `SmarterJSON::GenerateError`. |
+| `:sort_keys`    | `false` | Emit object keys in sorted order (Symbol keys sorted by their string form). Useful for canonical, diff-friendly output. |
 
-Any other `:format` value, a negative/non-Integer `:indent`, or combining `:indent` with `:ndjson`, raises `ArgumentError`.
+Configuration is validated up front: an unknown option key, a known key with the wrong type or value (a non-Symbol `:format`, a negative/non-Integer `:indent`, a non-boolean flag), or combining `:indent` with `:ndjson`, raises `ArgumentError`.
 
 ```ruby
 SmarterJSON.generate([1, 2, 3])                              # => "[1,2,3]"   (default :json — a single JSON array)

data/ext/smarter_json/smarter_json.c

@@ -40,6 +40,7 @@ static ID    fj_call_id;    /* cached :call (invoking the on_warning handler) */
 static VALUE fj_sym_empty_slot;
 static VALUE fj_sym_empty_value;
 static VALUE fj_sym_duplicate_key;
+static VALUE fj_sym_number_overflow;
 static ID    fj_bigdecimal_id; /* cached BigDecimal() method id (set in Init) */
 static ID    fj_to_sym_id;     /* cached :to_sym (symbolize_keys) */
 static ID    fj_key_p_id;      /* cached :key? (non-default duplicate_key modes) */
@@ -262,6 +263,8 @@ static inline int fj_needs_ws_skip(int b) {
 /* forward declarations (mutual recursion) */
 static VALUE fj_parse_value(fj_state *st);
 static VALUE fj_parse_member_value(fj_state *st);
+static int   fj_smart_quote_kind(fj_state *st);
+static VALUE fj_parse_smart_string(fj_state *st, int kind);
 
 static void fj_append_utf8(VALUE buf, unsigned long cp) {
   char tmp[4];
@@ -579,7 +582,8 @@ static VALUE fj_float_strtod(const char *p, long n) {
 }
 
 /* e10 is the final base-10 exponent (already adjusted by the fraction length). */
-static FJ_ALWAYS_INLINE VALUE fj_float_from_parts(uint64_t m10, int m10digits, int64_t e10, int neg, int overflow, const char *p, long n) {
+static FJ_ALWAYS_INLINE VALUE fj_float_from_parts(fj_state *st, uint64_t m10, int m10digits, int64_t e10, int neg, int overflow, const char *p, long n) {
+  double d;
   /* Fast path by mantissa width (our scanner accumulates m10 exactly up to 18
      digits, flagging overflow beyond):
        1..18 digits -> Eisel-Lemire, correctly-rounded for any exact uint64 mantissa
@@ -589,10 +593,18 @@ static FJ_ALWAYS_INLINE VALUE fj_float_from_parts(uint64_t m10, int m10digits, i
      >18 digits / overflow / extreme exponent -> strtod (round-to-odd). */
   if (!overflow && m10digits >= 1 && m10digits <= 18 && (long)m10digits + e10 >= -307) {
     if (m10 == 0) return rb_float_new(neg ? -0.0 : 0.0);
-    return rb_float_new(fj_eisel_lemire_s2d(e10, m10, neg));
+    d = fj_eisel_lemire_s2d(e10, m10, neg);
+  } else {
+    /* Fallback for >18 digits / extreme or subnormal exponents. */
+    d = RFLOAT_VALUE(fj_float_strtod(p, n));
   }
-  /* Fallback for >18 digits / extreme or subnormal exponents. */
-  return fj_float_strtod(p, n);
+  /* A finite literal whose magnitude exceeds Float range (e.g. 1e400) becomes
+     ±Infinity — a silent data change. Report it via :number_overflow (the value is
+     still returned). The Infinity/NaN keywords take separate paths and never get here.
+     Gate isinf on a listening handler (matches the Ruby float_or_warn): no handler ->
+     no point detecting, and it keeps the test off the hot number path. */
+  if (st->on_warning != Qnil && isinf(d)) fj_warn(st, fj_sym_number_overflow, "number literal out of Float range — collapsed to Infinity");
+  return rb_float_new(d);
 }
 
 /* Scan an already-bounded quoteless token [p, p+n) exactly once: validate it as a
@@ -677,7 +689,7 @@ static int fj_try_decimal(fj_state *st, const char *p, long n, VALUE *out) {
       (st->decimal_precision == 1 && m10digits > 16 && fj_sig_digits(p, n) > 16)) {
     *out = fj_to_bigdecimal_token(p, n);
   } else {
-    *out = fj_float_from_parts(m10, m10digits, e10, neg, overflow, p, n);
+    *out = fj_float_from_parts(st, m10, m10digits, e10, neg, overflow, p, n);
   }
   return 1;
 }
@@ -789,7 +801,7 @@ static VALUE fj_parse_number(fj_state *st) {
       (st->decimal_precision == 1 && m10digits > 16 && fj_sig_digits(np, nlen) > 16)) {
     return fj_to_bigdecimal_token(np, nlen);
   }
-  return fj_float_from_parts(m10, m10digits, e10, neg, overflow, np, nlen);
+  return fj_float_from_parts(st, m10, m10digits, e10, neg, overflow, np, nlen);
 }
 
 static VALUE fj_parse_literal(fj_state *st, const char *word, VALUE value) {
@@ -842,6 +854,7 @@ static VALUE fj_parse_identifier_key(fj_state *st) {
 
 static VALUE fj_parse_object_key(fj_state *st) {
   int b = fj_byte(st);
+  int kind;
 
   /* Quoted key. The common case has no escapes: intern straight from the buffer
    * with no throwaway allocation. An escaped key (rare) falls through to the
@@ -862,6 +875,12 @@ static VALUE fj_parse_object_key(fj_state *st) {
     return fj_parse_string(st, b);
   }
 
+  /* A key may open with a smart/curly quote too (a word-processor paste curls the
+   * keys, not just the values) — route to the same reader the value path uses.
+   * Mirrors the Ruby fallback's parse_object_key; Hash#[]= dedups the key on store. */
+  kind = fj_smart_quote_kind(st);
+  if (kind) return fj_parse_smart_string(st, kind);
+
   if (fj_is_key_start(b)) return fj_parse_identifier_key(st);
 
   fj_error(st, "expected a key");
@@ -1197,7 +1216,7 @@ static int fj_try_member_number(fj_state *st, VALUE *out) {
       (st->decimal_precision == 1 && m10digits > 16 && fj_sig_digits(np, nlen) > 16)) {
     *out = fj_to_bigdecimal_token(np, nlen);
   } else {
-    *out = fj_float_from_parts(m10, m10digits, e10, neg, overflow, np, nlen);
+    *out = fj_float_from_parts(st, m10, m10digits, e10, neg, overflow, np, nlen);
   }
   return 1;
 }
@@ -1625,6 +1644,7 @@ void Init_smarter_json(void) {
   fj_sym_empty_slot = ID2SYM(rb_intern("empty_slot"));
   fj_sym_empty_value = ID2SYM(rb_intern("empty_value"));
   fj_sym_duplicate_key = ID2SYM(rb_intern("duplicate_key"));
+  fj_sym_number_overflow = ID2SYM(rb_intern("number_overflow"));
   fj_sym_encoding = ID2SYM(rb_intern("encoding"));
   fj_sym_symbolize_keys = ID2SYM(rb_intern("symbolize_keys"));
   fj_sym_first_wins = ID2SYM(rb_intern("first_wins"));

data/lib/smarter_json/generator.rb

@@ -34,7 +34,17 @@ module SmarterJSON
     # (including multi-byte UTF-8) is emitted raw — valid JSON.
     ESCAPE_RE = /["\\\x00-\x1f]/.freeze
 
+    # Strict configuration: an unknown writer option is a caller bug, so it raises
+    # rather than being silently ignored.
+    KNOWN_OPTIONS = %i[format indent ascii_only script_safe sort_keys coerce allow_nan].freeze
+
     def initialize(options = {})
+      unknown = options.keys - KNOWN_OPTIONS
+      unless unknown.empty?
+        raise ArgumentError, "SmarterJSON.generate: unknown option#{unknown.size == 1 ? '' : 's'} " \
+                             "#{unknown.map(&:inspect).join(', ')} — valid keys: #{KNOWN_OPTIONS.map(&:inspect).join(', ')}"
+      end
+
       @format = options.fetch(:format, :json)
       unless %i[json ndjson].include?(@format)
         raise ArgumentError, "unknown writer format: #{@format.inspect} (expected :json or :ndjson)"
@@ -50,10 +60,11 @@ module SmarterJSON
 
       @pretty = @indent > 0
 
-      @ascii_only  = options.fetch(:ascii_only, false)  # escape non-ASCII as \uXXXX
-      @script_safe = options.fetch(:script_safe, false) # escape </ and U+2028 / U+2029
-      @sort_keys   = options.fetch(:sort_keys, false)   # emit object keys in sorted order
-      @coerce      = options.fetch(:coerce, false)      # convert unknown types via as_json / to_json
+      @ascii_only  = boolean_option(options, :ascii_only)  # escape non-ASCII as \uXXXX
+      @script_safe = boolean_option(options, :script_safe) # escape </ and U+2028 / U+2029
+      @sort_keys   = boolean_option(options, :sort_keys)   # emit object keys in sorted order
+      @coerce      = boolean_option(options, :coerce)      # convert unknown types via as_json / to_json
+      @allow_nan   = boolean_option(options, :allow_nan)   # emit NaN / Infinity / -Infinity (JSON5) instead of raising
       @escape_re   = build_escape_re
     end
 
@@ -77,7 +88,48 @@ module SmarterJSON
 
     private
 
-    def emit(obj, buf, level = 0)
+    # A boolean writer option must be exactly true or false — a wrong type is a
+    # caller bug, so it raises rather than being coerced or ignored.
+    def boolean_option(options, key)
+      value = options.fetch(key, false)
+      return value if value == true || value == false
+
+      raise ArgumentError, "#{key} must be true or false (got #{value.inspect})"
+    end
+
+    # Iterative serializer — an explicit frame stack (one frame per open container),
+    # mirroring the recursive structure but heap-allocated, so arbitrarily deep input
+    # cannot overflow the call stack (parity with the iterative parser). Output is
+    # byte-identical to the former recursive version. A frame is a small Array:
+    #   [members, idx, is_hash, before_first, before_rest, colon, closer, level]
+    def emit(obj, buf)
+      stack = []
+      push_value(obj, 0, buf, stack)
+      until stack.empty?
+        frame   = stack.last
+        members = frame[0]
+        i       = frame[1]
+        if i == members.length
+          buf << frame[6] # closer
+          stack.pop
+          next
+        end
+        frame[1] = i + 1
+        buf << (i.zero? ? frame[3] : frame[4]) # opener-pad / separator-pad
+        if frame[2] # hash
+          k, v = members[i]
+          emit_string(k.is_a?(String) ? k : k.to_s, buf) # Symbol/other keys -> string
+          buf << frame[5] # colon
+          push_value(v, frame[7] + 1, buf, stack)
+        else
+          push_value(members[i], frame[7] + 1, buf, stack)
+        end
+      end
+    end
+
+    # Emit one value at `level`: a scalar appends directly; a non-empty container writes
+    # its opener and pushes a frame for the driver above to walk (no recursion into it).
+    def push_value(obj, level, buf, stack)
       case obj
       when nil        then buf << "null"
       when true       then buf << "true"
@@ -87,22 +139,30 @@ module SmarterJSON
       when Integer    then buf << obj.to_s
       when Float      then emit_float(obj, buf)
       when BigDecimal then emit_bigdecimal(obj, buf)
-      when Array      then emit_array(obj, buf, level)
-      when Hash       then emit_hash(obj, buf, level)
+      when Array
+        return buf << "[]" if obj.empty? # empty stays inline, even in pretty mode
+
+        buf << (@pretty ? "[\n" : "[")
+        stack << container_frame(obj, false, level)
+      when Hash
+        return buf << "{}" if obj.empty? # empty stays inline, even in pretty mode
+
+        pairs = @sort_keys ? obj.sort_by { |k, _| k.is_a?(String) ? k : k.to_s } : obj.to_a
+        buf << (@pretty ? "{\n" : "{")
+        stack << container_frame(pairs, true, level)
       else
-        return emit_coerced(obj, buf, level) if @coerce
+        return push_coerced(obj, level, buf, stack) if @coerce
 
         raise SmarterJSON::GenerateError, "SmarterJSON.generate cannot serialize #{obj.class}"
       end
     end
 
-    # coerce: true — let a value that isn't natively supported convert itself.
-    # Prefer as_json (its result is re-emitted through the normal pipeline, so the
-    # escaping/format options still apply); fall back to to_json (spliced as-is, so
-    # ascii_only / script_safe do not reach inside it). Raise if it defines neither.
-    def emit_coerced(obj, buf, level)
+    # coerce: true — prefer as_json (re-emitted through the normal pipeline, so the
+    # escaping/format options still apply); else to_json (spliced as-is, so ascii_only /
+    # script_safe do not reach inside it); else raise.
+    def push_coerced(obj, level, buf, stack)
       if obj.respond_to?(:as_json)
-        emit(obj.as_json, buf, level)
+        push_value(obj.as_json, level, buf, stack)
       elsif obj.respond_to?(:to_json)
         buf << obj.to_json
       else
@@ -110,57 +170,16 @@ module SmarterJSON
       end
     end
 
-    def emit_array(arr, buf, level)
-      return buf << "[]" if arr.empty? # empty stays inline, even in pretty mode
-
-      if @pretty
-        pad = " " * (@indent * (level + 1))
-        buf << "[\n"
-        arr.each_with_index do |v, i|
-          buf << ",\n" unless i.zero?
-          buf << pad
-          emit(v, buf, level + 1)
-        end
-        buf << "\n" << (" " * (@indent * level)) << "]"
-      else
-        buf << "["
-        arr.each_with_index do |v, i|
-          buf << "," unless i.zero?
-          emit(v, buf, level)
-        end
-        buf << "]"
-      end
-    end
-
-    def emit_hash(hash, buf, level)
-      return buf << "{}" if hash.empty? # empty stays inline, even in pretty mode
-
-      pairs = @sort_keys ? hash.sort_by { |k, _| k.is_a?(String) ? k : k.to_s } : hash
-
+    # Build a frame for an open container at `level`, precomputing its punctuation/indent
+    # once (as the recursive version computed `pad` once per container).
+    def container_frame(members, is_hash, level)
+      close_glyph = is_hash ? "}" : "]"
       if @pretty
-        pad = " " * (@indent * (level + 1))
-        buf << "{\n"
-        first = true
-        pairs.each do |k, v|
-          buf << ",\n" unless first
-          first = false
-          buf << pad
-          emit_string(k.is_a?(String) ? k : k.to_s, buf) # Symbol/other keys -> string
-          buf << ": "
-          emit(v, buf, level + 1)
-        end
-        buf << "\n" << (" " * (@indent * level)) << "}"
+        pad  = " " * (@indent * (level + 1))
+        padl = " " * (@indent * level)
+        [members, 0, is_hash, pad, ",\n#{pad}", ": ", "\n#{padl}#{close_glyph}", level]
       else
-        buf << "{"
-        first = true
-        pairs.each do |k, v|
-          buf << "," unless first
-          first = false
-          emit_string(k.is_a?(String) ? k : k.to_s, buf) # Symbol/other keys -> string
-          buf << ":"
-          emit(v, buf, level)
-        end
-        buf << "}"
+        [members, 0, is_hash, "", ",", ":", close_glyph, level]
       end
     end
 
@@ -195,15 +214,31 @@ module SmarterJSON
     end
 
     def emit_float(flt, buf)
-      raise SmarterJSON::GenerateError, "SmarterJSON.generate cannot serialize non-finite Float #{flt}" unless flt.finite?
+      unless flt.finite?
+        raise SmarterJSON::GenerateError, "SmarterJSON.generate cannot serialize non-finite Float #{flt}" unless @allow_nan
+
+        return buf << non_finite_literal(flt)
+      end
 
       buf << flt.to_s # Ruby's Float#to_s is shortest round-trippable; e-notation is valid JSON
     end
 
     def emit_bigdecimal(num, buf)
-      raise SmarterJSON::GenerateError, "SmarterJSON.generate cannot serialize non-finite BigDecimal" unless num.finite?
+      unless num.finite?
+        raise SmarterJSON::GenerateError, "SmarterJSON.generate cannot serialize non-finite BigDecimal" unless @allow_nan
+
+        return buf << non_finite_literal(num)
+      end
 
       buf << num.to_s("F") # plain decimal notation (BigDecimal's default "0.1e1" is not valid JSON)
     end
+
+    # JSON5-style literals for non-finite numbers, emitted only when allow_nan: true.
+    # `infinite?` returns 1 / -1 / nil for both Float and BigDecimal.
+    def non_finite_literal(num)
+      return "NaN" if num.nan?
+
+      num.infinite? == 1 ? "Infinity" : "-Infinity"
+    end
   end
 end

data/lib/smarter_json/options.rb

@@ -7,7 +7,7 @@ module SmarterJSON
   module Options
     DEFAULT_OPTIONS = {
       acceleration: true,        # use the C extension when available; false forces pure Ruby
-      encoding: nil,             # label the input's encoding (no transcoding); nil keeps the input's own
+      encoding: nil,             # label the input's encoding (no transcoding); nil keeps the input's own (valid-UTF-8 ASCII-8BIT → UTF-8)
       symbolize_keys: false,     # Symbol keys instead of String
       duplicate_key: :last_wins, # :last_wins | :first_wins  (repeats are also reported via on_warning)
       decimal_precision: :auto,  # :auto | :float | :bigdecimal  (Oj-compatible decimal handling)
@@ -24,17 +24,30 @@ module SmarterJSON
     end
 
     # Raise ArgumentError (consistent with the generator's option checks) listing
-    # every invalid setting at once. Unknown keys are ignored, matching the lenient
-    # design — an option SmarterJSON doesn't recognize simply has no effect.
+    # every problem at once. Configuration is strict — unlike the lenient *data*
+    # handling, an unknown option key or a bad value raises, so a caller's typo or
+    # wrong type is caught immediately instead of silently having no effect.
     def validate_options!(options)
       errors = []
 
+      unknown = options.keys - DEFAULT_OPTIONS.keys
+      unless unknown.empty?
+        errors << "unknown option#{unknown.size == 1 ? '' : 's'} #{unknown.map(&:inspect).join(', ')} " \
+                  "— valid keys: #{DEFAULT_OPTIONS.keys.map(&:inspect).join(', ')}"
+      end
+
       unless %i[auto float bigdecimal].include?(options[:decimal_precision])
         errors << "decimal_precision must be :auto, :float, or :bigdecimal (got #{options[:decimal_precision].inspect})"
       end
       unless %i[last_wins first_wins].include?(options[:duplicate_key])
         errors << "duplicate_key must be :last_wins or :first_wins (got #{options[:duplicate_key].inspect})"
       end
+      unless [true, false].include?(options[:acceleration])
+        errors << "acceleration must be true or false (got #{options[:acceleration].inspect})"
+      end
+      unless [true, false].include?(options[:symbolize_keys])
+        errors << "symbolize_keys must be true or false (got #{options[:symbolize_keys].inspect})"
+      end
       on_warning = options[:on_warning]
       unless on_warning.nil? || on_warning.respond_to?(:call)
         errors << "on_warning must be nil or a callable (got #{on_warning.class})"

data/lib/smarter_json/parser.rb

@@ -109,6 +109,25 @@ module SmarterJSON
     end
   end
 
+  # Smart default for the nil :encoding option. A String tagged ASCII-8BIT (BINARY)
+  # is how Net::HTTP and many HTTP libraries hand back a response body even when the
+  # bytes are UTF-8. JSON's interchange encoding is UTF-8, so we relabel such input
+  # to UTF-8 when its bytes are valid UTF-8 — otherwise string values would come back
+  # tagged ASCII-8BIT and compare unequal to UTF-8 literals (a silent footgun). When
+  # the bytes are NOT valid UTF-8 we raise EncodingError rather than guess a legacy
+  # encoding — pass an explicit :encoding for that. An explicit (non-nil) :encoding,
+  # or any non-BINARY tag, is left untouched (the per-path force_encoding / validation
+  # handles it). Only relabels — never transcodes.
+  def normalize_default_encoding(input, options)
+    return input unless options[:encoding].nil?
+    return input unless input.encoding == Encoding::ASCII_8BIT
+
+    utf8 = input.dup.force_encoding(Encoding::UTF_8)
+    return utf8 if utf8.valid_encoding?
+
+    raise EncodingError, "input is tagged ASCII-8BIT and is not valid UTF-8 — pass encoding: to declare its encoding"
+  end
+
   # Stream documents from an IO incrementally, yielding each recovered top-level
   # document without slurping the whole input into memory first.
   def stream_io(io, options, &block)
@@ -411,6 +430,7 @@ module SmarterJSON
     module_function
 
     def process_string(input, options, &block)
+      input = SmarterJSON.send(:normalize_default_encoding, input, options)
       return SmarterJSON.send(:process_content, input, options, &block) unless input.valid_encoding?
 
       # Recovery is REACTIVE: parse first, and only fall back to wrapper extraction when
@@ -1220,6 +1240,12 @@ module SmarterJSON
       b = byte
       return parse_string(DQUOTE) if b == DQUOTE
       return parse_string(SQUOTE) if b == SQUOTE
+
+      # A key may open with a smart/curly quote too (word-processor paste curls keys,
+      # not just values) — route to the same reader values already use.
+      kind = smart_quote_kind(@pos)
+      return parse_smart_string(kind) if kind
+
       raise error("expected a key") unless b && key_start_byte?(b)
 
       parse_identifier_key
@@ -1365,12 +1391,25 @@ module SmarterJSON
     # than 16 significant digits (Oj's DEC_MAX threshold), else Float.
     def decimal_value(body)
       case @decimal_precision
-      when :float      then body.to_f
+      when :float      then float_or_warn(body)
       when :bigdecimal then to_big_decimal(body)
-      else                  significant_digits(body) > 16 ? to_big_decimal(body) : body.to_f
+      else                  significant_digits(body) > 16 ? to_big_decimal(body) : float_or_warn(body)
       end
     end
 
+    # A finite numeric literal whose magnitude exceeds Float range (e.g. 1e400) becomes
+    # ±Infinity — a silent data change. Report it via :number_overflow (the value is still
+    # returned; we warn rather than raise or invent). The Infinity/NaN *keywords* go through
+    # a separate path and never reach here, so they don't warn.
+    def float_or_warn(body)
+      f = body.to_f
+      # Only test for overflow when an on_warning handler is listening: `f.infinite?` is a
+      # per-float method call we don't want on the hot number path otherwise, and with no
+      # handler the warning would go nowhere anyway. Overflow is vanishingly rare.
+      warn(:number_overflow, "number literal out of Float range — collapsed to #{f}") if @on_warning && f.infinite?
+      f
+    end
+
     # Count significant mantissa digits (leading zeros excluded, exponent ignored) to pick
     # Float vs BigDecimal in :auto mode. A single byte-scan — the old three-regex version
     # (strip exponent, strip non-digits, strip leading zeros, .length) ran on every float

data/lib/smarter_json/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module SmarterJSON
-  VERSION = "0.9.9"
+  VERSION = "1.0.0"
 end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: smarter_json
 version: !ruby/object:Gem::Version
-  version: 0.9.9
+  version: 1.0.0
 platform: ruby
 authors:
 - Tilo Sloboda
 bindir: exe
 cert_chain: []
-date: 2026-06-07 00:00:00.000000000 Z
+date: 2026-06-09 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bigdecimal
@@ -23,10 +23,16 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-description: 'SmarterJSON is a permissive JSON/JSON5 parser: comments, trailing commas,
-  different quote styles, Python/JS keywords, and more, all parse to the same Ruby
-  objects. Purposely no strict mode, always best-effort, blazing fast. Handles BOM,
-  smart quotes, messy input. Compatible with config/data files and API responses alike.
+description: 'A lenient, fast JSON processor for Ruby. It extracts strict JSON, NDJSON,
+  JSON5, HJSON-style config, and the messy JSON-ish input humans and LLMs actually
+  write — comments, trailing commas, single / unquoted / smart quotes, Python and
+  JS keywords, a UTF-8 BOM, and more all parse to the same Ruby objects, with no modes
+  or flags to set. Where a traditional parser stops at the first deviation and throws
+  away the whole document, SmarterJSON keeps going — it optimizes for getting your
+  data out, not for policing the JSON spec. It reads multi-document NDJSON / JSONL
+  in one call (and streams it with a block), and in benchmarks its C extension matches
+  or beats Oj on nearly every file. SmarterJSON is opinionated: we want your JSON
+  processing to be successful.
 
   '
 email:
@@ -86,6 +92,6 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubygems_version: 3.6.9
 specification_version: 4
-summary: 'SmarterJSON: A lenient, robust, streaming JSON parser for Ruby supporting
-  JSON, JSON5, NDJSON, and HJSON-style input.'
+summary: A lenient, fast JSON processor for Ruby — reads strict JSON, NDJSON, JSON5,
+  HJSON, and the messy JSON humans and LLMs actually write.
 test_files: []