smarter_json 0.5.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: cbc1f54c56cf5a1fd4c569660faf9f4115e5e7ed8442ac9e8e7105bc880a3912
+  data.tar.gz: 5e68d7b1dafa55347cf5de1ee10e8ac39a97b645996fd0c58538799bbbb1191d
+SHA512:
+  metadata.gz: 0e84fb4caf1fc9b192aa0e88f5111c3178557078abd8a31e7ce73373590e2487344c8df9ad61e718756e78a15239b460fd05b5f7a6fdbede35d8859f1873f5be
+  data.tar.gz: 9d808b8b3e8465ce7b12ae861053e67a77ef1550df425c6383ffe4b799ba401a6f927f692b09709942fdb9e690a1c9d25a5f4453d57b9b532e18f979d171199d

data/.gitignore

@@ -0,0 +1,46 @@
+# General ignores for a Ruby gem
+*.gem
+
+# Ignore Bundler lockfile for gems (leave out for gems, include for apps)
+Gemfile.lock
+
+# Ignore RSpec status persistence
+.rspec_status
+
+# Ignore build output
+/pkg/
+/tmp/
+/log/
+/coverage/
+
+# Ignore RuboCop, Yard, docs, bundler
+/.bundle/
+.yardoc
+/.yardoc/
+/.byebug_history
+/.pry_history
+/.irb-history
+
+# Ignore Mac, editor, IDE artifacts
+.DS_Store
+*~
+*.swp
+*.swo
+*.tmp
+.vscode/
+.idea/
+
+# Ignore node_modules just in case
+node_modules/
+
+# Ignore test coverage output
+overage/
+
+# Ignore binary object files, extensions
+*.so
+*.o
+*.bundle
+*.rbc
+
+.claude/
+CLAUDE.md

data/CHANGELOG.md

@@ -0,0 +1,70 @@
+
+# SmarterJSON Change Log
+
+## 0.5.1 (2026-06-01)
+- Unified the error classes under a single `SmarterJSON::Error` base: `ParseError` and `EncodingError` now inherit from it, and `generate` raises a new `GenerateError`. `rescue SmarterJSON::Error` now catches everything the gem raises.
+- Added a CI test matrix (Ruby 2.6–4.0 + head, on Ubuntu and macOS).
+- Fixed the C extension build on Ruby 2.6 (declare `rb_hash_bulk_insert`, which 2.6 exports but does not declare in its headers); set the minimum Ruby to 2.6.
+
+## 0.5.0 (2026-05-31 unreleased)
+- add JSON generation, incl. NDJSON generation
+- add test coverage
+
+## 0.4.0 (2026-05-31 unreleased)
+- rename `flex_json` -> `smarter_json`
+
+## 0.3.10 (2026-05-31 unreleased)
+- change interface to use `.process` and `.process_file`
+
+
+## 0.3.9 (2026-05-31 unreleased)
+- `parse` (no block) now handles any input automatically: 0 documents (empty / whitespace / comment-only) → `nil`, 1 document → the value itself, 2+ documents (NDJSON / JSONL / concatenated / whitespace-separated) → an Array of the values. It no longer raises on trailing content.
+- Detection is free (the same trailing-content check that used to raise) and the single-document path allocates no Array, so single-value parsing is unchanged in speed.
+- The block form (`parse(input) { |doc| … }`) is kept as the bounded-memory streaming path. `parse_file(path) { |doc| … }` now forwards the block too, so files stream the same way (previously the block was silently ignored). Bracketless comma lists (`1, 2, 3`) still raise — commas don't separate top-level documents (implicit-root array remains unsupported).
+- The block form allows individual processing of each line in NDJSON files.
+- Supersedes the earlier "raise on trailing content, match Oj" behavior.
+
+## 0.3.8 (2026-05-30 unreleased)
+- Reordered single-character checks so the more common byte is tested first (`-` before `+`).
+- Quoteless-token boundary scan now uses a 256-byte class table: ordinary bytes are classified in one table lookup, and the lookahead byte is read only at a `#`/`/` instead of on every byte. Speeds up quoteless / config-style input (the lenient case the JSON benchmarks don't exercise).
+
+## 0.3.7 (2026-05-30 unreleased)
+- Escaped-string literal runs are bulk-copied with the NEON scanner instead of one byte at a time.
+- Added branch hints (`__builtin_expect`) and prefetch to the hot string-scan loop. Sped up string-heavy files (string_array, github_events, twitter all 12–16% faster).
+
+## 0.3.6 (2026-05-30 unreleased)
+- Fast path for plain numbers inside objects/arrays (`fj_try_member_number`): one scan straight from the cursor, committing when the number meets a delimiter and falling back to the quoteless scanner otherwise. Skips the quoteless boundary scan + classify dispatch for the common case. Broad gains on number-in-container files (weather, canada, usgs, big_decimals).
+
+## 0.3.5 (2026-05-30 unreleased)
+- Rewrote `fj_parse_number` (top-level numbers) as a single pass: finds the token end and accumulates the mantissa/exponent at once, using the string's NUL terminator as a scan sentinel (no per-byte bounds check) and a digit loop that skips the underscore check until an underscore actually appears.
+- Added `fj_try_decimal` for the quoteless path: validates and extracts the number in one scan, replacing the old three scans (validate + significant-digit count + mantissa extraction); skips the significant-digit scan when the number has ≤16 digits.
+- Both number paths now build values through the shared `fj_int_from_parts` / `fj_float_from_parts` helpers so they can't drift; removed the now-dead `fj_validate_decimal` / `fj_int_value` / `fj_decimal_value`.
+
+## 0.3.4 (2026-05-30 unreleased)
+- Dropped a per-member Ruby method call (`key?`) that fired for every object member under the default duplicate-key mode — pure waste on object-heavy files (twitter, github_events, citm).
+- Build objects and arrays from a C value stack with a pre-sized hash + bulk insert (and size-based duplicate detection), instead of inserting one member/element at a time.
+- Added a per-parse key cache so repeated object keys are interned once instead of every occurrence.
+
+## 0.3.3 (2026-05-30 unreleased)
+- Vendored Ryū (Ulf Adams, Apache-2.0) for correctly-rounded string→double conversion: the mantissa is accumulated in one pass and converted with no `strtod`. Large win on float-heavy files (canada, big_decimals).
+
+## 0.3.3 (2026-05-29 unreleased)
+- performance fixes
+
+## 0.3.2 (2026-05-29 unreleased)
+- performance fixes
+
+## 0.3.1 (2026-05-29 unreleased)
+- performance fixes
+
+## 0.3.0 (2026-05-29 unreleased)
+- iterative parser
+
+## 0.2.0 (2026-05-29 unreleased)
+- recursive parser
+
+## 0.1.1 (2026-05-29 unreleased)
+- MVP complete
+
+## 0.1.0 (2026-05-28 unreleased)
+- Initial Ruby version

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Tilo Sloboda
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,110 @@
+# SmarterJSON
+
+![Gem Version](https://img.shields.io/gem/v/smarter_json) [![codecov](https://codecov.io/gh/tilo/smarter_json/branch/main/graph/badge.svg)](https://codecov.io/gh/tilo/smarter_json) [![Downloads](https://img.shields.io/gem/dt/smarter_json)](https://rubygems.org/gems/smarter_json) [![RubyGems](https://img.shields.io/badge/RubyGems-smarter__json-brightgreen?logo=rubygems&logoColor=white)](https://rubygems.org/gems/smarter_json) [![Ruby Toolbox](https://img.shields.io/badge/Ruby%20Toolbox-smarter__json-brightgreen)](https://www.ruby-toolbox.com/projects/smarter_json)
+
+A lenient, fast JSON parser for Ruby. It parses strict JSON, JSON5, HJSON-style config, and the messy JSON-ish input humans actually write — and in benchmarks it matches or beats Oj on nearly every file. SmarterJSON is opinionated: we want your JSON processing to be successful. Other parsers are strict - they stop at the first deviation - SmarterJSON keeps going - it optimizes for getting your data out, not for policing the JSON spec.
+
+## Why SmarterJSON?
+
+Most JSON parsers reject anything that isn't perfectly strict JSON. SmarterJSON is built on the opposite principle: **you shouldn't have to care what flavor of JSON you were handed.** Give it strict JSON, JSON5, an HJSON-style config file, newline-delimited JSON, or a copy-pasted blob with comments and trailing commas — it just parses it.
+
+Three things set it apart:
+
+1. **One parser, no modes, no flags.** There is no `dialect:` option and no "strict mode" — `SmarterJSON.process(input)` accepts the whole superset, and strict JSON is simply the narrowest case. You don't configure the parser to match your input; it adapts to whatever you give it.
+
+2. **It parses multi-document input automatically — a distinguishing feature.** `SmarterJSON.process` handles NDJSON / JSONL / concatenated JSON with **no block and no special method**: one document returns its value, several documents return an `Array`, empty input returns `nil`. **Only SmarterJSON parses multi-document input via plain `process` — Oj and the stdlib `json` library raise without a block.** For input larger than memory, pass a block to stream one document at a time.
+
+3. **It's fast.** A C extension (with a pure-Ruby fallback that runs everywhere) puts it ahead of Oj on nearly every file we benchmark, and competitive with the stdlib `json` C parser — the fastest general-purpose Ruby JSON parser.
+
+## What it accepts, beyond strict JSON
+
+- `//`, `/* … */`, and `#` comments (a `#`/`//` only starts a comment when preceded by whitespace, so `url: http://x.com` parses as a string, not a truncated value)
+- Trailing commas; unquoted keys (`{host: localhost}`); single-quoted, triple-quoted (`'''…'''`), and quoteless string values
+- Implicit root object — a config file that starts with `key: value`, no outer `{}`
+- `NaN`, `Infinity`, hex (`0xFF`), leading `+` / `.`, underscores in numbers (`1_000_000`)
+- UTF-8 BOM, smart/curly quotes, Python literals (`True` / `False` / `None`), JavaScript `undefined`
+- Mixed CR / LF / CRLF line endings, and any Ruby-supported input encoding (via `encoding:`)
+- Duplicate keys (last value wins by default; configurable)
+
+It raises only on genuinely unparseable input (unterminated string, mismatched bracket), with line and column in the message — never on valid-but-lenient input.
+
+## Installation
+
+```ruby
+# Gemfile
+gem "smarter_json"
+```
+
+```bash
+gem install smarter_json
+```
+
+The C extension is built on install and used automatically. On platforms where it can't build, the pure-Ruby parser runs instead and produces identical results.
+
+## Documentation
+
+  * [Introduction](docs/_introduction.md)
+  * [The Basic Read API](docs/basic_read_api.md)
+  * [The Basic Write API](docs/basic_write_api.md)
+  * [Configuration Options](docs/options.md)
+  * [Examples](docs/examples.md)
+
+## Usage
+
+```ruby
+require "smarter_json"
+
+SmarterJSON.process('{"a": 1, "b": [2, 3]}')          # => {"a"=>1, "b"=>[2, 3]}
+SmarterJSON.process("host: localhost\nport: 5432")     # => {"host"=>"localhost", "port"=>5432}  (no braces needed)
+SmarterJSON.process_file("config.json5")               # read a file, then parse
+
+# Multiple documents (NDJSON / JSONL / concatenated) — no block, no special method:
+SmarterJSON.process(%({"id":1}\n{"id":2}\n{"id":3}))   # => [{"id"=>1}, {"id"=>2}, {"id"=>3}]
+SmarterJSON.process('{"id":1}')                         # => {"id"=>1}   (one document → the value itself)
+SmarterJSON.process("")                                 # => nil          (zero documents)
+
+# For input larger than memory, stream one document at a time with a block
+# (process and process_file both forward the block):
+SmarterJSON.process_file("events.ndjson") { |event| EventJob.perform_async(event) }
+```
+
+### Options
+
+| option            | default      | meaning                                                                 |
+|-------------------|--------------|-------------------------------------------------------------------------|
+| `symbolize_keys`  | `false`      | return object keys as Symbols instead of Strings                        |
+| `duplicate_key`   | `:last_wins` | `:last_wins` / `:first_wins` / `:raise` for repeated keys in one object |
+| `bigdecimal_load` | `:auto`      | `:auto` keeps high-precision decimals as `BigDecimal`; `:float` forces `Float`; `:bigdecimal` forces `BigDecimal` |
+| `acceleration`    | `true`       | `true` uses the C extension when compiled and loadable; `false` forces pure Ruby (identical results) |
+| `encoding`        | `"UTF-8"`    | labels the input's encoding (no transcoding pass; see below)            |
+
+## Performance
+
+Benchmarks: p10 of 40 runs, Apple M1 Max, Ruby 3.4.7, on the standard JSON corpus (canada, citm_catalog, twitter, github_events, …). The apples-to-apples comparisons are **SmarterJSON/C** vs **Oj/strict** vs **stdlib `json`**, all producing `Float` (run `rake report` in `json_benchmarks/` for the full table — numbers vary run to run).
+
+- **vs Oj:** SmarterJSON/C matches or beats Oj on nearly every file — typically **1.1–1.7× faster** (e.g. deeply-nested ~1.7×, citm ~1.3×, twitter ~1.3×, usgs/weather ~1.2–1.3×).
+- **vs stdlib `json` (C):** competitive with the fastest Ruby JSON parser — it matches `json` on number- and string-heavy files (e.g. big_decimals, string_array) and trails by ~1.2–1.6× on others.
+- **Numbers:** floats are parsed with Ryū (correctly rounded, single-pass), so number-heavy data is fast and bit-exact.
+
+**Two notes on fair comparison:**
+
+- **NDJSON:** on multi-document files, **only SmarterJSON parses the input via plain `process`** — Oj and `json` raise without a block, so their cells are `N/A`. That `N/A` reflects real default behavior, not a measurement gap. Plain `process` collects every document into an Array at ~270 MB/s; the streaming block form runs faster (~440 MB/s) because it doesn't hold all documents in memory at once — use it for input larger than RAM.
+- **High-precision decimals (e.g. `canada.json`):** SmarterJSON's default `:auto` mode preserves high-precision numbers as `BigDecimal` (matching Oj's default), which is intrinsically slower than `Float`. Against `Float`-producing parsers it looks slower on such files; pass `bigdecimal_load: :float` to compare like-for-like (it then runs much faster). Against the equivalent `BigDecimal`-producing Oj mode, SmarterJSON is faster.
+
+## Encoding
+
+`encoding:` (default `"UTF-8"`) labels what the input is — it does **not** trigger a transcoding pass. The parser works on the bytes in their native encoding and emits string values with the same encoding tag, the same way `smarter_csv` handles encodings. Bytes that are invalid for the claimed encoding raise `SmarterJSON::EncodingError` (a kind of `SmarterJSON::ParseError`).
+
+## Nesting & untrusted input
+
+Both the C extension and the pure-Ruby parser are **iterative, not recursive** — they track nesting on an explicit, heap-allocated stack rather than the call stack. So deeply nested input **cannot overflow the call stack or segfault**: nesting is bounded only by available memory, the same posture as Oj (which also ships no nesting limit; the stdlib `json` caps at 100). The `deeply_nested.json` benchmark (212 MB of nesting) parses without issue.
+
+The trade-off: there is currently **no fixed nesting or input-size limit**, so extremely large or adversarially-nested untrusted input is bounded by memory (it can exhaust RAM), not by a crash. If you parse untrusted input and want a hard cap, that's a planned opt-in guard — for now, size-limit upstream of the parser.
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies, then `rake compile` to build the C extension and `rake spec` to run the tests. The test suite runs every example against **both** the C and pure-Ruby paths, so the two stay behavior-identical.
+
+## License
+
+Available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/Rakefile

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+require "rubocop/rake_task"
+
+RuboCop::RakeTask.new
+
+require "rake/extensiontask"
+Rake::ExtensionTask.new("smarter_json") do |ext|
+  ext.ext_dir = "ext/smarter_json"
+  ext.lib_dir = "lib/smarter_json"
+end
+
+task spec: :compile
+# rubocop is NOT in the default task: `bundle exec rake` = build + test only, so it
+# runs on every Ruby in the CI matrix (incl. 2.5–2.7, where the latest rubocop won't
+# install). Lint runs as its own CI step on one modern Ruby. Locally: `rake rubocop`.
+task default: %i[clobber compile spec]

data/docs/_introduction.md

@@ -0,0 +1,48 @@
+
+### Contents
+
+  * [**Introduction**](./_introduction.md)
+  * [The Basic Read API](./basic_read_api.md)
+  * [The Basic Write API](./basic_write_api.md)
+  * [Configuration Options](./options.md)
+  * [Examples](./examples.md)
+
+--------------
+
+# SmarterJSON Introduction
+
+`smarter_json` is a fast, lenient JSON parser and writer for Ruby. It reads strict JSON, JSON5, HJSON-style config, newline-delimited JSON (NDJSON / JSONL), and the messy JSON-ish input humans actually paste — and in benchmarks it matches or beats Oj on nearly every file. It is opinionated: it optimizes for getting your data out, not for policing the JSON spec. Where other parsers stop at the first deviation, SmarterJSON keeps going.
+
+## Why another JSON library?
+
+Most JSON parsers reject anything that isn't perfectly strict JSON, and they make you tell them up front what shape the input is. SmarterJSON is built on the opposite principle: **you shouldn't have to care what flavor of JSON you were handed.** Give it strict JSON, JSON5, an HJSON-style config file, several concatenated documents, or a copy-pasted blob with comments and trailing commas — it just reads it.
+
+## What sets it apart
+
+* **One reader, no modes, no flags.** There is no `dialect:` option and no "strict mode" — `SmarterJSON.process(input)` accepts the whole superset, and strict JSON is simply the narrowest case. You don't configure the reader to match your input; it adapts to whatever you give it.
+
+* **It reads multi-document input automatically — a distinguishing feature.** `SmarterJSON.process` handles NDJSON / JSONL / concatenated JSON with **no block and no special method**: zero documents returns `nil`, one document returns its value, two or more return an `Array`. **Only SmarterJSON reads multi-document input via plain `process` — Oj and the stdlib `json` library raise without a block.** For input larger than memory, pass a block to stream one document at a time. See [The Basic Read API](./basic_read_api.md).
+
+* **It's fast.** A C extension (with a pure-Ruby fallback that runs everywhere) puts it ahead of Oj on nearly every file we benchmark, and competitive with the stdlib `json` C parser. Floats are parsed with Ryū (correctly rounded, single-pass), so number-heavy data is fast and bit-exact.
+
+* **It writes JSON too.** `SmarterJSON.generate` turns Ruby values into strict, interoperable JSON — or into NDJSON, one element per line, the exact inverse of reading NDJSON back into an Array. See [The Basic Write API](./basic_write_api.md).
+
+## What it accepts, beyond strict JSON
+
+* `//`, `/* … */`, and `#` comments (a `#`/`//` only starts a comment when preceded by whitespace, so `url: http://x.com` reads as a string, not a truncated value)
+* Trailing commas; unquoted keys (`{host: localhost}`); single-quoted, triple-quoted (`'''…'''`), and quoteless string values
+* Implicit root object — a config file that starts with `key: value`, no outer `{}`
+* `NaN`, `Infinity`, hex (`0xFF`), leading `+` / `.`, underscores in numbers (`1_000_000`)
+* UTF-8 BOM, smart/curly quotes, Python literals (`True` / `False` / `None`), JavaScript `undefined`
+* Mixed CR / LF / CRLF line endings, and any Ruby-supported input encoding (via `encoding:`)
+* Duplicate keys (last value wins by default; configurable — see [Configuration Options](./options.md))
+
+It raises only on genuinely unparseable input (unterminated string, mismatched bracket), with line and column in the message — never on valid-but-lenient input.
+
+## Nesting & untrusted input
+
+Both the C extension and the pure-Ruby parser are **iterative, not recursive** — they track nesting on an explicit, heap-allocated stack rather than the call stack. So deeply nested input **cannot overflow the call stack or segfault**: nesting is bounded only by available memory, the same posture as Oj (the stdlib `json` caps at 100). The trade-off: there is currently **no fixed nesting or input-size limit**, so size-limit untrusted input upstream of the parser.
+
+---------------
+
+NEXT: [The Basic Read API](./basic_read_api.md) | UP: [README](../README.md)

data/docs/basic_read_api.md

@@ -0,0 +1,72 @@
+
+### Contents
+
+  * [Introduction](./_introduction.md)
+  * [**The Basic Read API**](./basic_read_api.md)
+  * [The Basic Write API](./basic_write_api.md)
+  * [Configuration Options](./options.md)
+  * [Examples](./examples.md)
+
+--------------
+
+# SmarterJSON Basic Read API
+
+Reading JSON has one entry point for content and one for files. Both accept the same [options](./options.md), and both take an optional block for streaming.
+
+## `SmarterJSON.process` — read a String or an IO
+
+```ruby
+require "smarter_json"
+
+SmarterJSON.process('{"a": 1, "b": [2, 3]}')          # => {"a"=>1, "b"=>[2, 3]}
+SmarterJSON.process("host: localhost\nport: 5432")     # => {"host"=>"localhost", "port"=>5432}  (no braces needed)
+```
+
+`process` is polymorphic: its first argument is **either a String of JSON content or an IO to read from**. A String is always treated as content, never as a filename — use `process_file` for paths.
+
+```ruby
+SmarterJSON.process(io)         # an open IO (File, StringIO, socket, …) — reads it and parses
+SmarterJSON.process(some_string) # JSON content
+```
+
+### Return value depends on how many documents the input holds
+
+This is the distinguishing feature: `process` reads multi-document input (NDJSON / JSONL / concatenated / whitespace-separated) automatically, with no block and no special method.
+
+```ruby
+SmarterJSON.process("")                                 # => nil          (zero documents)
+SmarterJSON.process('{"id":1}')                          # => {"id"=>1}    (one document → the value itself)
+SmarterJSON.process(%({"id":1}\n{"id":2}\n{"id":3}))     # => [{"id"=>1}, {"id"=>2}, {"id"=>3}]  (two or more → an Array)
+```
+
+Documents are separated by whitespace, newlines, or simple concatenation — **not** by commas (a comma between top-level documents would be read as an implicit root array, which is not supported). Only SmarterJSON reads this via plain `process`: Oj and the stdlib `json` library raise without a block.
+
+## `SmarterJSON.process_file` — read a file by path
+
+```ruby
+SmarterJSON.process_file("config.json5")     # read the file, then parse — same return-value rules as process
+```
+
+`process_file` opens the file, reads it with the labeled [`encoding:`](./options.md) (default `"UTF-8"`, no transcoding pass), and parses it.
+
+## Streaming with a block (bounded memory)
+
+For input larger than memory, pass a block. Each top-level document is yielded as it is read, and the method returns `nil` (it never collects the documents into an Array). Both `process` and `process_file` forward the block.
+
+```ruby
+# Stream straight from disk, one document at a time — the whole file is never loaded:
+SmarterJSON.process_file("events.ndjson") { |event| EventJob.perform_async(event) }
+
+# Same for an IO:
+SmarterJSON.process(io) { |doc| handle(doc) }
+```
+
+The streaming path reads the input as newline-delimited documents (NDJSON / JSONL), one document per line. A single document that spans multiple lines is not supported by the streaming path — read it without a block instead.
+
+## The C extension and the pure-Ruby fallback
+
+By default (`acceleration: :auto`) the C extension is used when it is compiled and loadable (`SmarterJSON::HAS_ACCELERATION` is then `true`); otherwise the pure-Ruby parser runs and produces identical results. Pass `acceleration: false` to force the pure-Ruby path. See [Configuration Options](./options.md).
+
+---------------
+
+PREVIOUS: [Introduction](./_introduction.md) | NEXT: [The Basic Write API](./basic_write_api.md) | UP: [README](../README.md)

data/docs/basic_write_api.md

@@ -0,0 +1,91 @@
+
+### Contents
+
+  * [Introduction](./_introduction.md)
+  * [The Basic Read API](./basic_read_api.md)
+  * [**The Basic Write API**](./basic_write_api.md)
+  * [Configuration Options](./options.md)
+  * [Examples](./examples.md)
+
+--------------
+
+# SmarterJSON Basic Write API
+
+Writing JSON has one entry point: `SmarterJSON.generate`. It turns a Ruby value into a JSON String — strict, interoperable output by default, or NDJSON when you ask for it.
+
+## `SmarterJSON.generate` — write a Ruby value as JSON
+
+```ruby
+require "smarter_json"
+
+SmarterJSON.generate({ "a" => 1, "b" => [2, 3] })   # => '{"a":1,"b":[2,3]}'
+SmarterJSON.generate([1, 2, 3])                       # => '[1,2,3]'
+SmarterJSON.generate("hi")                            # => '"hi"'
+SmarterJSON.generate(42)                              # => '42'
+SmarterJSON.generate(nil)                             # => 'null'
+```
+
+The output is always **valid, strict JSON** — there is no lenient write mode. (We are lenient about what we *read*, strict about what we *write*, so the output interoperates with every other JSON parser.)
+
+## How Ruby values map to JSON
+
+| Ruby                                   | JSON output                                             |
+|----------------------------------------|---------------------------------------------------------|
+| `Hash`                                 | object `{…}` — keys are stringified (Symbol keys too)   |
+| `Array`                                | array `[…]`                                             |
+| `String`                               | quoted string, escaped (see below)                      |
+| `Symbol`                               | quoted string (`:sym` → `"sym"`)                        |
+| `Integer`                              | number                                                  |
+| `Float`                                | number (non-finite raises — see below)                  |
+| `BigDecimal`                           | number, full precision (not a string)                   |
+| `true` / `false` / `nil`               | `true` / `false` / `null`                               |
+
+```ruby
+SmarterJSON.generate({ a: 1, b: :sym })                       # => '{"a":1,"b":"sym"}'   (Symbol key and value → strings)
+SmarterJSON.generate(BigDecimal("65.613616999999977"))         # => '65.613616999999977' (a number, full precision)
+SmarterJSON.generate("café\tx")                                # => '"café\tx"'          (control chars escaped, UTF-8 raw)
+```
+
+Strings escape `"`, `\`, and the control characters `0x00–0x1F`; everything else — including multi-byte UTF-8 — is emitted raw, which is valid JSON.
+
+## What raises
+
+`generate` raises `SmarterJSON::Error` on input it cannot represent as strict JSON:
+
+```ruby
+SmarterJSON.generate(Time.now)          # raises SmarterJSON::Error — unsupported type
+SmarterJSON.generate(Float::INFINITY)   # raises SmarterJSON::Error — non-finite Float
+SmarterJSON.generate(Float::NAN)        # raises SmarterJSON::Error — non-finite Float
+```
+
+(`Infinity` and `NaN` are accepted on the *read* side as a leniency, but they are not valid JSON to *write*.)
+
+## Writing NDJSON
+
+Pass `format: :ndjson` to write newline-delimited JSON. An `Array` writes **one element per line**; any other value writes as a single line. This is the exact inverse of [reading NDJSON](./basic_read_api.md) back into an Array.
+
+```ruby
+SmarterJSON.generate([{ "id" => 1 }, { "id" => 2 }], format: :ndjson)   # => "{\"id\":1}\n{\"id\":2}\n"
+SmarterJSON.generate({ "id" => 1 }, format: :ndjson)                     # => "{\"id\":1}\n"   (single value → one line)
+SmarterJSON.generate([], format: :ndjson)                               # => ""              (empty array → no lines)
+```
+
+Note the difference from the default `format: :json`, where a top-level Array is written as a single JSON array (`[…]`), not as NDJSON. See [Configuration Options](./options.md) for the full list of writer options.
+
+## Round-tripping
+
+`process` and `generate` are inverses:
+
+```ruby
+obj = { "a" => 1, "b" => [2, "three", nil, true] }
+SmarterJSON.process(SmarterJSON.generate(obj)) == obj                                  # => true
+
+arr = [{ "id" => 1 }, { "id" => 2 }, { "id" => 3 }]
+SmarterJSON.process(SmarterJSON.generate(arr, format: :ndjson)) == arr                 # => true
+```
+
+Check out the [RSpec tests](../spec/generator_spec.rb) for more examples.
+
+---------------
+
+PREVIOUS: [The Basic Read API](./basic_read_api.md) | NEXT: [Configuration Options](./options.md) | UP: [README](../README.md)

data/docs/examples.md

@@ -0,0 +1,140 @@
+
+### Contents
+
+  * [Introduction](./_introduction.md)
+  * [The Basic Read API](./basic_read_api.md)
+  * [The Basic Write API](./basic_write_api.md)
+  * [Configuration Options](./options.md)
+  * [**Examples**](./examples.md)
+
+--------------
+
+# Examples
+
+**Rescue from `SmarterJSON::Error` (recommended):** SmarterJSON raises only on genuinely unparseable input (an unterminated string, a mismatched bracket), with line and column in the message. Rescuing from `SmarterJSON::Error` lets your application handle bad input gracefully.
+
+---
+
+1. [Read a JSON String](#example-1-read-a-json-string)
+2. [Read a JSON File](#example-2-read-a-json-file)
+3. [Implicit Root Object (config-style, no braces)](#example-3-implicit-root-object-config-style-no-braces)
+4. [Multiple Documents (NDJSON) → Array](#example-4-multiple-documents-ndjson--array)
+5. [Streaming a Large File with a Block](#example-5-streaming-a-large-file-with-a-block)
+6. [Symbolize Keys](#example-6-symbolize-keys)
+7. [Duplicate Keys](#example-7-duplicate-keys)
+8. [High-Precision Numbers: BigDecimal vs Float](#example-8-high-precision-numbers-bigdecimal-vs-float)
+9. [Lenient Input: Comments, Trailing Commas, Unquoted Keys](#example-9-lenient-input-comments-trailing-commas-unquoted-keys)
+10. [Write JSON](#example-10-write-json)
+11. [Write NDJSON](#example-11-write-ndjson)
+12. [Round-Trip Read and Write](#example-12-round-trip-read-and-write)
+
+---
+
+### Example 1: Read a JSON String
+
+```ruby
+require "smarter_json"
+
+SmarterJSON.process('{"a": 1, "b": [2, 3]}')   # => {"a"=>1, "b"=>[2, 3]}
+```
+
+### Example 2: Read a JSON File
+
+```ruby
+SmarterJSON.process_file("config.json")        # => the parsed value
+```
+
+`process_file` opens the file, reads it with the labeled [`encoding:`](./options.md) (default `"UTF-8"`), and parses it.
+
+### Example 3: Implicit Root Object (config-style, no braces)
+
+A config file that starts with `key: value` and has no outer `{}` is read as an object:
+
+```ruby
+SmarterJSON.process("host: localhost\nport: 5432")   # => {"host"=>"localhost", "port"=>5432}
+```
+
+### Example 4: Multiple Documents (NDJSON) → Array
+
+Plain `process` reads NDJSON / JSONL / concatenated documents with no block and no special method. Zero documents → `nil`, one → its value, two or more → an `Array`:
+
+```ruby
+SmarterJSON.process(%({"id":1}\n{"id":2}\n{"id":3}))   # => [{"id"=>1}, {"id"=>2}, {"id"=>3}]
+SmarterJSON.process('{"id":1}')                          # => {"id"=>1}
+SmarterJSON.process("")                                  # => nil
+```
+
+### Example 5: Streaming a Large File with a Block
+
+For input larger than memory, pass a block. Each document is yielded as it is read; the whole file is never loaded:
+
+```ruby
+SmarterJSON.process_file("events.ndjson") { |event| EventJob.perform_async(event) }
+```
+
+### Example 6: Symbolize Keys
+
+```ruby
+SmarterJSON.process('{"a": 1, "b": 2}', symbolize_keys: true)   # => {:a=>1, :b=>2}
+```
+
+### Example 7: Duplicate Keys
+
+By default the last value wins. Choose `:first_wins` or `:raise` instead:
+
+```ruby
+SmarterJSON.process('{"a":1,"a":2}')                          # => {"a"=>2}   (:last_wins, the default)
+SmarterJSON.process('{"a":1,"a":2}', duplicate_key: :first_wins)  # => {"a"=>1}
+SmarterJSON.process('{"a":1,"a":2}', duplicate_key: :raise)   # raises SmarterJSON::ParseError
+```
+
+### Example 8: High-Precision Numbers: BigDecimal vs Float
+
+The default `:auto` keeps high-precision decimals as `BigDecimal` (matching Oj). Force `Float` for raw speed when you don't need the precision:
+
+```ruby
+SmarterJSON.process("65.613616999999977")                        # => BigDecimal (:auto, the default)
+SmarterJSON.process("65.613616999999977", bigdecimal_load: :float)  # => 65.613616999999977 (a Float)
+```
+
+### Example 9: Lenient Input: Comments, Trailing Commas, Unquoted Keys
+
+```ruby
+SmarterJSON.process(<<~JSON)
+  {
+    host: localhost,   # unquoted key, quoteless value, and a trailing comma
+    port: 5432,
+    /* block comment */
+    url: http://example.com
+  }
+JSON
+# => {"host"=>"localhost", "port"=>5432, "url"=>"http://example.com"}
+```
+
+A `#`/`//` only starts a comment when preceded by whitespace, so `http://example.com` stays a string rather than being truncated.
+
+### Example 10: Write JSON
+
+```ruby
+SmarterJSON.generate({ "a" => 1, "b" => [2, 3] })   # => '{"a":1,"b":[2,3]}'
+SmarterJSON.generate([1, 2, 3])                       # => '[1,2,3]'
+```
+
+### Example 11: Write NDJSON
+
+An Array writes one element per line:
+
+```ruby
+SmarterJSON.generate([{ "id" => 1 }, { "id" => 2 }], format: :ndjson)   # => "{\"id\":1}\n{\"id\":2}\n"
+```
+
+### Example 12: Round-Trip Read and Write
+
+```ruby
+obj = { "a" => 1, "b" => [2, "three", nil, true] }
+SmarterJSON.process(SmarterJSON.generate(obj)) == obj   # => true
+```
+
+---------------
+
+PREVIOUS: [Configuration Options](./options.md) | UP: [README](../README.md)