json 2.6.1 → 2.21.1

data/README.md

@@ -1,22 +1,14 @@
 # JSON implementation for Ruby
 
-[![CI](https://github.com/flori/json/actions/workflows/ci.yml/badge.svg)](https://github.com/flori/json/actions/workflows/ci.yml)
+[![CI](https://github.com/ruby/json/actions/workflows/ci.yml/badge.svg)](https://github.com/ruby/json/actions/workflows/ci.yml)
 
 ## Description
 
-This is a implementation of the JSON specification according to RFC 7159
-http://www.ietf.org/rfc/rfc7159.txt . Starting from version 1.0.0 on there
-will be two variants available:
+This is an implementation of the JSON specification according to RFC 7159
+http://www.ietf.org/rfc/rfc7159.txt .
 
-* A pure ruby variant, that relies on the iconv and the stringscan
-  extensions, which are both part of the ruby standard library.
-* The quite a bit faster native extension variant, which is in parts
-  implemented in C or Java and comes with its own unicode conversion
-  functions and a parser generated by the ragel state machine compiler
-  http://www.complang.org/ragel/ .
-
-Both variants of the JSON generator generate UTF-8 character sequences by
-default. If an :ascii\_only option with a true value is given, they escape all
+The JSON generator generate UTF-8 character sequences by default.
+If an :ascii\_only option with a true value is given, they escape all
 non-ASCII and control characters with \uXXXX escape sequences, and support
 UTF-16 surrogate pairs in order to be able to generate the whole range of
 unicode code points.
@@ -29,52 +21,15 @@ endpoint.
 
 ## Installation
 
-It's recommended to use the extension variant of JSON, because it's faster than
-the pure ruby variant. If you cannot build it on your system, you can settle
-for the latter.
-
-Just type into the command line as root:
-
-```
-# rake install
-```
-
-The above command will build the extensions and install them on your system.
-
-```
-# rake install_pure
-```
-
-or
-
-```
-# ruby install.rb
-```
-
-will just install the pure ruby implementation of JSON.
-
-If you use Rubygems you can type
+Install the gem and add to the application's Gemfile by executing:
 
-```
-# gem install json
-```
-
-instead, to install the newest JSON version.
-
-There is also a pure ruby json only variant of the gem, that can be installed
-with:
+    $ bundle add json
 
-```
-# gem install json_pure
-```
+If bundler is not being used to manage dependencies, install the gem by executing:
 
-## Compiling the extensions yourself
+    $ gem install json
 
-If you want to create the `parser.c` file from its `parser.rl` file or draw nice
-graphviz images of the state machines, you need ragel from:
-http://www.complang.org/ragel/
-
-## Usage
+## Basic Usage
 
 To use JSON you can
 
@@ -82,20 +37,6 @@ To use JSON you can
 require 'json'
 ```
 
-to load the installed variant (either the extension `'json'` or the pure
-variant `'json_pure'`). If you have installed the extension variant, you can
-pick either the extension variant or the pure variant by typing
-
-```ruby
-require 'json/ext'
-```
-
-or
-
-```ruby
-require 'json/pure'
-```
-
 Now you can parse a JSON document into a ruby data structure by calling
 
 ```ruby
@@ -111,59 +52,102 @@ You can also use the `pretty_generate` method (which formats the output more
 verbosely and nicely) or `fast_generate` (which doesn't do any of the security
 checks generate performs, e. g. nesting deepness checks).
 
-There are also the JSON and JSON[] methods which use parse on a String or
-generate a JSON document from an array or hash:
+## Casting non native types
 
-```ruby
-document = JSON 'test'  => 23 # => "{\"test\":23}"
-document = JSON['test' => 23] # => "{\"test\":23}"
-```
+JSON documents can only support Hashes, Arrays, Strings, Integers and Floats.
 
-and
+By default if you attempt to serialize something else, `JSON.generate` will
+search for a `#to_json` method on that object:
 
 ```ruby
-data = JSON '{"test":23}'  # => {"test"=>23}
-data = JSON['{"test":23}'] # => {"test"=>23}
+Position = Struct.new(:latitude, :longitude) do
+  def to_json(state = nil, *)
+    JSON::State.from_state(state).generate({
+      latitude: latitude,
+      longitude: longitude,
+    })
+  end
+end
+
+JSON.generate([
+  Position.new(12323.234, 435345.233),
+  Position.new(23434.676, 159435.324),
+]) # => [{"latitude":12323.234,"longitude":435345.233},{"latitude":23434.676,"longitude":159435.324}]
 ```
 
-You can choose to load a set of common additions to ruby core's objects if
-you
+If a `#to_json` method isn't defined on the object, `JSON.generate` will fallback to call `#to_s`:
 
 ```ruby
-require 'json/add/core'
+JSON.generate(Object.new) # => "#<Object:0x000000011e768b98>"
 ```
 
-After requiring this you can, e. g., serialise/deserialise Ruby ranges:
+Both of these behavior can be disabled using the `strict: true` option:
 
 ```ruby
-JSON JSON(1..10) # => 1..10
+JSON.generate(Object.new, strict: true) # => Object not allowed in JSON (JSON::GeneratorError)
+JSON.generate(Position.new(1, 2), strict: true) # => Position not allowed in JSON (JSON::GeneratorError)
 ```
 
-To find out how to add JSON support to other or your own classes, read the
-section "More Examples" below.
+## JSON::Coder
+
+Since `#to_json` methods are global, it can sometimes be problematic if you need a given type to be
+serialized in different ways in different locations.
 
-To get the best compatibility to rails' JSON implementation, you can
+Instead it is recommended to use the newer `JSON::Coder` API:
 
 ```ruby
-require 'json/add/rails'
+module MyApp
+  API_JSON_CODER = JSON::Coder.new do |object, is_object_key|
+    case object
+    when Time
+      object.iso8601(3)
+    else
+      object
+    end
+  end
+end
+
+puts MyApp::API_JSON_CODER.dump(Time.now.utc) # => "2025-01-21T08:41:44.286Z"
 ```
 
-Both of the additions attempt to require `'json'` (like above) first, if it has
-not been required yet.
+The provided block is called for all objects that don't have a native JSON equivalent, and
+must return a Ruby object that has a native JSON equivalent.
 
-## Serializing exceptions
+It is also called for objects that do have a JSON equivalent, but are used as Hash keys, for instance `{ 1 => 2}`,
+as well as for strings that aren't valid UTF-8:
 
-The JSON module doesn't extend `Exception` by default. If you convert an `Exception`
-object to JSON, it will by default only include the exception message.
+```ruby
+coder = JSON::Coder.new do |object, is_object_key|
+  case object
+  when String
+    if !object.valid_encoding? || object.encoding != Encoding::UTF_8
+      Base64.encode64(object)
+    else
+      object
+    end
+  else
+    object
+  end
+end
+```
+
+## Combining JSON fragments
 
-To include the full details, you must either load the `json/add/core` mentioned
-above, or specifically load the exception addition:
+To combine JSON fragments into a bigger JSON document, you can use `JSON::Fragment`:
 
 ```ruby
-require 'json/add/exception'
+posts_json = cache.fetch_multi(post_ids) do |post_id|
+  JSON.generate(Post.find(post_id))
+end
+posts_json.map! { |post_json| JSON::Fragment.new(post_json) }
+JSON.generate({ posts: posts_json, count: posts_json.count })
 ```
 
-## More Examples
+## Round-tripping arbitrary types
+
+> [!CAUTION]
+> You should never use `JSON.unsafe_load` nor `JSON.parse(str, create_additions: true)` to parse untrusted user input,
+> as it can lead to remote code execution vulnerabilities.
 
 To create a JSON document from a ruby data structure, you can call
 `JSON.generate` like that:
@@ -229,7 +213,7 @@ JSON.parse json
 # => [1, 2, {"a"=>3.141}, false, true, nil, 4..10]
 json = JSON.generate [1, 2, {"a"=>3.141}, false, true, nil, 4..10]
 # => "[1,2,{\"a\":3.141},false,true,null,{\"json_class\":\"Range\",\"data\":[4,10,false]}]"
-JSON.parse json, :create_additions => true
+JSON.unsafe_load json
 # => [1, 2, {"a"=>3.141}, false, true, nil, 4..10]
 ```
 
@@ -265,136 +249,35 @@ There are also the methods `Kernel#j` for generate, and `Kernel#jj` for
 `pretty_generate` output to the console, that work analogous to Core Ruby's `p` and
 the `pp` library's `pp` methods.
 
-The script `tools/server.rb` contains a small example if you want to test, how
-receiving a JSON object from a webrick server in your browser with the
-JavaScript prototype library http://www.prototypejs.org works.
-
-## Speed Comparisons
+## Security
 
-I have created some benchmark results (see the benchmarks/data-p4-3Ghz
-subdir of the package) for the JSON-parser to estimate the speed up in the C
-extension:
+When parsing or serializing untrusted input, parser and generator options should never be user controlled.
 
-```
- Comparing times (call_time_mean):
-  1 ParserBenchmarkExt#parser   900 repeats:
-        553.922304770 (  real) ->   21.500x
-          0.001805307
-  2 ParserBenchmarkYAML#parser  1000 repeats:
-        224.513358139 (  real) ->    8.714x
-          0.004454078
-  3 ParserBenchmarkPure#parser  1000 repeats:
-         26.755020642 (  real) ->    1.038x
-          0.037376163
-  4 ParserBenchmarkRails#parser 1000 repeats:
-         25.763381731 (  real) ->    1.000x
-          0.038814780
-            calls/sec (  time) ->    speed  covers
-            secs/call
+```ruby
+# Dangerous, DO NOT DO THIS.
+JSON.generate(params[:data], params[:options])
 ```
 
-In the table above 1 is `JSON::Ext::Parser`, 2 is `YAML.load` with YAML
-compatible JSON document, 3 is is `JSON::Pure::Parser`, and 4 is
-`ActiveSupport::JSON.decode`. The ActiveSupport JSON-decoder converts the
-input first to YAML and then uses the YAML-parser, the conversion seems to
-slow it down so much that it is only as fast as the `JSON::Pure::Parser`!
-
-If you look at the benchmark data you can see that this is mostly caused by
-the frequent high outliers - the median of the Rails-parser runs is still
-overall smaller than the median of the `JSON::Pure::Parser` runs:
-
-```
- Comparing times (call_time_median):
-  1 ParserBenchmarkExt#parser   900 repeats:
-        800.592479481 (  real) ->   26.936x
-          0.001249075
-  2 ParserBenchmarkYAML#parser  1000 repeats:
-        271.002390644 (  real) ->    9.118x
-          0.003690004
-  3 ParserBenchmarkRails#parser 1000 repeats:
-         30.227910865 (  real) ->    1.017x
-          0.033082008
-  4 ParserBenchmarkPure#parser  1000 repeats:
-         29.722384421 (  real) ->    1.000x
-          0.033644676
-            calls/sec (  time) ->    speed  covers
-            secs/call
-```
+Security vulnerability reports relying on attacker controlled parsing or generator options will be handled as regular bug fixes.
 
-I have benchmarked the `JSON-Generator` as well. This generated a few more
-values, because there are different modes that also influence the achieved
-speed:
+## Development
 
-```
- Comparing times (call_time_mean):
-  1 GeneratorBenchmarkExt#generator_fast    1000 repeats:
-        547.354332608 (  real) ->   15.090x
-          0.001826970
-  2 GeneratorBenchmarkExt#generator_safe    1000 repeats:
-        443.968212317 (  real) ->   12.240x
-          0.002252414
-  3 GeneratorBenchmarkExt#generator_pretty  900 repeats:
-        375.104545883 (  real) ->   10.341x
-          0.002665923
-  4 GeneratorBenchmarkPure#generator_fast   1000 repeats:
-         49.978706968 (  real) ->    1.378x
-          0.020008521
-  5 GeneratorBenchmarkRails#generator       1000 repeats:
-         38.531868759 (  real) ->    1.062x
-          0.025952543
-  6 GeneratorBenchmarkPure#generator_safe   1000 repeats:
-         36.927649925 (  real) ->    1.018x 7 (>=3859)
-          0.027079979
-  7 GeneratorBenchmarkPure#generator_pretty 1000 repeats:
-         36.272134441 (  real) ->    1.000x 6 (>=3859)
-          0.027569373
-            calls/sec (  time) ->    speed  covers
-            secs/call
-```
+### Prerequisites
 
-In the table above 1-3 are `JSON::Ext::Generator` methods. 4, 6, and 7 are
-`JSON::Pure::Generator` methods and 5 is the Rails JSON generator. It is now a
-bit faster than the `generator_safe` and `generator_pretty` methods of the pure
-variant but slower than the others.
+1. Clone the repository
+2. Install dependencies with `bundle install`
 
-To achieve the fastest JSON document output, you can use the `fast_generate`
-method. Beware, that this will disable the checking for circular Ruby data
-structures, which may cause JSON to go into an infinite loop.
+### Testing
 
-Here are the median comparisons for completeness' sake:
+The full test suite can be run with:
 
+```bash
+bundle exec rake test
 ```
- Comparing times (call_time_median):
-  1 GeneratorBenchmarkExt#generator_fast    1000 repeats:
-        708.258020939 (  real) ->   16.547x
-          0.001411915
-  2 GeneratorBenchmarkExt#generator_safe    1000 repeats:
-        569.105020353 (  real) ->   13.296x
-          0.001757145
-  3 GeneratorBenchmarkExt#generator_pretty  900 repeats:
-        482.825371244 (  real) ->   11.280x
-          0.002071142
-  4 GeneratorBenchmarkPure#generator_fast   1000 repeats:
-         62.717626652 (  real) ->    1.465x
-          0.015944481
-  5 GeneratorBenchmarkRails#generator       1000 repeats:
-         43.965681162 (  real) ->    1.027x
-          0.022745013
-  6 GeneratorBenchmarkPure#generator_safe   1000 repeats:
-         43.929073409 (  real) ->    1.026x 7 (>=3859)
-          0.022763968
-  7 GeneratorBenchmarkPure#generator_pretty 1000 repeats:
-         42.802514491 (  real) ->    1.000x 6 (>=3859)
-          0.023363113
-            calls/sec (  time) ->    speed  covers
-            secs/call
-```
-
-## Development
 
 ### Release
 
-Update the json.gemspec and json-java.gemspec.
+Update the `lib/json/version.rb` file.
 
 ```
 rbenv shell 2.6.5