json_pure 1.2.4 → 1.4.6

data/CHANGES

@@ -1,3 +1,35 @@
+2010-08-09 (1.4.6)
+  * Fixed oversight reported in http://github.com/flori/json/issues/closed#issue/23,
+    always create a new object from the state prototype.
+  * Made pure and ext api more similar again.
+2010-08-07 (1.4.5)
+  * Manage data structure nesting depth in state object during generation. This
+    should reduce problems with to_json method definіtions that only have one
+    argument.
+  * Some fixes in the state objects and additional tests.
+2010-08-06 (1.4.4)
+  * Fixes build problem for rubinius under OS X, http://github.com/flori/json/issues/closed#issue/25
+  * Fixes crashes described in http://github.com/flori/json/issues/closed#issue/21 and
+    http://github.com/flori/json/issues/closed#issue/23
+2010-05-05 (1.4.3)
+  * Fixed some test assertions, from Ruby r27587 and r27590, patch by nobu.
+  * Fixed issue http://github.com/flori/json/issues/#issue/20 reported by
+    electronicwhisper@github. Thx!
+2010-04-26 (1.4.2)
+  * Applied patch from naruse Yui NARUSE <naruse@airemix.com> to make building with
+    Microsoft Visual C possible again.
+  * Applied patch from devrandom <c1.github@niftybox.net> in order to allow building of
+    json_pure if extensiontask is not present.
+  * Thanks to Dustin Schneider <dustin@stocktwits.com>, who reported a memory
+    leak, which is fixed in this release.
+  * Applied 993f261ccb8f911d2ae57e9db48ec7acd0187283 patch from josh@github.
+2010-04-25 (1.4.1)
+  * Fix for a bug reported by Dan DeLeo <dan@kallistec.com>, caused by T_FIXNUM
+    being different on 32bit/64bit architectures.
+2010-04-23 (1.4.0)
+  * Major speed improvements and building with simplified
+    directory/file-structure.
+  * Extension should at least be comapatible with MRI, YARV and Rubinius.
 2010-04-07 (1.2.4)
   * Triger const_missing callback to make Rails' dynamic class loading work.
 2010-03-11 (1.2.3)

data/README

@@ -1,6 +1,4 @@
-== json - JSON Implementation for Ruby
-
-=== Description
+== Description
 
 This is a implementation of the JSON specification according to RFC 4627
 http://www.ietf.org/rfc/rfc4627.txt . Starting from version 1.0.0 on there
@@ -13,13 +11,11 @@ will be two variants available:
   generated by the ragel state machine compiler
   http://www.cs.queensu.ca/~thurston/ragel .
 
-Both variants of the JSON generator escape all non-ASCII and control characters
-with \uXXXX escape sequences, and support UTF-16 surrogate pairs in order to be
-able to generate the whole range of unicode code points. This means that
-generated JSON document is encoded as UTF-8 (because ASCII is a subset of
-UTF-8) and at the same time avoids decoding problems for receiving endpoints,
-that don't expect UTF-8 encoded texts. On the negative side this may lead to a
-bit longer strings than necessarry.
+Both variants of the JSON generator generate UTF-8 character sequences by
+default. If an :ascii_only option with a true value is given, they escape all
+non-ASCII and control characters with \uXXXX escape sequences, and support
+UTF-16 surrogate pairs in order to be able to generate the whole range of
+unicode code points.
 
 All strings, that are to be encoded as JSON strings, should be UTF-8 byte
 sequences on the Ruby side. To encode raw binary strings, that aren't UTF-8
@@ -34,7 +30,7 @@ String#encoding set. If a document string has ASCII-8BIT as an encoding the
 parser attempts to figure out which of the UTF encodings from above it is and
 trys to parse it.
 
-=== Installation
+== Installation
 
 It's recommended to use the extension variant of JSON, because it's faster than
 the pure ruby variant. If you cannot build it on your system, you can settle
@@ -65,7 +61,7 @@ with:
 
   # gem install json_pure
 
-=== Compiling the extensions yourself
+== Compiling the extensions yourself
 
 If you want to build the extensions yourself you need rake:
 
@@ -82,7 +78,7 @@ If you want to create the parser.c file from its parser.rl file or draw nice
 graphviz images of the state machines, you need ragel from: http://www.cs.queensu.ca/~thurston/ragel
 
 
-=== Usage
+== Usage
 
 To use JSON you can
   require 'json'
@@ -136,7 +132,7 @@ To get the best compatibility to rails' JSON implementation, you can
 Both of the additions attempt to require 'json' (like above) first, if it has
 not been required yet.
 
-=== More Examples
+== More Examples
 
 To create a JSON document from a ruby data structure, you can call
 JSON.generate like that:
@@ -226,7 +222,7 @@ The script tools/server.rb contains a small example if you want to test, how
 receiving a JSON object from a webrick server in your browser with the
 javasript prototype library http://www.prototypejs.org works.
 
-=== Speed Comparisons
+== Speed Comparisons
 
 I have created some benchmark results (see the benchmarks/data-p4-3Ghz
 subdir of the package) for the JSON-parser to estimate the speed up in the C
@@ -339,17 +335,17 @@ Here are the median comparisons for completeness' sake:
             calls/sec (  time) ->    speed  covers
             secs/call
 
-=== Author
+== Author
 
 Florian Frank <mailto:flori@ping.de>
 
-=== License
+== License
 
 Ruby License, see the COPYING file included in the source distribution. The
 Ruby License includes the GNU General Public License (GPL), Version 2, so see
 the file GPL as well.
 
-=== Download
+== Download
 
 The latest version of this library can be downloaded at
 

data/Rakefile

@@ -9,30 +9,31 @@ rescue LoadError
   puts "WARNING: rake-compiler is not installed. You will not be able to build the json gem until you install it."
 end
 
-require 'rake/clean'
-CLOBBER.include Dir['benchmarks/data/*.{dat,log}']
-
 require 'rbconfig'
 include Config
 
+require 'rake/clean'
+CLOBBER.include Dir['benchmarks/data/*.{dat,log}'], FileList['**/*.rbc']
+CLEAN.include FileList['diagrams/*.*'], 'doc', 'coverage', 'tmp',
+  FileList["ext/**/{Makefile,mkmf.log}"],
+  FileList["{ext,lib}/**/*.{so,bundle,#{CONFIG['DLEXT']},o,obj,pdb,lib,manifest,exp,def}"]
+
 MAKE = ENV['MAKE'] || %w[gmake make].find { |c| system(c, '-v') }
 PKG_NAME          = 'json'
+PKG_TITLE         = 'JSON Implementation for Ruby'
 PKG_VERSION       = File.read('VERSION').chomp
 PKG_FILES         = FileList["**/*"].exclude(/CVS|pkg|tmp|coverage|Makefile|\.nfs\./).exclude(/\.(so|bundle|o|#{CONFIG['DLEXT']})$/)
 EXT_ROOT_DIR      = 'ext/json/ext'
 EXT_PARSER_DIR    = "#{EXT_ROOT_DIR}/parser"
-EXT_PARSER_DL     = "#{EXT_ROOT_DIR}/parser.#{CONFIG['DLEXT']}"
+EXT_PARSER_DL     = "#{EXT_PARSER_DIR}/parser.#{CONFIG['DLEXT']}"
 EXT_PARSER_SRC    = "#{EXT_PARSER_DIR}/parser.c"
 PKG_FILES << EXT_PARSER_SRC
 EXT_GENERATOR_DIR = "#{EXT_ROOT_DIR}/generator"
-EXT_GENERATOR_DL  = "#{EXT_ROOT_DIR}/generator.#{CONFIG['DLEXT']}"
+EXT_GENERATOR_DL  = "#{EXT_GENERATOR_DIR}/generator.#{CONFIG['DLEXT']}"
 EXT_GENERATOR_SRC = "#{EXT_GENERATOR_DIR}/generator.c"
 RAGEL_CODEGEN     = %w[rlcodegen rlgen-cd ragel].find { |c| system(c, '-v') }
 RAGEL_DOTGEN      = %w[rlgen-dot rlgen-cd ragel].find { |c| system(c, '-v') }
 RAGEL_PATH        = "#{EXT_PARSER_DIR}/parser.rl"
-CLEAN.include FileList['diagrams/*.*'], 'doc', 'coverage', 'tmp',
-  FileList["ext/**/{Makefile,mkmf.log}"],
-  FileList["{ext,lib}/**/*.{so,bundle,#{CONFIG['DLEXT']},o,obj,pdb,lib,manifest,exp,def}"]
 
 def myruby(*args, &block)
   @myruby ||= File.join(CONFIG['bindir'], CONFIG['ruby_install_name'])
@@ -158,12 +159,14 @@ desc "Benchmarking parser"
 task :benchmark_parser do
   ENV['RUBYOPT'] = "-Ilib:ext #{ENV['RUBYOPT']}"
   myruby 'benchmarks/parser_benchmark.rb'
+  myruby 'benchmarks/parser2_benchmark.rb'
 end
 
 desc "Benchmarking generator"
 task :benchmark_generator do
   ENV['RUBYOPT'] = "-Ilib:ext #{ENV['RUBYOPT']}"
   myruby 'benchmarks/generator_benchmark.rb'
+  myruby 'benchmarks/generator2_benchmark.rb'
 end
 
 desc "Benchmarking library"
@@ -171,14 +174,14 @@ task :benchmark => [ :benchmark_parser, :benchmark_generator ]
 
 desc "Create RDOC documentation"
 task :doc => [ :version, EXT_PARSER_SRC ] do
-  sh "rdoc -o doc -m README README lib/json.rb #{FileList['lib/json/**/*.rb']} #{EXT_PARSER_SRC} #{EXT_GENERATOR_SRC}"
+  sh "sdoc -o doc -t '#{PKG_TITLE}' -m README README lib/json.rb #{FileList['lib/json/**/*.rb']} #{EXT_PARSER_SRC} #{EXT_GENERATOR_SRC}"
 end
 
-if defined?(Gem) and defined?(Rake::GemPackageTask) and defined?(Rake::ExtensionTask)
+if defined?(Gem) and defined?(Rake::GemPackageTask)
   spec_pure = Gem::Specification.new do |s|
     s.name = 'json_pure'
     s.version = PKG_VERSION
-    s.summary = "A JSON implementation in Ruby"
+    s.summary = PKG_TITLE
     s.description = "This is a JSON implementation in pure Ruby."
 
     s.files = PKG_FILES
@@ -192,7 +195,7 @@ if defined?(Gem) and defined?(Rake::GemPackageTask) and defined?(Rake::Extension
     s.has_rdoc = true
     s.extra_rdoc_files << 'README'
     s.rdoc_options <<
-      '--title' <<  'JSON -- A JSON implemention' << '--main' << 'README'
+      '--title' <<  'JSON implemention for ruby' << '--main' << 'README'
     s.test_files.concat Dir['tests/*.rb']
 
     s.author = "Florian Frank"
@@ -205,11 +208,13 @@ if defined?(Gem) and defined?(Rake::GemPackageTask) and defined?(Rake::Extension
       pkg.need_tar = true
       pkg.package_files = PKG_FILES
   end
+end
 
+if defined?(Gem) and defined?(Rake::GemPackageTask) and defined?(Rake::ExtensionTask)
   spec_ext = Gem::Specification.new do |s|
     s.name = 'json'
     s.version = PKG_VERSION
-    s.summary = "A JSON implementation as a Ruby extension"
+    s.summary = PKG_TITLE
     s.description = "This is a JSON implementation as a Ruby extension in C."
 
     s.files = PKG_FILES
@@ -227,7 +232,7 @@ if defined?(Gem) and defined?(Rake::GemPackageTask) and defined?(Rake::Extension
     s.has_rdoc = true
     s.extra_rdoc_files << 'README'
     s.rdoc_options <<
-      '--title' <<  'JSON -- A JSON implemention' << '--main' << 'README'
+      '--title' <<  'JSON implemention for Ruby' << '--main' << 'README'
     s.test_files.concat Dir['tests/*.rb']
 
     s.author = "Florian Frank"

data/VERSION

@@ -1 +1 @@
-1.2.4
+1.4.6

data/benchmarks/generator2_benchmark.rb

@@ -0,0 +1,222 @@
+#!/usr/bin/env ruby
+# CODING: UTF-8
+
+require 'rbconfig'
+RUBY_PATH=File.join(Config::CONFIG['bindir'], Config::CONFIG['ruby_install_name'])
+RAKE_PATH=File.join(Config::CONFIG['bindir'], 'rake')
+require 'bullshit'
+case ARGV.first
+when 'ext'
+  require 'json/ext'
+when 'pure'
+  require 'json/pure'
+when 'rails'
+  require 'active_support'
+when 'yajl'
+  require 'yajl'
+  require 'yajl/json_gem'
+  require 'stringio'
+end
+
+module JSON
+  def self.[](*) end
+end
+
+module Generator2BenchmarkCommon
+  include JSON
+
+  def setup
+    @big = eval File.read(File.join(File.dirname(__FILE__), 'ohai.ruby'))
+  end
+
+  def generic_reset_method
+    @result and @result.size >= 16 or raise @result.to_s
+  end
+end
+
+module JSONGeneratorCommon
+  include Generator2BenchmarkCommon
+
+  def benchmark_generator_fast
+    @result = JSON.fast_generate(@big)
+  end
+
+  alias reset_benchmark_generator_fast generic_reset_method
+
+  def benchmark_generator_safe
+    @result = JSON.generate(@big)
+  end
+
+  alias reset_benchmark_generator_safe generic_reset_method
+
+  def benchmark_generator_pretty
+    @result = JSON.pretty_generate(@big)
+  end
+
+  alias reset_benchmark_generator_pretty generic_reset_method
+
+  def benchmark_generator_ascii
+    @result = JSON.generate(@big, :ascii_only => true)
+  end
+
+  alias reset_benchmark_generator_ascii generic_reset_method
+end
+
+class Generator2BenchmarkExt < Bullshit::RepeatCase
+  include JSONGeneratorCommon
+
+  warmup      yes
+  iterations  2000
+
+  truncate_data do
+    enabled false
+    alpha_level 0.05
+    window_size 50
+    slope_angle 0.1
+  end
+
+  autocorrelation do
+    alpha_level 0.05
+    max_lags    50
+    file        yes
+  end
+
+
+  output_dir File.join(File.dirname(__FILE__), 'data')
+  output_filename benchmark_name + '.log'
+  data_file yes
+  histogram yes
+end
+
+class Generator2BenchmarkPure < Bullshit::RepeatCase
+  include JSONGeneratorCommon
+
+  warmup      yes
+  iterations  400
+
+  truncate_data do
+    enabled false
+    alpha_level 0.05
+    window_size 50
+    slope_angle 0.1
+  end
+
+  autocorrelation do
+    alpha_level 0.05
+    max_lags    50
+    file        yes
+  end
+
+  output_dir File.join(File.dirname(__FILE__), 'data')
+  output_filename benchmark_name + '.log'
+  data_file yes
+  histogram yes
+end
+
+class Generator2BenchmarkRails < Bullshit::RepeatCase
+  include Generator2BenchmarkCommon
+
+  warmup      yes
+  iterations  400
+
+  truncate_data do
+    enabled false
+    alpha_level 0.05
+    window_size 50
+    slope_angle 0.1
+  end
+
+  autocorrelation do
+    alpha_level 0.05
+    max_lags    50
+    file        yes
+  end
+
+  output_dir File.join(File.dirname(__FILE__), 'data')
+  output_filename benchmark_name + '.log'
+  data_file yes
+  histogram yes
+
+  def benchmark_generator
+    @result = @big.to_json
+  end
+
+  alias reset_benchmark_generator generic_reset_method
+end
+
+class Generator2BenchmarkYajl < Bullshit::RepeatCase
+  include Generator2BenchmarkCommon
+
+  warmup      yes
+  iterations  2000
+
+  truncate_data do
+    enabled false
+    alpha_level 0.05
+    window_size 50
+    slope_angle 0.1
+  end
+
+  autocorrelation do
+    alpha_level 0.05
+    max_lags    50
+    file        yes
+  end
+
+  output_dir File.join(File.dirname(__FILE__), 'data')
+  output_filename benchmark_name + '.log'
+  data_file yes
+  histogram yes
+
+  def benchmark_generator
+    output = StringIO.new
+    Yajl::Encoder.new.encode(@big, output)
+    @result = output.string
+  end
+
+  def benchmark_generator_gem_api
+    @result = @big.to_json
+  end
+
+  def reset_benchmark_generator
+    generic_reset_method
+  end
+end
+
+if $0 == __FILE__
+  Bullshit::Case.autorun false
+
+  case ARGV.first
+  when 'ext'
+    Generator2BenchmarkExt.run
+  when 'pure'
+    Generator2BenchmarkPure.run
+  when 'rails'
+    Generator2BenchmarkRails.run
+  when 'yajl'
+    Generator2BenchmarkYajl.run
+  else
+    system "#{RAKE_PATH} clean"
+    system "#{RUBY_PATH} #$0 rails"
+    system "#{RUBY_PATH} #$0 pure"
+    system "#{RAKE_PATH} compile_ext"
+    system "#{RUBY_PATH} #$0 ext"
+    system "#{RUBY_PATH} #$0 yajl"
+    Bullshit.compare do
+      output_filename File.join(File.dirname(__FILE__), 'data', 'Generator2BenchmarkComparison.log')
+
+      benchmark Generator2BenchmarkExt,    :generator_fast,    :load => yes
+      benchmark Generator2BenchmarkExt,    :generator_safe,    :load => yes
+      benchmark Generator2BenchmarkExt,    :generator_pretty,  :load => yes
+      benchmark Generator2BenchmarkExt,    :generator_ascii,   :load => yes
+      benchmark Generator2BenchmarkPure,   :generator_fast,    :load => yes
+      benchmark Generator2BenchmarkPure,   :generator_safe,    :load => yes
+      benchmark Generator2BenchmarkPure,   :generator_pretty,  :load => yes
+      benchmark Generator2BenchmarkPure,   :generator_ascii,   :load => yes
+      benchmark Generator2BenchmarkRails,  :generator,         :load => yes
+      benchmark Generator2BenchmarkYajl,   :generator,         :load => yes
+      benchmark Generator2BenchmarkYajl,   :generator_gem_api, :load => yes
+    end
+  end
+end
+

data/benchmarks/generator_benchmark.rb

@@ -12,6 +12,10 @@ when 'pure'
   require 'json/pure'
 when 'rails'
   require 'active_support'
+when 'yajl'
+  require 'yajl'
+  require 'yajl/json_gem'
+  require 'stringio'
 end
 
 module JSON
@@ -23,7 +27,7 @@ module GeneratorBenchmarkCommon
 
   def setup
     a = [ nil, false, true, "fÖßÄr", [ "n€st€d", true ], { "fooß" => "bär", "quux" => true } ]
-    puts a.to_json
+    puts a.to_json if a.respond_to?(:to_json)
     @big = a * 100
   end
 
@@ -52,15 +56,22 @@ module JSONGeneratorCommon
   end
 
   alias reset_benchmark_generator_pretty generic_reset_method
+
+  def benchmark_generator_ascii
+    @result = JSON.generate(@big, :ascii_only => true)
+  end
+
+  alias reset_benchmark_generator_ascii generic_reset_method
 end
 
 class GeneratorBenchmarkExt < Bullshit::RepeatCase
   include JSONGeneratorCommon
 
   warmup      yes
-  iterations  1000
+  iterations  2000
 
   truncate_data do
+    enabled false
     alpha_level 0.05
     window_size 50
     slope_angle 0.1
@@ -83,9 +94,10 @@ class GeneratorBenchmarkPure < Bullshit::RepeatCase
   include JSONGeneratorCommon
 
   warmup      yes
-  iterations  1000
+  iterations  400
 
   truncate_data do
+    enabled false
     alpha_level 0.05
     window_size 50
     slope_angle 0.1
@@ -107,9 +119,10 @@ class GeneratorBenchmarkRails < Bullshit::RepeatCase
   include GeneratorBenchmarkCommon
 
   warmup      yes
-  iterations  1000
+  iterations  400
 
   truncate_data do
+    enabled false
     alpha_level 0.05
     window_size 50
     slope_angle 0.1
@@ -133,6 +146,45 @@ class GeneratorBenchmarkRails < Bullshit::RepeatCase
   alias reset_benchmark_generator generic_reset_method
 end
 
+class GeneratorBenchmarkYajl < Bullshit::RepeatCase
+  include GeneratorBenchmarkCommon
+
+  warmup      yes
+  iterations  2000
+
+  truncate_data do
+    enabled false
+    alpha_level 0.05
+    window_size 50
+    slope_angle 0.1
+  end
+
+  autocorrelation do
+    alpha_level 0.05
+    max_lags    50
+    file        yes
+  end
+
+  output_dir File.join(File.dirname(__FILE__), 'data')
+  output_filename benchmark_name + '.log'
+  data_file yes
+  histogram yes
+
+  def benchmark_generator
+    output = StringIO.new
+    Yajl::Encoder.new.encode(@big, output)
+    @result = output.string
+  end
+
+  def benchmark_generator_gem_api
+    @result = @big.to_json
+  end
+
+  def reset_benchmark_generator
+    generic_reset_method
+  end
+end
+
 if $0 == __FILE__
   Bullshit::Case.autorun false
 
@@ -143,22 +195,29 @@ if $0 == __FILE__
     GeneratorBenchmarkPure.run
   when 'rails'
     GeneratorBenchmarkRails.run
+  when 'yajl'
+    GeneratorBenchmarkYajl.run
   else
     system "#{RAKE_PATH} clean"
     system "#{RUBY_PATH} #$0 rails"
     system "#{RUBY_PATH} #$0 pure"
     system "#{RAKE_PATH} compile_ext"
     system "#{RUBY_PATH} #$0 ext"
+    system "#{RUBY_PATH} #$0 yajl"
     Bullshit.compare do
       output_filename File.join(File.dirname(__FILE__), 'data', 'GeneratorBenchmarkComparison.log')
 
       benchmark GeneratorBenchmarkExt,    :generator_fast,    :load => yes
       benchmark GeneratorBenchmarkExt,    :generator_safe,    :load => yes
       benchmark GeneratorBenchmarkExt,    :generator_pretty,  :load => yes
+      benchmark GeneratorBenchmarkExt,    :generator_ascii,    :load => yes
       benchmark GeneratorBenchmarkPure,   :generator_fast,    :load => yes
       benchmark GeneratorBenchmarkPure,   :generator_safe,    :load => yes
       benchmark GeneratorBenchmarkPure,   :generator_pretty,  :load => yes
+      benchmark GeneratorBenchmarkPure,   :generator_ascii,   :load => yes
       benchmark GeneratorBenchmarkRails,  :generator,         :load => yes
+      benchmark GeneratorBenchmarkYajl,   :generator,         :load => yes
+      benchmark GeneratorBenchmarkYajl,   :generator_gem_api, :load => yes
     end
   end
 end