nokogiri 1.5.5.rc3-java → 1.5.6-java

data/CHANGELOG.ja.rdoc

@@ -1,7 +1,47 @@
-== 1.5.5 / unreleased
+== 1.5.6 / unreleased
 
 * Features
 
+  * XML::Document#collect_namespaces　メソッドのパフォーマンスを改善した。 #761 (ありがとう、Juergen Mangler!)
+  * SAX::Document#processing_instructionに新しいcallbackが追加 (ありがとう、Kitaiti Makoto!)
+  * Node#native_content= メソッドでエスケープされていない文字列をセットできるようにした。 #768
+  * 名前空間を付けて xpath 式を書く場合に、シンボルキーを使えるようにした。#729 (ありがとう、Ben Langfeld.)
+  * XML::Node#[]= メソッド内で受け取った引数を文字列に変換するようにした。#729 (ありがとう、Ben Langfeld.)
+  * bin/nokogiri コマンドが $stdin からドキュメントを読んで処理できるようにした。
+  * bin/nokogiri -e を指定することでコマンドラインプログラムを実行できるようにした。
+  * bin/nokogiri --version will print the Xerces and NekoHTML versions when ran with JRuby.
+
+
+* Bugfixes
+  * Nokogiri はこのバージョンからXSLT変換のエラーを検出するようになった。#731 (ありがとう、Justin Fitzsimmons!)
+  * Don't throw an Error when trying to replace top-level text node in DocumentFragment. #775
+  * SAXパーザに不正なエンコーディングに渡された場合はArgumentErrorを投げるようにした。#756 (ありがとう、Bradley Schaefer!)
+  * [JRuby] XML宣言の前にスペースがあると、ドキュメントのパーズに失敗する。(#748の修正でこれもなおっている) #790
+  * [JRuby] Nokogiri::XML::Node#content のJRubyの振る舞いがCRubyと同じではない。#794, #797
+  * [JRuby] で '#' で始まる文字列を名前とする EntityReference を作ろうとすると INVALID_CHARACTER_ERR という例外がはっせいする。 #719
+  * [JRuby] では Nodeのサブクラスのnamespaceを正しく文字列に変換しない。 #715
+  * [JRuby] Node#contentがこのバージョンから改行コードを正しく表示するようになった。#737 (ありがとう、Piotr Szmielew!)
+  * [JRuby] recover optionが指定されている場合は宣言の無いネームスペースを無視するようにした。#748
+  * [JRuby] ネームスペースを検出するXPathが続けて実行されても例外を投げてはいけない。#764
+  * [JRuby] XMLを表示(出力)する際のホワイトスペースの扱いをlibxml2バージョンとさらに同様になるようにした。#771
+  * [JRuby] ネームスペース付きの属性を含むXMLドキュメントを文字列でbuilderに追加しようとすると失敗する。#770
+  * [JRuby] Nokogiri::XML::Document#wrapを使って生成したドキュメントに << でノードを追加しようとすると
+     undefined method `length' for nil:NilClassのエラーが発生する #781
+  * [JRuby] 開いているファイルのデスクリプタを閉じようとすると、"bad file descriptor" が発生する。#495
+  * [JRuby] JRuby/CRuby incompatibility for attribute decorators. #785
+  * [JRuby] Issues parsing valid XML with no internal subset in the DTD. #547, #811
+  * [JRuby] Issues parsing valid node content when it contains colons. #728
+  * [JRuby] Correctly parse the doc type of html documents. #733
+  * [JRuby] Include dtd in the xml output when a builder is used with create_internal_subset. #751
+  * [JRuby] builder requires textwrappers for valid utf8 in jruby, not in mri. #784
+
+
+== 1.5.5 / ２０１２年６月２４日
+
+* Features
+
+  * Much-improved support for JRuby in 1.9 mode! Yay!
+
 * Bugfixes
 
   * JRuby Nokogiri の add_previous_sibling が以前は動いていたのに今は動かない(1.5.0 -> 1.5.1)。 #691 (ありがとう, John Shahid!)
@@ -15,6 +55,7 @@
   * JRuby で Nokogiri::XML::Node を継承したクラスを定義すると、namespace が表示されない。 #695
   * JRuby で RDF::RDFXML::Writer をインスタンス化しようとすると NAMESPACE_ERR (org.w3c.dom.DOMException) が発生する. #683
   * JRuby で xpath に namespaces を指定すると例外が発生する. #493
+  * JRuby の Entity 解決は C version の Nokogiri と同じ結果にならないといけない。#704, #647, #703
 
 
 == 1.5.4 / ２０１２年６月１２日

data/CHANGELOG.rdoc

@@ -1,7 +1,46 @@
-== 1.5.5 / unreleased
+== 1.5.6 / unreleased
 
 * Features
 
+  * Improved performance of XML::Document#collect_namespaces. #761 (Thanks, Juergen Mangler!)
+  * New callback SAX::Document#processing_instruction (Thanks, Kitaiti Makoto!)
+  * Node#native_content= allows setting unescaped node contant. #768
+  * XPath lookup with namespaces supports symbol keys. #729 (Thanks, Ben Langfeld.)
+  * XML::Node#[]= stringifies values. #729 (Thanks, Ben Langfeld.)
+  * bin/nokogiri will process a document from $stdin
+  * bin/nokogiri -e will execute a program from the command line
+  * bin/nokogiri --version will print the Xerces and NekoHTML versions when ran with JRuby.
+
+
+* Bugfixes
+  * Nokogiri now detects XSLT transform errors. #731 (Thanks, Justin Fitzsimmons!)
+  * Don't throw an Error when trying to replace top-level text node in DocumentFragment. #775
+  * Raise an ArgumentError if an invalid encoding is passed to the SAX parser. #756 (Thanks, Bradley Schaefer!)
+  * [JRuby] space prior to xml preamble causes nokogiri to fail parsing. (fixed along with #748) #790
+  * [JRuby] Fixed the bug Nokogiri::XML::Node#content inconsistency between Java and C. #794, #797
+  * [JRuby] raises INVALID_CHARACTER_ERR exception when EntityReference name starts with '#'. #719
+  * [JRuby] doesn't coerce namespaces out of strings on a direct subclass of Node. #715
+  * [JRuby] Node#content now renders newlines properly. #737 (Thanks, Piotr Szmielew!)
+  * [JRuby] Unknown namespace are ignore when the recover option is used. #748
+  * [JRuby] XPath queries for namespaces should not throw exceptions when called twice in a row. #764
+  * [JRuby] More consistent (with libxml2) whitespace formatting when emitting XML. #771
+  * [JRuby] namespaced attributes broken when appending raw xml to builder. #770
+  * [JRuby] Nokogiri::XML::Document#wrap raises undefined method `length' for nil:NilClass when trying to << to a node. #781
+  * [JRuby] Fixed "bad file descriptor" bug when closing open file descriptors. #495
+  * [JRuby] JRuby/CRuby incompatibility for attribute decorators. #785
+  * [JRuby] Issues parsing valid XML with no internal subset in the DTD. #547, #811
+  * [JRuby] Issues parsing valid node content when it contains colons. #728
+  * [JRuby] Correctly parse the doc type of html documents. #733
+  * [JRuby] Include dtd in the xml output when a builder is used with create_internal_subset. #751
+  * [JRuby] builder requires textwrappers for valid utf8 in jruby, not in mri. #784
+
+
+== 1.5.5 / 2012-06-24
+
+* Features
+
+  * Much-improved support for JRuby in 1.9 mode! Yay!
+
 * Bugfixes
 
   * Regression in JRuby Nokogiri add_previous_sibling (1.5.0 -> 1.5.1) #691 (Thanks, John Shahid!)
@@ -15,6 +54,7 @@
   * JRuby renders nodes without their namespace when subclassing Node. #695
   * JRuby raises NAMESPACE_ERR (org.w3c.dom.DOMException) while instantiating RDF::RDFXML::Writer. #683
   * JRuby is not able to use namespaces in xpath. #493
+  * JRuby's Entity resolving should be consistent with C-Nokogiri #704, #647, #703
 
 
 == 1.5.4 / 2012-06-12

data/Manifest.txt

@@ -43,8 +43,13 @@ ext/java/nokogiri/XmlSyntaxError.java
 ext/java/nokogiri/XmlText.java
 ext/java/nokogiri/XmlXpathContext.java
 ext/java/nokogiri/XsltStylesheet.java
+ext/java/nokogiri/internals/ClosedStreamException.java
 ext/java/nokogiri/internals/HtmlDomParserContext.java
+ext/java/nokogiri/internals/NokogiriBlockingQueueInputStream.java
 ext/java/nokogiri/internals/NokogiriDocumentCache.java
+ext/java/nokogiri/internals/NokogiriDomParser.java
+ext/java/nokogiri/internals/NokogiriEncodingReaderWrapper.java
+ext/java/nokogiri/internals/NokogiriEntityResolver.java
 ext/java/nokogiri/internals/NokogiriErrorHandler.java
 ext/java/nokogiri/internals/NokogiriHandler.java
 ext/java/nokogiri/internals/NokogiriHelpers.java
@@ -62,7 +67,6 @@ ext/java/nokogiri/internals/ReaderNode.java
 ext/java/nokogiri/internals/SaveContextVisitor.java
 ext/java/nokogiri/internals/SchemaErrorHandler.java
 ext/java/nokogiri/internals/XmlDeclHandler.java
-ext/java/nokogiri/internals/XmlDomParser.java
 ext/java/nokogiri/internals/XmlDomParserContext.java
 ext/java/nokogiri/internals/XmlSaxParser.java
 ext/java/nokogiri/internals/XsltExtensionFunction.java
@@ -230,6 +234,9 @@ test/files/snuggles.xml
 test/files/staff.dtd
 test/files/staff.xml
 test/files/staff.xslt
+test/files/test_document_url/bar.xml
+test/files/test_document_url/document.dtd
+test/files/test_document_url/document.xml
 test/files/tlm.html
 test/files/to_be_xincluded.xml
 test/files/valid_bar.xml

data/README.ja.rdoc

@@ -1,4 +1,4 @@
-= Nokogiri (鋸)
+= Nokogiri (鋸) {<img src="https://secure.travis-ci.org/sparklemotion/nokogiri.png?rvm=1.9.3" />}[http://travis-ci.org/sparklemotion/nokogiri] {<img src="https://codeclimate.com/badge.png" />}[https://codeclimate.com/github/sparklemotion/nokogiri]
 
 * http://nokogiri.org/
 * http://github.com/sparklemotion/nokogiri/wikis

data/README.rdoc

@@ -1,4 +1,4 @@
-= Nokogiri {<img src="https://secure.travis-ci.org/sparklemotion/nokogiri.png?rvm=1.9.3" />}[http://travis-ci.org/sparklemotion/nokogiri]
+= Nokogiri {<img src="https://secure.travis-ci.org/sparklemotion/nokogiri.png?rvm=1.9.3" />}[http://travis-ci.org/sparklemotion/nokogiri] {<img src="https://codeclimate.com/badge.png" />}[https://codeclimate.com/github/sparklemotion/nokogiri]
 
 * http://nokogiri.org
 * http://github.com/sparklemotion/nokogiri/wikis
@@ -122,13 +122,10 @@ Developing Nokogiri requires racc and rexical to generate the parser and
 tokenizer.  To start development, make sure you have `libxml2` and `libxslt`
 installed.
 
-Then install hoe and rake-compiler:
+Then install core gems and bootstrap:
 
-    $ gem install hoe rake-compiler racc rexical minitest
-
-Then run rake:
-
-    $ rake
+    $ gem install hoe rake-compiler mini_portile
+    $ rake newb
 
 === Developing on JRuby
 
@@ -151,7 +148,7 @@ Then run rake:
 
 Copyright (c) 2008 - 2012:
 
-* {Aaron Patterson}[http://sparklemotionmaking.com]
+* {Aaron Patterson}[http://tenderlovemaking.com]
 * {Mike Dalessio}[http://mike.daless.io]
 * {Charles Nutter}[http://blog.headius.com]
 * {Sergio Arbeo}[http://www.serabe.com]

data/ROADMAP.md

@@ -19,8 +19,9 @@
 * https://github.com/sparklemotion/nokogiri/issues/679
   Mixing in Enumerable has some unintended consequences; plus we want to improve the attributes API
 
-* (closed) https://github.com/sparklemotion/nokogiri/issues/666
-  Some ideas for a better attributes API?
+* Some ideas for a better attributes API?
+  * (closed) https://github.com/sparklemotion/nokogiri/issues/666
+  * https://github.com/sparklemotion/nokogiri/issues/765
 
 
 ## improve CSS query parsing
@@ -69,6 +70,9 @@
   * we should standardize on a hash of options for these and other calls
 * what should NodeSet#xpath return?
   * https://github.com/sparklemotion/nokogiri/issues/656
+* also, clean up or unify the implementations of #xpath-and-friends in Node and NodeSet
+  * implementations are very similar, but no shared code :(
+  * decorate nodes in a consistent manner
 
 ## Encoding
 

data/Rakefile

@@ -17,6 +17,8 @@ def java?
   !! (RUBY_PLATFORM =~ /java/)
 end
 
+ENV['LANG'] = "en_US.UTF-8" # UBUNTU 10.04, Y U NO DEFAULT TO UTF-8?
+
 require 'tasks/nokogiri.org'
 
 HOE = Hoe.spec 'nokogiri' do
@@ -46,14 +48,10 @@ HOE = Hoe.spec 'nokogiri' do
     ["mini_portile",    ">= 0.2.2"],
     ["minitest",        "~> 2.2.2"],
     ["rake",            ">= 0.9"],
-    ["rake-compiler",   "=  0.8.0"]
+    ["rake-compiler",   "=  0.8.0"],
+    ["racc",            ">= 1.4.6"],
+    ["rexical",         ">= 1.0.5"]
   ]
-  if ! java?
-    self.extra_dev_deps += [
-      ["racc",            ">= 1.4.6"],
-      ["rexical",         ">= 1.0.5"]
-    ]
-  end
 
   if java?
     self.spec_extras = { :platform => 'java' }
@@ -118,6 +116,20 @@ desc "Generate css/parser.rb and css/tokenizer.rex"
 task 'generate' => [GENERATED_PARSER, GENERATED_TOKENIZER]
 task 'gem:spec' => 'generate' if Rake::Task.task_defined?("gem:spec")
 
+# This is a big hack to make sure that the racc and rexical
+# dependencies in the Gemfile are constrainted to ruby platforms
+# (i.e. MRI and Rubinius). There's no way to do that through hoe,
+# and any solution will require changing hoe and hoe-bundler.
+old_gemfile_task = Rake::Task['bundler:gemfile'] rescue nil
+task 'bundler:gemfile' do
+  old_gemfile_task.invoke if old_gemfile_task
+
+  lines = File.open('Gemfile', 'r') { |f| f.readlines }.map do |line|
+    line =~ /racc|rexical/ ? "#{line.strip}, :platform => :ruby" : line
+  end
+  File.open('Gemfile', 'w') { |f| lines.each { |line| f.puts line } }
+end
+
 file GENERATED_PARSER => "lib/nokogiri/css/parser.y" do |t|
   racc = RbConfig::CONFIG['target_os'] =~ /mswin32/ ? '' : `which racc`.strip
   racc = "#{::RbConfig::CONFIG['bindir']}/racc" if racc.empty?
@@ -149,9 +161,18 @@ task :java_debug do
   ENV['JAVA_OPTS'] = '-Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y' if java? && ENV['JAVA_DEBUG']
 end
 
+if java?
+  task :test_18 => :test
+  task :test_19 do
+    ENV['JRUBY_OPTS'] = "--1.9"
+    Rake::Task["test"].invoke
+  end
+end
+
 Rake::Task[:test].prerequisites << :compile
 Rake::Task[:test].prerequisites << :java_debug
 Rake::Task[:test].prerequisites << :check_extra_deps unless java?
+
 if Hoe.plugins.include?(:debugging)
   ['valgrind', 'valgrind:mem', 'valgrind:mem0'].each do |task_name|
     Rake::Task["test:#{task_name}"].prerequisites << :compile
@@ -162,6 +183,7 @@ end
 
 desc "build a windows gem without all the ceremony."
 task "gem:windows" => "gem" do
+  # TODO: 1.8.7-p358, 1.9.3-p194
   cross_rubies = ["1.8.7-p330", "1.9.2-p136"]
   ruby_cc_version = cross_rubies.collect { |_| _.split("-").first }.join(":") # e.g., "1.8.7:1.9.2"
   rake_compiler_config_path = "#{ENV['HOME']}/.rake-compiler/config.yml"

data/bin/nokogiri

@@ -16,6 +16,7 @@ opts = OptionParser.new do |opts|
   opts.separator "Examples:"
   opts.separator "  nokogiri http://www.ruby-lang.org/"
   opts.separator "  nokogiri ./public/index.html"
+  opts.separator "  curl -s http://nokogiri.org | nokogiri -e'p $_.css(\"h1\").length'"
   opts.separator ""
   opts.separator "Options:"
 
@@ -27,6 +28,10 @@ opts = OptionParser.new do |opts|
     encoding = v
   end
 
+  opts.on("-e command", "Specifies script from command-line.") do |v|
+    @script = v
+  end
+
   opts.on("--rng <uri|path>", "Validate using this rng file.") do |v|
     @rng = open(v) {|f| Nokogiri::XML::RelaxNG(f)}
   end
@@ -45,19 +50,29 @@ opts.parse!
 
 uri = ARGV.shift
 
-if uri.to_s.strip.empty?
+if uri.to_s.strip.empty? && $stdin.tty?
   puts opts
   exit 1
 end
 
-@doc = parse_class.parse(open(uri).read, nil, encoding)
+if $stdin.tty?
+  @doc = parse_class.parse(open(uri).read, nil, encoding)
+else
+  @doc = parse_class.parse($stdin, nil, encoding)
+end
+
+$_ = @doc
 
 if @rng
   @rng.validate(@doc).each do |error|
     puts error.message
   end
 else
-  puts "Your document is stored in @doc..."
-  IRB.start
+  if @script
+    eval @script, binding, '<main>'
+  else
+    puts "Your document is stored in @doc..."
+    IRB.start
+  end
 end
 

data/build_all

@@ -2,13 +2,39 @@
 #
 #  script to build gems for all relevant platforms:
 #  - MRI et al (standard gem)
-#  - windows (x86-mingw32 and x86-msin32-60)
+#  - windows (x86-mingw32 and x86-mswin32-60)
 #  - jruby
 #
-#  prerequisite is the mingw32 packages.
-#  on ubuntu, `sudo apt-get install mingw32`
-#  for others, read up at https://github.com/luislavena/rake-compiler
+#  here's what I recommend for building all the gems:
 #
+#   1. set up a vagrant VM guest running ubuntu lucid 32-bit.
+#   2. install rvm, and install 1.8.7, 1.9.3 and jruby.
+#   3. `sudo apt-get install mingw32`
+#
+#  as you build, you may run into these problems:
+#
+#  - if you're using Virtualbox shared directories, you'll get a mingw
+#    "Protocol error" at linktime. Boo! Either use NFS or a
+#    locally-checked-out repository.
+#
+#  - on ubuntus 11 and later, you may have issues with building
+#    rake-compiler's rubies against openssl v2. Just comment the lines
+#    out from ossl_ssl.c and you'll be fine.
+#
+#  - you may have issues with Pathname conversion to String in
+#    bundler. Add this to the offending bundler file:
+#
+#      class Pathname
+#        def to_str
+#          to_s
+#        end
+#      end
+#
+#  - you may also have to hack rubygems.rb to eliminate a reference to
+#    RUBY_ENGINE
+#
+
+HOST=
 
 # Load RVM into a shell session *as a function*
 if [[ -s "$HOME/.rvm/scripts/rvm" ]] ; then
@@ -21,38 +47,51 @@ fi
 
 function rvm_use {
     current_ruby=$1
-    rvm use "${1}@nokogiri" --create
+    rvm use "${1}@nokogiri" --create || rvm -v
 }
 
 set -o errexit
 
 # initialize
 rvm_use 1.8.7
+bundle install --quiet --local || bundle install
 rm -rf tmp pkg
-bundle exec rake clean clobber
+bundle exec rake clean
 
 # holding pen
 rm -rf gems
 mkdir -p gems
 
-# MRI
-rvm_use 1.8.7
-bundle exec rake gem
-cp -v pkg/nokogiri*.gem gems # should only be one at this point in the script
-
 # windows
+platform=$(uname -i)
+if [[ $platform =~ "64" ]] ; then
+    echo ""
+    echo "ERROR: You need to build the windows gem on a 32-bit machine!"
+    echo ""
+    exit 1
+fi
 rvm_use 1.8.7
-bundle exec rake-compiler cross-ruby VERSION=1.8.7-p330
-bundle exec rake-compiler cross-ruby VERSION=1.9.2-p136
+if [[ ! -a ${HOME}/.rake-compiler/ruby/ruby-1.8.7-p330/lib/ruby/1.8.7/x86_64-linux/rbconfig.rb ]] ; then
+    bundle exec rake-compiler cross-ruby VERSION=1.8.7-p330
+fi
+if [[ ! -a ${HOME}/.rake-compiler/ruby/ruby-1.9.2-p136/lib/ruby/1.9.1/x86_64-linux/rbconfig.rb ]] ; then
+    bundle exec rake-compiler cross-ruby VERSION=1.9.2-p136
+fi
 bundle exec rake cross
-rake gem:windows # don't use bundler here. it blows up. *shrug*
+bundle exec rake gem:windows
 cp -v pkg/nokogiri*x86-{mingw32,mswin32}*.gem gems
 
+# MRI
+rvm_use 1.8.7
+bundle exec rake gem
+cp -v pkg/nokogiri*.gem gems # should only be one at this point in the script
+
 # jruby
-rvm_use jruby-1.6.5
+rvm_use jruby
+bundle install --quiet --local || bundle install
 bundle exec rake clean clobber
 rvm_use 1.8.7
 bundle exec rake generate
-rvm_use jruby-1.6.5
+rvm_use jruby
 bundle exec rake gem
-cp -v pkg/nokogiri*java.gem gems
+cp -v pkg/nokogiri*java.gem gems

data/ext/java/nokogiri/HtmlDocument.java

@@ -55,6 +55,10 @@ import org.w3c.dom.NodeList;
  */
 @JRubyClass(name="Nokogiri::HTML::Document", parent="Nokogiri::XML::Document")
 public class HtmlDocument extends XmlDocument {
+    private static final String DEFAULT_CONTENT_TYPE = "html";
+    private static final String DEFAULT_PUBLIC_ID = "-//W3C//DTD HTML 4.01//EN";
+    private static final String DEFAULT_SYTEM_ID = "http://www.w3.org/TR/html4/strict.dtd";
+
     private String parsed_encoding = null;
 
     public HtmlDocument(Ruby ruby, RubyClass klazz) {
@@ -82,6 +86,28 @@ public class HtmlDocument extends XmlDocument {
         return htmlDocument;
     }
 
+    public IRubyObject getInternalSubset(ThreadContext context) {
+        IRubyObject internalSubset = super.getInternalSubset(context);
+
+        // html documents are expected to have a default internal subset
+        // the default values are the same ones used when the following
+        // feature is turned on
+        // "http://cyberneko.org/html/features/insert-doctype"
+        // the reason we don't turn it on, is because it overrides the document's
+        // declared doctype declaration.
+
+        if (internalSubset.isNil()) {
+            internalSubset = XmlDtd.newEmpty(context.getRuntime(),
+                                             getDocument(),
+                                             context.getRuntime().newString(DEFAULT_CONTENT_TYPE),
+                                             context.getRuntime().newString(DEFAULT_PUBLIC_ID),
+                                             context.getRuntime().newString(DEFAULT_SYTEM_ID));
+            setInternalSubset(internalSubset);
+        }
+
+        return internalSubset;
+    }
+
     public static IRubyObject do_parse(ThreadContext context,
                                        IRubyObject klass,
                                        IRubyObject[] args) {