nokogiri 1.5.0-java → 1.5.1-java

data/CHANGELOG.ja.rdoc

@@ -1,22 +1,66 @@
+== 1.5.1 / 未リリース
+
+* 新機能
+
+  * XML::Builder#comment はコメントノードを作れるようになった.
+  * CSS searches now support namespaced attributes. #593
+  * Java integration 機能が追加された. このバージョンから, 
+    XML::Document.wrap と XML::Document#to_java メソッドが利用可能。
+  * RelaxNG validator support in the `nokogiri` cli utility. #591 (thanks, Dan Radez!)
+
+* バグの修正
+
+  * エンコーディング自動認識において発生しうるメモリリークを修正.
+    Tim Elliottに感謝！
+  * homebrew はインストールしたら、extconf は homebrew のパスを読む。
+  * Inconsistent behavior of Nokogiri 1.5.0 Java #620
+  * Inheriting from Nokogiri::XML::Node on JRuby (1.6.4/5) fails #560
+  * XML::Attr nodes are not allowed to be added as node children, so an
+    exception is raised. #558
+  * No longer defensively "pickle" adjacent text nodes on
+    Node#add_next_sibling and Node#add_previous_sibling calls. #595.
+  * Java version inconsistency: it returns nil for empty attributes #589
+  * to_xhtml incorrectly generates <p /></p> when tag is empty #557
+  * Document#add_child now accepts a Node, NodeSet, DocumentFragment,
+    or String. #546.
+  * Document#create_element now recognizes namespaces containing
+    non-word characters (like "SOAP-ENV"). This is mostly relevant to
+    users of Builder, which calls Document#create_element for nearly
+    everything. #531.
+  * File encoding broken in 1.5.0 / jruby / windows #529
+  * Java version does not return namespace defs as attrs for ::HTML #542
+  * Bad file descriptor with Nokogiri 1.5.0 #495
+  * remove_namespace! doesn't work in pure java version #492
+  * The Nokogiri Java native build throws a null pointer exception when
+    ActiveSupport's .blank? method is called directly on a parsed object.
+    #489
+  * 1.5.0 Not using correct character encoding #488
+  * Raw XML string in XML Builder broken on JRuby #486
+  * Nokogiri 1.5.0 XML generation broken on JRuby #484
+  * Do not allow multiple root nodes. #550
+  * Fixes for custom XPath functions. #605, #606 (thanks, Juan Wajnerman!)
+  * Node#to_xml does not override :save_with if it is provided. #505
+  * Node#set is a private method [JRuby]. #564 (thanks, Nick Sieger!)
+  * C14n cleanup and Node#canonicalize (thanks, Ivan Pirlik!) #563
+
 == 1.5.0 / 2011年7月1日
 
-* Notes
+* 註
 
-  * See changelog from 1.4.7
+  * 1.4.7からの変更点を参照
 
-* Features
+* 新機能
 
-  * extracted sets of Node::SaveOptions into Node::SaveOptions::DEFAULT_{X,H,XH}TML (refactor)
+  * 各文書形式用のデフォルトのNode::SaveOptionsの組合せを定数化.
+    (Node::SaveOptions::DEFAULT_{X,H,XH}TML)
 
-* Bugfixes
+* バグの修正
 
-  * default output of XML on JRuby is no longer formatted due to
-    inconsistent whitespace handling. #415
-  * (JRuby) making empty NodeSets with null `nodes` member safe to operate on. #443
-  * Fix a bug in advanced encoding detection that leads to partially
-    duplicated document when parsing an HTML file with unknown
-    encoding.
-  * Add support for <meta charset="...">.
+  * JRuby版ではホワイトスペースの扱いに難があるため、XML出力（to_xml）において
+    自動整形をデフォルトでは行わないように変更. #415
+  * JRuby版でNodeのないNodeSetでNullPointerExceptionが発生するのを修正. #443
+  * エンコーディング宣言のないHTMLファイルで部分的に重複したドキュメントが生成される問題を修正した. #478
+  * <meta charset="..."> を認識するようになった.
 
 
 == 1.5.0 beta3 ２０１０年１２月２日

data/CHANGELOG.rdoc

@@ -1,3 +1,48 @@
+== 1.5.1 / unreleased
+
+* Features
+
+  * XML::Builder#comment allows creation of comment nodes.
+  * CSS searches now support namespaced attributes. #593
+  * Java integration feature is added. Now, XML::Document.wrap
+    and XML::Document#to_java methods are available.
+  * RelaxNG validator support in the `nokogiri` cli utility. #591 (thanks, Dan Radez!)
+
+* Bugfixes
+
+  * Fix many memory leaks and segfault opportunities. Thanks, Tim Elliott!
+  * extconf searches homebrew paths if homebrew is installed.
+  * Inconsistent behavior of Nokogiri 1.5.0 Java #620
+  * Inheriting from Nokogiri::XML::Node on JRuby (1.6.4/5) fails #560
+  * XML::Attr nodes are not allowed to be added as node children, so an
+    exception is raised. #558
+  * No longer defensively "pickle" adjacent text nodes on
+    Node#add_next_sibling and Node#add_previous_sibling calls. #595.
+  * Java version inconsistency: it returns nil for empty attributes #589
+  * to_xhtml incorrectly generates <p /></p> when tag is empty #557
+  * Document#add_child now accepts a Node, NodeSet, DocumentFragment,
+    or String. #546.
+  * Document#create_element now recognizes namespaces containing
+    non-word characters (like "SOAP-ENV"). This is mostly relevant to
+    users of Builder, which calls Document#create_element for nearly
+    everything. #531.
+  * File encoding broken in 1.5.0 / jruby / windows #529
+  * Java version does not return namespace defs as attrs for ::HTML #542
+  * Bad file descriptor with Nokogiri 1.5.0 #495
+  * remove_namespace! doesn't work in pure java version #492
+  * The Nokogiri Java native build throws a null pointer exception
+    when ActiveSupport's .blank? method is called directly on a parsed
+    object. #489
+  * 1.5.0 Not using correct character encoding #488
+  * Raw XML string in XML Builder broken on JRuby #486
+  * Nokogiri 1.5.0 XML generation broken on JRuby #484    
+  * Do not allow multiple root nodes. #550
+  * Fixes for custom XPath functions. #605, #606 (thanks, Juan Wajnerman!)
+  * Node#to_xml does not override :save_with if it is provided. #505
+  * Node#set is a private method [JRuby]. #564 (thanks, Nick Sieger!)
+  * C14n cleanup and Node#canonicalize (thanks, Ivan Pirlik!) #563
+
+
 == 1.5.0 / 2011-07-01
 
 * Notes

data/C_CODING_STYLE.rdoc

@@ -0,0 +1,27 @@
+= C/C++ mode style for Nokogiri
+
+Please don't propose commits that only change whitespace. However, if your
+commit touches a function or section that is not using MRI Ruby conventions,
+feel free to update whitespace in the surrounding code.
+
+= WHITESPACE:
+
+indent level: 2
+indent type: Always spaces
+Line Breaks: LF
+
+= FUNCTION DECLARATION:
+
+ANSI C style:
+
+    type name(args)
+    {
+      declarations
+    
+      code
+    }
+
+= SOURCES:
+
+* <3<3<3
+

data/Manifest.txt

@@ -72,6 +72,8 @@ ext/nokogiri/html_entity_lookup.c
 ext/nokogiri/html_entity_lookup.h
 ext/nokogiri/html_sax_parser_context.c
 ext/nokogiri/html_sax_parser_context.h
+ext/nokogiri/html_sax_push_parser.c
+ext/nokogiri/html_sax_push_parser.h
 ext/nokogiri/nokogiri.c
 ext/nokogiri/nokogiri.h
 ext/nokogiri/xml_attr.c
@@ -154,6 +156,7 @@ lib/nokogiri/html/element_description_defaults.rb
 lib/nokogiri/html/entity_lookup.rb
 lib/nokogiri/html/sax/parser.rb
 lib/nokogiri/html/sax/parser_context.rb
+lib/nokogiri/html/sax/push_parser.rb
 lib/nokogiri/syntax_error.rb
 lib/nokogiri/version.rb
 lib/nokogiri/xml.rb
@@ -270,6 +273,7 @@ test/xml/test_node_attributes.rb
 test/xml/test_node_encoding.rb
 test/xml/test_node_reparenting.rb
 test/xml/test_node_set.rb
+test/xml/test_node_inheritance.rb
 test/xml/test_parse_options.rb
 test/xml/test_processing_instruction.rb
 test/xml/test_reader_encoding.rb

data/README.rdoc

@@ -1,4 +1,4 @@
-= Nokogiri
+= Nokogiri {<img src="https://secure.travis-ci.org/tenderlove/nokogiri.png?rvm=1.9.3" />}[http://travis-ci.org/tenderlove/nokogiri]
 
 * http://nokogiri.org
 * http://github.com/tenderlove/nokogiri/wikis
@@ -25,6 +25,10 @@ correctly implemented CSS3 selector support as well as XPath support.
 
 == SUPPORT:
 
+Before filing a bug report, please read our {submission guidelines}[http://nokogiri.org/tutorials/getting_help.html] at:
+
+  * http://nokogiri.org/tutorials/getting_help.html
+
 The Nokogiri {mailing list}[http://groups.google.com/group/nokogiri-talk]
 is available here:
 
@@ -50,19 +54,19 @@ The IRC channel is #nokogiri on freenode.
 
   ####
   # Search for nodes by css
-  doc.css('h3.r a.l').each do |link|
+  doc.css('h3.r a').each do |link|
     puts link.content
   end
 
   ####
   # Search for nodes by xpath
-  doc.xpath('//h3/a[@class="l"]').each do |link|
+  doc.xpath('//h3/a').each do |link|
     puts link.content
   end
 
   ####
   # Or mix and match.
-  doc.search('h3.r a.l', '//h3/a[@class="l"]').each do |link|
+  doc.search('h3.r a.l', '//h3/a').each do |link|
     puts link.content
   end
 
@@ -128,8 +132,8 @@ Then run rake:
 
 === Developing on JRuby
 
-Currently, development with JRuby depends on C Ruby being installed.  With
-normal C Ruby, install racc and rexical:
+Currently, development with JRuby depends on CRuby being installed.  With
+CRuby, install racc and rexical:
 
     $ gem install racc rexical
 
@@ -145,7 +149,7 @@ Then run rake:
 
 (The MIT License)
 
-Copyright (c) 2008 - 2010:
+Copyright (c) 2008 - 2012:
 
 * {Aaron Patterson}[http://tenderlovemaking.com]
 * {Mike Dalessio}[http://mike.daless.io]

data/Rakefile

@@ -31,24 +31,22 @@ HOE = Hoe.spec 'nokogiri' do
 
   self.clean_globs += [
     'nokogiri.gemspec',
-    'lib/nokogiri/*.{o,so,bundle,a,log,dll}',
-    'lib/nokogiri/nokogiri.{so,dylib,rb,bundle}',
-    'lib/nokogiri/nokogiri.rb',
+    'lib/nokogiri/nokogiri.{bundle,jar,rb,so}',
     'lib/nokogiri/1.{8,9}',
     GENERATED_PARSER,
     GENERATED_TOKENIZER
   ]
 
   self.extra_dev_deps += [
+    ["hoe-bundler",     ">= 1.1"],
+    ["hoe-debugging",   ">= 1.0.3"],
+    ["hoe-gemspec",     ">= 1.0"],
+    ["hoe-git",         ">= 1.4"],
+    ["mini_portile",    ">= 0.2.2"],
+    ["minitest",        "~> 2.2.2"],
     ["racc",            ">= 1.4.6"],
+    ["rake-compiler",   "=  0.8.0"],
     ["rexical",         ">= 1.0.5"],
-    ["rake-compiler",   ">= 0.7.9"],
-    ["minitest",        "~> 2.2.2"],
-    ["mini_portile",    ">= 0.2.2"],
-    ["hoe-debugging",   ">= 0"],
-    ["hoe-git",         ">= 0"],
-    ["hoe-gemspec",     ">= 0"],
-    ["hoe-bundler",     ">= 0"]
   ]
 
   if java?
@@ -83,18 +81,28 @@ if java?
     HOE.spec.files += ['lib/nokogiri/nokogiri.jar']
   end
 else
-  require 'tasks/cross_compile'
+  mingw_available = true
+  begin
+    require 'tasks/cross_compile'
+  rescue
+    mingw_available = false
+  end
   require "rake/extensiontask"
+
+  HOE.spec.files.reject! { |f| f =~ %r{\.(java|jar)$} }
+
   Rake::ExtensionTask.new("nokogiri", HOE.spec) do |ext|
     ext.lib_dir = File.join(*['lib', 'nokogiri', ENV['FAT_DIR']].compact)
     ext.config_options << ENV['EXTOPTS']
-    ext.cross_compile  = true
-    ext.cross_platform = ["x86-mswin32-60", "x86-mingw32"]
-    ext.cross_config_options << "--with-xml2-include=#{File.join($recipes[:libxml2].path, 'include', 'libxml2')}"
-    ext.cross_config_options << "--with-xml2-lib=#{File.join($recipes[:libxml2].path, 'lib')}"
-    ext.cross_config_options << "--with-iconv-dir=#{$recipes[:libiconv].path}"
-    ext.cross_config_options << "--with-xslt-dir=#{$recipes[:libxslt].path}"
-    ext.cross_config_options << "--with-zlib-dir=#{CROSS_DIR}"
+    if mingw_available
+      ext.cross_compile  = true
+      ext.cross_platform = ["x86-mswin32-60", "x86-mingw32"]
+      ext.cross_config_options << "--with-xml2-include=#{File.join($recipes[:libxml2].path, 'include', 'libxml2')}"
+      ext.cross_config_options << "--with-xml2-lib=#{File.join($recipes[:libxml2].path, 'lib')}"
+      ext.cross_config_options << "--with-iconv-dir=#{$recipes[:libiconv].path}"
+      ext.cross_config_options << "--with-xslt-dir=#{$recipes[:libxslt].path}"
+      ext.cross_config_options << "--with-zlib-dir=#{CROSS_DIR}"
+    end
   end
 end
 
@@ -107,6 +115,7 @@ task 'gem:spec' => 'generate' if Rake::Task.task_defined?("gem:spec")
 file GENERATED_PARSER => "lib/nokogiri/css/parser.y" do |t|
   racc = RbConfig::CONFIG['target_os'] =~ /mswin32/ ? '' : `which racc`.strip
   racc = "#{::RbConfig::CONFIG['bindir']}/racc" if racc.empty?
+  racc = %x{command -v racc}.strip if racc.empty?
   sh "#{racc} -l -o #{t.name} #{t.prerequisites.first}"
 end
 
@@ -141,11 +150,20 @@ end
 # ----------------------------------------
 
 desc "build a windows gem without all the ceremony."
-task "gem:windows" do
-  rake_compiler_config = YAML.load_file("#{ENV['HOME']}/.rake-compiler/config.yml")
+task "gem:windows" => "gem" do
+  cross_rubies = ["1.8.7-p330", "1.9.2-p136"]
+  ruby_cc_version = cross_rubies.collect { |_| _.split("-").first }.join(":") # e.g., "1.8.7:1.9.2"
+  rake_compiler_config_path = "#{ENV['HOME']}/.rake-compiler/config.yml"
+
+  unless File.exists? rake_compiler_config_path
+    raise "rake-compiler has not installed any cross rubies. try running 'env --unset=HOST rake-compiler cross-ruby VERSION=#{cross_rubies.first}'"
+  end
+  rake_compiler_config = YAML.load_file(rake_compiler_config_path)
 
-  # check that rake-compiler config contains the right patchlevels of 1.8.6 and 1.9.1. see #279.
-  ["1.8.6-p383", "1.9.1-p243"].each do |version|
+  # check that rake-compiler config contains the right patchlevels. see #279 for background,
+  # and http://blog.mmediasys.com/2011/01/22/rake-compiler-updated-list-of-supported-ruby-versions-for-cross-compilation/
+  # for more up-to-date docs.
+  cross_rubies.each do |version|
     majmin, patchlevel = version.split("-")
     rbconfig = "rbconfig-#{majmin}"
     unless rake_compiler_config.key?(rbconfig) && rake_compiler_config[rbconfig] =~ /-#{patchlevel}/
@@ -153,12 +171,12 @@ task "gem:windows" do
     end
   end
 
-  # verify that --export-all is in the 1.9.1 rbconfig. see #279,#374,#375.
-  rbconfig_191 = rake_compiler_config["rbconfig-1.9.1"]
-  raise "rbconfig #{rbconfig_191} needs --export-all in its DLDFLAGS value" if File.read(rbconfig_191).grep(/CONFIG\["DLDFLAGS"\].*--export-all/).empty?
+  # verify that --export-all is in the 1.9 rbconfig. see #279,#374,#375.
+  rbconfig_19 = rake_compiler_config["rbconfig-1.9.2"]
+  raise "rbconfig #{rbconfig_19} needs --export-all in its DLDFLAGS value" if File.read(rbconfig_19).split("\n").grep(/CONFIG\["DLDFLAGS"\].*--export-all/).empty?
 
   pkg_config_path = [:libxslt, :libxml2].collect { |pkg| File.join($recipes[pkg].path, "lib/pkgconfig") }.join(":")
-  sh("env PKG_CONFIG_PATH=#{pkg_config_path} RUBY_CC_VERSION=1.8.6:1.9.1 rake cross native gem") || raise("build failed!")
+  sh("env PKG_CONFIG_PATH=#{pkg_config_path} RUBY_CC_VERSION=#{ruby_cc_version} rake cross native gem") || raise("build failed!")
 end
 
 # vim: syntax=Ruby

data/bin/nokogiri

@@ -27,6 +27,10 @@ opts = OptionParser.new do |opts|
     encoding = v
   end
 
+  opts.on("--rng <uri|path>", "Validate using this rng file.") do |v|
+    @rng = open(v) {|f| Nokogiri::XML::RelaxNG(f)}
+  end
+
   opts.on_tail("-?", "--help", "Show this message") do
     puts opts
     exit
@@ -48,6 +52,10 @@ end
 
 @doc = parse_class.parse(open(uri).read, nil, encoding)
 
-puts "Your document is stored in @doc..."
-IRB.start
+if @rng
+  puts @rng.validate(@doc)
+else
+  puts "Your document is stored in @doc..."
+  IRB.start
+end
 

data/ext/java/nokogiri/HtmlDocument.java

@@ -33,7 +33,6 @@
 package nokogiri;
 
 import nokogiri.internals.HtmlDomParserContext;
-
 import org.jruby.Ruby;
 import org.jruby.RubyClass;
 import org.jruby.anno.JRubyClass;
@@ -42,7 +41,11 @@ import org.jruby.javasupport.util.RuntimeHelpers;
 import org.jruby.runtime.Arity;
 import org.jruby.runtime.ThreadContext;
 import org.jruby.runtime.builtin.IRubyObject;
+import org.w3c.dom.Attr;
 import org.w3c.dom.Document;
+import org.w3c.dom.NamedNodeMap;
+import org.w3c.dom.Node;
+import org.w3c.dom.NodeList;
 
 /**
  * Class for Nokogiri::HTML::Document.
@@ -68,7 +71,7 @@ public class HtmlDocument extends XmlDocument {
         try {
             Document docNode = createNewDocument();
             htmlDocument = (HtmlDocument) NokogiriService.HTML_DOCUMENT_ALLOCATOR.allocate(context.getRuntime(), (RubyClass) klazz);
-            htmlDocument.setNode(context, docNode);
+            htmlDocument.setDocumentNode(context, docNode);
         } catch (Exception ex) {
             throw context.getRuntime().newRuntimeError("couldn't create document: "+ex.toString());
         }
@@ -88,6 +91,38 @@ public class HtmlDocument extends XmlDocument {
         ctx.setInputSource(context, args[0], args[1]);
         return ctx.parse(context, klass, args[1]);
     }
+    
+    public void setDocumentNode(ThreadContext context, Node node) {
+        super.setNode(context, node);
+        Ruby runtime = context.getRuntime();
+        if (node != null) {
+            Document document = (Document)node;
+            document.normalize();
+            stabilzeAttrValue(document.getDocumentElement());
+        }
+        setInstanceVariable("@decorators", runtime.getNil());
+    }
+    
+    private void stabilzeAttrValue(Node node) {
+        if (node == null) return;
+        if (node.hasAttributes()) {
+            NamedNodeMap nodeMap = node.getAttributes();
+            for (int i=0; i<nodeMap.getLength(); i++) {
+                Node n = nodeMap.item(i);
+                if (n instanceof Attr) {
+                    Attr attr = (Attr)n;
+                    String attrName = attr.getName();
+                    // not sure, but need to get value always before document is referred.
+                    // or lose attribute value
+                    String attrValue = attr.getValue(); // don't delete this line
+                }
+            }
+        }
+        NodeList children = node.getChildNodes();
+        for (int i=0; i<children.getLength(); i++) {
+            stabilzeAttrValue(children.item(i));
+        }
+    }
 
     /*
      * call-seq: