nokogiri 1.13.10-x64-mingw-ucrt → 1.14.0.rc1-x64-mingw-ucrt

data/README.md

@@ -2,7 +2,7 @@
 
 # Nokogiri
 
-Nokogiri (鋸) makes it easy and painless to work with XML and HTML from Ruby. It provides a sensible, easy-to-understand API for [reading](https://nokogiri.org/tutorials/parsing_an_html_xml_document.html), writing, [modifying](https://nokogiri.org/tutorials/modifying_an_html_xml_document.html), and [querying](https://nokogiri.org/tutorials/searching_a_xml_html_document.html) documents. It is fast and standards-compliant by relying on native parsers like libxml2 (CRuby) and xerces (JRuby).
+Nokogiri (鋸) makes it easy and painless to work with XML and HTML from Ruby. It provides a sensible, easy-to-understand API for [reading](https://nokogiri.org/tutorials/parsing_an_html_xml_document.html), writing, [modifying](https://nokogiri.org/tutorials/modifying_an_html_xml_document.html), and [querying](https://nokogiri.org/tutorials/searching_a_xml_html_document.html) documents. It is fast and standards-compliant by relying on native parsers like libxml2, libgumbo, and xerces.
 
 ## Guiding Principles
 
@@ -40,10 +40,6 @@ Some guiding principles Nokogiri tries to follow:
 
 All official documentation is posted at https://nokogiri.org (the source for which is at https://github.com/sparklemotion/nokogiri.org/, and we welcome contributions).
 
-Consider subscribing to [Tidelift][tidelift] which provides license assurances and timely security notifications for your open source dependencies, including Nokogiri. [Tidelift][tidelift] subscriptions also help the Nokogiri maintainers fund our [automated testing](https://ci.nokogiri.org) which in turn allows us to ship releases, bugfixes, and security updates more often.
-
-  [tidelift]: https://tidelift.com/subscription/pkg/rubygems-nokogiri?utm_source=rubygems-nokogiri&utm_medium=referral&utm_campaign=readme
-
 ### Reading
 
 Your first stops for learning more about Nokogiri should be:
@@ -57,7 +53,6 @@ Your first stops for learning more about Nokogiri should be:
 
 There are a few ways to ask exploratory questions:
 
-- The Ruby Discord chat server is active at https://discord.gg/UyQnKrT
 - The Nokogiri mailing list is active at https://groups.google.com/group/nokogiri-talk
 - Open an issue using the "Help Request" template at https://github.com/sparklemotion/nokogiri/issues
 
@@ -103,12 +98,21 @@ We bump `Major.Minor.Patch` versions following this guidance:
 - Updating packaged libraries for security-related reasons.
 
 
+### Sponsorship
+
+You can help sponsor the maintainers of this software through one of these organizations:
+
+- [github.com/sponsors/flavorjones](https://github.com/sponsors/flavorjones)
+- [opencollective.com/nokogiri](https://opencollective.com/nokogiri)
+- [tidelift.com/subscription/pkg/rubygems-nokogiri](https://tidelift.com/subscription/pkg/rubygems-nokogiri?utm_source=rubygems-nokogiri&utm_medium=referral&utm_campaign=readme)
+
+
 ## Installation
 
 Requirements:
 
-- Ruby >= 2.6
-- JRuby >= 9.3.0.0
+- Ruby >= 2.7
+- JRuby >= 9.4.0.0
 
 
 ### Native Gems: Faster, more reliable installation
@@ -119,10 +123,13 @@ Requirements:
 
 Nokogiri ships pre-compiled, "native" gems for the following platforms:
 
-- Linux: `x86-linux` and `x86_64-linux` (req: `glibc >= 2.17`), including musl platforms like Alpine
+- Linux:
+  - `x86-linux` and `x86_64-linux` (req: `glibc >= 2.17`)
+  - `aarch64-linux` and `arm-linux` (req: `glibc >= 2.29`)
+  - Note that musl platforms like Alpine **are** supported
 - Darwin/MacOS: `x86_64-darwin` and `arm64-darwin`
-- Windows: `x86-mingw32` and `x64-mingw32`
-- Java: any platform running JRuby 9.3 or higher
+- Windows: `x86-mingw32`, `x64-mingw32`, and `x64-mingw-ucrt`
+- Java: any platform running JRuby 9.4 or higher
 
 To determine whether your system supports one of these gems, look at the output of `bundle platform` or `ruby -e 'puts Gem::Platform.local.to_s'`.
 

data/dependencies.yml

@@ -14,10 +14,28 @@ zlib:
   # SHA-256 hash provided on http://zlib.net/
 
 libiconv:
-  version: "1.16"
-  sha256: "e6a1b1b589654277ee790cce3734f07876ac4ccfaecbee8afa0b649cf529cc04"
-  # gpg: Signature made Fri 26 Apr 2019 03:36:38 PM EDT
-  # gpg:                using RSA key 4F494A942E4616C2
-  # gpg: Good signature from "Bruno Haible (Open Source Development) <bruno@clisp.org>" [expired]
-  # gpg: Note: This key has expired!
-  # Primary key fingerprint: 68D9 4D8A AEEA D48A E7DC  5B90 4F49 4A94 2E46 16C2
+  version: "1.17"
+  sha256: "8f74213b56238c85a50a5329f77e06198771e70dd9a739779f4c02f65d971313"
+  # signature verified by following this path:
+  # - release announced at https://savannah.gnu.org/forum/forum.php?forum_id=10175
+  # - which links to https://savannah.gnu.org/users/haible as the releaser
+  # - which links to https://savannah.gnu.org/people/viewgpg.php?user_id=1871 as the gpg key
+  #
+  # So:
+  # - wget -q -O - https://savannah.gnu.org/people/viewgpg.php?user_id=1871 | gpg --import
+  #     gpg: key F5BE8B267C6A406D: 1 signature not checked due to a missing key
+  #     gpg: key F5BE8B267C6A406D: public key "Bruno Haible (Open Source Development) <bruno@clisp.org>" imported
+  #     gpg: Total number processed: 1
+  #     gpg:               imported: 1
+  #     gpg: marginals needed: 3  completes needed: 1  trust model: pgp
+  #     gpg: depth: 0  valid:   4  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 4u
+  #     gpg: next trustdb check due at 2024-05-09
+  # - gpg --verify libiconv-1.17.tar.gz.sig ports/archives/libiconv-1.17.tar.gz
+  #     gpg: Signature made Sun 15 May 2022 11:26:42 AM EDT
+  #     gpg:                using RSA key 9001B85AF9E1B83DF1BDA942F5BE8B267C6A406D
+  #     gpg: Good signature from "Bruno Haible (Open Source Development) <bruno@clisp.org>" [unknown]
+  #     gpg: WARNING: This key is not certified with a trusted signature!
+  #     gpg:          There is no indication that the signature belongs to the owner.
+  #     Primary key fingerprint: 9001 B85A F9E1 B83D F1BD  A942 F5BE 8B26 7C6A 406D
+  #
+  # And this sha256sum is calculated from that verified tarball.

data/ext/nokogiri/extconf.rb

@@ -200,7 +200,7 @@ def nix?
 end
 
 def truffle?
-  ::RUBY_ENGINE == "truffleruby"
+  RUBY_ENGINE == "truffleruby"
 end
 
 def concat_flags(*args)
@@ -211,6 +211,16 @@ def local_have_library(lib, func = nil, headers = nil)
   have_library(lib, func, headers) || have_library("lib#{lib}", func, headers)
 end
 
+def zlib_source(version_string)
+  # As of 2022-12, I'm starting to see failed downloads often enough from zlib.net that I want to
+  # change the default to github.
+  if ENV["NOKOGIRI_USE_CANONICAL_ZLIB_SOURCE"]
+    "https://zlib.net/fossils/zlib-#{version_string}.tar.gz"
+  else
+    "https://github.com/madler/zlib/releases/download/v#{version_string}/zlib-#{version_string}.tar.gz"
+  end
+end
+
 def gnome_source
   # As of 2022-02-20, some mirrors have expired SSL certificates. I'm able to retrieve from my home,
   # but whatever host is resolved on the github actions workers see an expired cert.
@@ -400,8 +410,8 @@ def iconv_configure_flags
     return ["--with-iconv=yes"]
   end
 
-  config = preserving_globals { have_package_configuration("libiconv") }
-  if config && try_link_iconv("pkg-config libiconv") { have_package_configuration("libiconv") }
+  config = preserving_globals { pkg_config("libiconv") }
+  if config && try_link_iconv("pkg-config libiconv") { pkg_config("libiconv") }
     cflags, ldflags, libs = config
 
     return [
@@ -430,10 +440,12 @@ def process_recipe(name, version, static_p, cross_p, cacheable_p = true)
       "#{@target}/#{RUBY_PLATFORM}/#{@name}/#{@version}"
     end
 
-    recipe.target = File.join(PACKAGE_ROOT_DIR, "ports") if cacheable_p
-    # Prefer host_alias over host in order to use the correct compiler prefix for cross build, but
-    # use host if not set.
+    # We use 'host' to set compiler prefix for cross-compiling. Prefer host_alias over host. And
+    # prefer i686 (what external dev tools use) to i386 (what ruby's configure.ac emits).
     recipe.host = RbConfig::CONFIG["host_alias"].empty? ? RbConfig::CONFIG["host"] : RbConfig::CONFIG["host_alias"]
+    recipe.host = recipe.host.gsub(/i386/, "i686")
+
+    recipe.target = File.join(PACKAGE_ROOT_DIR, "ports") if cacheable_p
     recipe.configure_options << "--libdir=#{File.join(recipe.path, "lib")}"
 
     yield recipe
@@ -525,7 +537,6 @@ def process_recipe(name, version, static_p, cross_p, cacheable_p = true)
 
       EOM
 
-      pp(recipe.files)
       chdir_for_build { recipe.cook }
       FileUtils.touch(checkpoint)
     end
@@ -573,6 +584,34 @@ def do_clean
   exit!(0)
 end
 
+# In ruby 3.2, symbol resolution changed on Darwin, to introduce the `-bundle_loader` flag to
+# resolve symbols against the ruby binary.
+#
+# This makes it challenging to build a single extension that works with both a ruby with
+# `--enable-shared` and one with `--disable-shared. To work around that, we choose to add
+# `-flat_namespace` to the link line (later in this file).
+#
+# The `-flat_namespace` line introduces its own behavior change, which is that (similar to on
+# Linux), any symbols in the extension that are exported may now be resolved by shared libraries
+# loaded by the Ruby process. Specifically, that means that libxml2 and libxslt, which are
+# statically linked into the nokogiri bundle, will resolve (at runtime) to a system libxml2 loaded
+# by Ruby on Darwin. And it appears that often Ruby on Darwin does indeed load the system libxml2,
+# and that messes with our assumptions about whether we're running with a patched libxml2 or a
+# vanilla libxml2.
+#
+# We choose to use `-load_hidden` in this case to prevent exporting those symbols from libxml2 and
+# libxslt, which ensures that they will be resolved to the static libraries in the bundle. In other
+# words, when we use `load_hidden`, what happens in the extension stays in the extension.
+#
+# See https://github.com/rake-compiler/rake-compiler-dock/issues/87 for more info.
+#
+# Anyway, this method is the logical bit to tell us when to turn on these workarounds.
+def needs_darwin_linker_hack
+  config_cross_build? &&
+    darwin? &&
+    Gem::Requirement.new("~> 3.2").satisfied_by?(Gem::Version.new(RbConfig::CONFIG["ruby_version"].split("+").first))
+end
+
 #
 #  main
 #
@@ -616,6 +655,9 @@ $LIBS = concat_flags($LIBS, ENV["LIBS"])
 # errors/warnings. see #2302
 append_cflags(["-std=c99", "-Wno-declaration-after-statement"])
 
+# gumbo html5 serialization is slower with O3, let's make sure we use O2
+append_cflags("-O2")
+
 # always include debugging information
 append_cflags("-g")
 
@@ -625,8 +667,18 @@ append_cflags("-Winline")
 # good to have no matter what Ruby was compiled with
 append_cflags("-Wmissing-noreturn")
 
+# check integer loss of precision
+if darwin?
+  append_cflags("-Wshorten-64-to-32")
+else
+  append_cflags("-Wconversion -Wno-sign-conversion")
+end
+
 # handle clang variations, see #1101
-append_cflags("-Wno-error=unused-command-line-argument-hard-error-in-future") if darwin?
+if darwin?
+  append_cflags("-Wno-error=unused-command-line-argument-hard-error-in-future")
+  append_cflags("-Wno-unknown-warning-option")
+end
 
 # these tend to be noisy, but on occasion useful during development
 # append_cflags(["-Wcast-qual", "-Wwrite-strings"])
@@ -666,6 +718,10 @@ else
   cross_build_p = config_cross_build?
   message "Cross build is #{cross_build_p ? "enabled" : "disabled"}.\n"
 
+  if needs_darwin_linker_hack
+    append_ldflags("-Wl,-flat_namespace")
+  end
+
   require "yaml"
   dependencies = YAML.load_file(File.join(PACKAGE_ROOT_DIR, "dependencies.yml"))
 
@@ -674,7 +730,7 @@ else
   if cross_build_p || windows?
     zlib_recipe = process_recipe("zlib", dependencies["zlib"]["version"], static_p, cross_build_p) do |recipe|
       recipe.files = [{
-        url: "https://zlib.net/fossils/#{recipe.name}-#{recipe.version}.tar.gz",
+        url: zlib_source(recipe.version),
         sha256: dependencies["zlib"]["sha256"],
       }]
       if windows?
@@ -873,9 +929,13 @@ else
   $libs = $libs.shellsplit.tap do |libs|
     [libxml2_recipe, libxslt_recipe].each do |recipe|
       libname = recipe.name[/\Alib(.+)\z/, 1]
-      File.join(recipe.path, "bin", "#{libname}-config").tap do |config|
+      config_basename = "#{libname}-config"
+      File.join(recipe.path, "bin", config_basename).tap do |config|
         # call config scripts explicit with 'sh' for compat with Windows
-        $CPPFLAGS = %x(sh #{config} --cflags).strip << " " << $CPPFLAGS
+        cflags = %x(sh #{config} --cflags).strip
+        message("#{config_basename} cflags: #{cflags}\n")
+        $CPPFLAGS = concat_flags(cflags, $CPPFLAGS) # prepend
+
         %x(sh #{config} --libs).strip.shellsplit.each do |arg|
           case arg
           when /\A-L(.+)\z/
@@ -894,7 +954,7 @@ else
       end
 
       patches_string = recipe.patch_files.map { |path| File.basename(path) }.join(" ")
-      append_cppflags(%[-DNOKOGIRI_#{recipe.name.upcase}_PATCHES="\\\"#{patches_string}\\\""])
+      append_cppflags(%[-DNOKOGIRI_#{recipe.name.upcase}_PATCHES="\\"#{patches_string}\\""])
 
       case libname
       when "xml2"
@@ -913,16 +973,17 @@ else
   end.shelljoin
 
   if static_p
+    static_archive_ld_flag = needs_darwin_linker_hack ? ["-load_hidden"] : []
     $libs = $libs.shellsplit.map do |arg|
       case arg
       when "-lxml2"
-        File.join(libxml2_recipe.path, "lib", libflag_to_filename(arg))
+        static_archive_ld_flag + [File.join(libxml2_recipe.path, "lib", libflag_to_filename(arg))]
       when "-lxslt", "-lexslt"
-        File.join(libxslt_recipe.path, "lib", libflag_to_filename(arg))
+        static_archive_ld_flag + [File.join(libxslt_recipe.path, "lib", libflag_to_filename(arg))]
       else
         arg
       end
-    end.shelljoin
+    end.flatten.shelljoin
   end
 
   ensure_func("xmlParseDoc", "libxml/parser.h")
@@ -958,7 +1019,7 @@ libgumbo_recipe = process_recipe("libgumbo", "1.0.0-nokogiri", static_p, cross_b
     end
 
     def compile
-      cflags = concat_flags(ENV["CFLAGS"], "-fPIC", "-g")
+      cflags = concat_flags(ENV["CFLAGS"], "-fPIC", "-O2", "-g")
 
       env = { "CC" => gcc_cmd, "CFLAGS" => cflags }
       if config_cross_build?
@@ -978,7 +1039,7 @@ end
 append_cppflags("-I#{File.join(libgumbo_recipe.path, "include")}")
 $libs = $libs + " " + File.join(libgumbo_recipe.path, "lib", "libgumbo.a")
 $LIBPATH = $LIBPATH | [File.join(libgumbo_recipe.path, "lib")]
-ensure_func("gumbo_parse_with_options", "gumbo.h")
+ensure_func("gumbo_parse_with_options", "nokogiri_gumbo.h")
 
 have_func("xmlHasFeature") || abort("xmlHasFeature() is missing.") # introduced in libxml 2.6.21
 have_func("xmlFirstElementChild") # introduced in libxml 2.7.3
@@ -989,10 +1050,8 @@ have_func("xmlSchemaSetParserStructuredErrors") # introduced in libxml 2.6.23
 have_func("rb_gc_location") # introduced in Ruby 2.7
 have_func("rb_category_warning") # introduced in Ruby 3.0
 
-have_func("vasprintf")
-
 other_library_versions_string = OTHER_LIBRARY_VERSIONS.map { |k, v| [k, v].join(":") }.join(",")
-append_cppflags(%[-DNOKOGIRI_OTHER_LIBRARY_VERSIONS="\\\"#{other_library_versions_string}\\\""])
+append_cppflags(%[-DNOKOGIRI_OTHER_LIBRARY_VERSIONS="\\"#{other_library_versions_string}\\""])
 
 unless config_system_libraries?
   if cross_build_p

data/ext/nokogiri/gumbo.c

@@ -23,13 +23,13 @@
 //
 // Processing starts by calling gumbo_parse_with_options. The resulting document tree
 // is then walked, a parallel libxml2 tree is constructed, and the final document is
-// then wrapped using Nokogiri_wrap_xml_document. This approach reduces memory and CPU
+// then wrapped using noko_xml_document_wrap. This approach reduces memory and CPU
 // requirements as Ruby objects are only built when necessary.
 //
 
 #include <nokogiri.h>
 
-#include "gumbo.h"
+#include "nokogiri_gumbo.h"
 
 VALUE cNokogiriHtml5Document;
 
@@ -281,12 +281,12 @@ add_errors(const GumboOutput *output, VALUE rdoc, VALUE input, VALUE url)
       rb_iv_set(syntax_error, "@code", INT2NUM(1));   // XML_ERR_INTERNAL_ERROR
       rb_iv_set(syntax_error, "@level", INT2NUM(2));  // XML_ERR_ERROR
       rb_iv_set(syntax_error, "@file", url);
-      rb_iv_set(syntax_error, "@line", INT2NUM(position.line));
+      rb_iv_set(syntax_error, "@line", SIZET2NUM(position.line));
       rb_iv_set(syntax_error, "@str1", str1);
       rb_iv_set(syntax_error, "@str2", Qnil);
       rb_iv_set(syntax_error, "@str3", Qnil);
       rb_iv_set(syntax_error, "@int1", INT2NUM(0));
-      rb_iv_set(syntax_error, "@column", INT2NUM(position.column));
+      rb_iv_set(syntax_error, "@column", SIZET2NUM(position.column));
       rb_ary_push(rerrors, syntax_error);
     }
     rb_iv_set(rdoc, "@errors", rerrors);
@@ -297,6 +297,7 @@ typedef struct {
   GumboOutput *output;
   VALUE input;
   VALUE url_or_frag;
+  VALUE klass;
   xmlDocPtr doc;
 } ParseArgs;
 
@@ -321,7 +322,7 @@ static VALUE parse_continue(VALUE parse_args);
  *  @!visibility protected
  */
 static VALUE
-parse(VALUE self, VALUE input, VALUE url, VALUE max_attributes, VALUE max_errors, VALUE max_depth)
+parse(VALUE self, VALUE input, VALUE url, VALUE max_attributes, VALUE max_errors, VALUE max_depth, VALUE klass)
 {
   GumboOptions options = kGumboDefaultOptions;
   options.max_attributes = NUM2INT(max_attributes);
@@ -333,6 +334,7 @@ parse(VALUE self, VALUE input, VALUE url, VALUE max_attributes, VALUE max_errors
     .output = output,
     .input = input,
     .url_or_frag = url,
+    .klass = klass,
     .doc = NULL,
   };
 
@@ -357,7 +359,9 @@ parse_continue(VALUE parse_args)
   }
   args->doc = doc; // Make sure doc gets cleaned up if an error is thrown.
   build_tree(doc, (xmlNodePtr)doc, output->document);
-  VALUE rdoc = Nokogiri_wrap_xml_document(cNokogiriHtml5Document, doc);
+  VALUE rdoc = noko_xml_document_wrap(args->klass, doc);
+  rb_iv_set(rdoc, "@url", args->url_or_frag);
+  rb_iv_set(rdoc, "@quirks_mode", INT2NUM(output->document->v.document.doc_type_quirks_mode));
   args->doc = NULL; // The Ruby runtime now owns doc so don't delete it.
   add_errors(output, rdoc, args->input, args->url_or_frag);
   return rdoc;
@@ -498,9 +502,11 @@ error:
     }
 
     // Encoding.
-    if (RSTRING_LEN(tag_name) == 14
+    if (ctx_ns == GUMBO_NAMESPACE_MATHML
+        && RSTRING_LEN(tag_name) == 14
         && !st_strcasecmp(ctx_tag, "annotation-xml")) {
       VALUE enc = rb_funcall(ctx, rb_intern_const("[]"),
+                             1,
                              rb_utf8_str_new_static("encoding", 8));
       if (RTEST(enc)) {
         Check_Type(enc, T_STRING);
@@ -512,8 +518,11 @@ error:
   // Quirks mode.
   VALUE doc = rb_funcall(doc_fragment, rb_intern_const("document"), 0);
   VALUE dtd = rb_funcall(doc, internal_subset, 0);
-  if (NIL_P(dtd)) {
+  VALUE doc_quirks_mode = rb_iv_get(doc, "@quirks_mode");
+  if (NIL_P(ctx) || NIL_P(doc_quirks_mode)) {
     quirks_mode = GUMBO_DOCTYPE_NO_QUIRKS;
+  } else if (NIL_P(dtd)) {
+    quirks_mode = GUMBO_DOCTYPE_QUIRKS;
   } else {
     VALUE dtd_name = rb_funcall(dtd, name, 0);
     VALUE pubid = rb_funcall(dtd, rb_intern_const("external_id"), 0);
@@ -560,6 +569,7 @@ fragment_continue(VALUE parse_args)
   args->doc = NULL; // The Ruby runtime owns doc so make sure we don't delete it.
   xmlNodePtr xml_frag = extract_xml_node(doc_fragment);
   build_tree(xml_doc, xml_frag, output->root);
+  rb_iv_set(doc_fragment, "@quirks_mode", INT2NUM(output->document->v.document.doc_type_quirks_mode));
   add_errors(output, doc_fragment, args->input, rb_utf8_str_new_static("#fragment", 9));
   return Qnil;
 }
@@ -577,7 +587,7 @@ noko_init_gumbo()
   parent = rb_intern_const("parent");
 
   // Define Nokogumbo module with parse and fragment methods.
-  rb_define_singleton_method(mNokogiriGumbo, "parse", parse, 5);
+  rb_define_singleton_method(mNokogiriGumbo, "parse", parse, 6);
   rb_define_singleton_method(mNokogiriGumbo, "fragment", fragment, 6);
 }
 

data/ext/nokogiri/html4_document.c

@@ -146,7 +146,7 @@ rb_html_document_type(VALUE self)
 {
   htmlDocPtr doc;
   Data_Get_Struct(self, xmlDoc, doc);
-  return INT2NUM((long)doc->type);
+  return INT2NUM(doc->type);
 }
 
 void

data/ext/nokogiri/html4_entity_lookup.c

@@ -20,7 +20,7 @@ get(VALUE _, VALUE rb_entity_name)
     return Qnil;
   }
 
-  rb_constructor_args[0] = INT2NUM((long)c_entity_desc->value);
+  rb_constructor_args[0] = UINT2NUM(c_entity_desc->value);
   rb_constructor_args[1] = NOKOGIRI_STR_NEW2(c_entity_desc->name);
   rb_constructor_args[2] = NOKOGIRI_STR_NEW2(c_entity_desc->desc);
 

data/ext/nokogiri/html4_sax_parser_context.c

@@ -5,13 +5,8 @@ VALUE cNokogiriHtml4SaxParserContext ;
 static void
 deallocate(xmlParserCtxtPtr ctxt)
 {
-  NOKOGIRI_DEBUG_START(ctxt);
-
   ctxt->sax = NULL;
-
   htmlFreeParserCtxt(ctxt);
-
-  NOKOGIRI_DEBUG_END(ctxt);
 }
 
 static VALUE

data/ext/nokogiri/nokogiri.c

@@ -49,34 +49,11 @@ void noko_init_html_sax_push_parser(void);
 void noko_init_gumbo(void);
 void noko_init_test_global_handlers(void);
 
-static ID id_read, id_write;
-
-
-#ifndef HAVE_VASPRINTF
-/*
- * Thank you Geoffroy Couprie for this implementation of vasprintf!
- */
-int
-vasprintf(char **strp, const char *fmt, va_list ap)
-{
-  /* Mingw32/64 have a broken vsnprintf implementation that fails when
-   * using a zero-byte limit in order to retrieve the required size for malloc.
-   * So we use a one byte buffer instead.
-   */
-  char tmp[1];
-  int len = vsnprintf(tmp, 1, fmt, ap) + 1;
-  char *res = (char *)malloc((unsigned int)len);
-  if (res == NULL) {
-    return -1;
-  }
-  *strp = res;
-  return vsnprintf(res, (unsigned int)len, fmt, ap);
-}
-#endif
+static ID id_read, id_write, id_external_encoding;
 
 
 static VALUE
-read_check(VALUE val)
+noko_io_read_check(VALUE val)
 {
   VALUE *args = (VALUE *)val;
   return rb_funcall(args[0], id_read, 1, args[1]);
@@ -84,68 +61,71 @@ read_check(VALUE val)
 
 
 static VALUE
-read_failed(VALUE arg, VALUE exc)
+noko_io_read_failed(VALUE arg, VALUE exc)
 {
   return Qundef;
 }
 
 
 int
-noko_io_read(void *ctx, char *buffer, int len)
+noko_io_read(void *io, char *c_buffer, int c_buffer_len)
 {
-  VALUE string, args[2];
-  size_t str_len, safe_len;
+  VALUE rb_io = (VALUE)io;
+  VALUE rb_read_string, rb_args[2];
+  size_t n_bytes_read, safe_len;
 
-  args[0] = (VALUE)ctx;
-  args[1] = INT2NUM(len);
+  rb_args[0] = rb_io;
+  rb_args[1] = INT2NUM(c_buffer_len);
 
-  string = rb_rescue(read_check, (VALUE)args, read_failed, 0);
+  rb_read_string = rb_rescue(noko_io_read_check, (VALUE)rb_args, noko_io_read_failed, 0);
 
-  if (NIL_P(string)) { return 0; }
-  if (string == Qundef) { return -1; }
-  if (TYPE(string) != T_STRING) { return -1; }
+  if (NIL_P(rb_read_string)) { return 0; }
+  if (rb_read_string == Qundef) { return -1; }
+  if (TYPE(rb_read_string) != T_STRING) { return -1; }
 
-  str_len = (size_t)RSTRING_LEN(string);
-  safe_len = str_len > (size_t)len ? (size_t)len : str_len;
-  memcpy(buffer, StringValuePtr(string), safe_len);
+  n_bytes_read = (size_t)RSTRING_LEN(rb_read_string);
+  safe_len = (n_bytes_read > (size_t)c_buffer_len) ? (size_t)c_buffer_len : n_bytes_read;
+  memcpy(c_buffer, StringValuePtr(rb_read_string), safe_len);
 
   return (int)safe_len;
 }
 
 
 static VALUE
-write_check(VALUE val)
+noko_io_write_check(VALUE rb_args)
 {
-  VALUE *args = (VALUE *)val;
-  return rb_funcall(args[0], id_write, 1, args[1]);
+  VALUE rb_io = ((VALUE *)rb_args)[0];
+  VALUE rb_output = ((VALUE *)rb_args)[1];
+  return rb_funcall(rb_io, id_write, 1, rb_output);
 }
 
 
 static VALUE
-write_failed(VALUE arg, VALUE exc)
+noko_io_write_failed(VALUE arg, VALUE exc)
 {
   return Qundef;
 }
 
 
 int
-noko_io_write(void *ctx, char *buffer, int len)
+noko_io_write(void *io, char *c_buffer, int c_buffer_len)
 {
-  VALUE args[2], size;
-
-  args[0] = (VALUE)ctx;
-  args[1] = rb_str_new(buffer, (long)len);
+  VALUE rb_args[2], rb_n_bytes_written;
+  VALUE rb_io = (VALUE)io;
+  rb_encoding *io_encoding = rb_to_encoding(rb_funcall(rb_io, id_external_encoding, 0));
 
-  size = rb_rescue(write_check, (VALUE)args, write_failed, 0);
+  rb_args[0] = rb_io;
+  rb_args[1] = rb_enc_str_new(c_buffer, (long)c_buffer_len, io_encoding);
 
-  if (size == Qundef) { return -1; }
+  rb_n_bytes_written = rb_rescue(noko_io_write_check, (VALUE)rb_args, noko_io_write_failed, 0);
+  if (rb_n_bytes_written == Qundef) { return -1; }
 
-  return NUM2INT(size);
+  return NUM2INT(rb_n_bytes_written);
 }
 
 
 int
-noko_io_close(void *ctx)
+noko_io_close(void *io)
 {
   return 0;
 }
@@ -275,4 +255,5 @@ Init_nokogiri()
 
   id_read = rb_intern("read");
   id_write = rb_intern("write");
+  id_external_encoding = rb_intern("external_encoding");
 }