word-to-markdown 1.1.9 → 1.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f2b816a2ad9402eb1c45f74f806482502608337ac9127d072647bd4f1f97cd39
-  data.tar.gz: ff35f5c9f2e89c0e781ea864552f6f20bafd23083301c4d72c5c95c7aae4f38c
+  metadata.gz: d5fff6778e90bf76409d47c5177fd2400a14308cf40be3ea2e3ee118917945b0
+  data.tar.gz: 18062303949a156da4c940daeb79854c820036fe623cdb9c4c07b1bce9dca242
 SHA512:
-  metadata.gz: e74c055913709cd0fa871ba95cf22b22a86089bb18fd9e88cd27d1dec3ec3b4927708fc9a591ba278b3e5cdc98150178cd17547ba76579702893645554370c83
-  data.tar.gz: 9b01d816e8f95d43fb19dc828213e3db4f7b0897957c6e3eba42b4f4fe888d3564db3b3730b416f9eb10837ca4911fd7083c73b3556c3313dd12cefe2f8ae597
+  metadata.gz: f041232dbf6ae3aee4738d9562fab3f820a492dfd8d71fdd257397b278ca03550a27a59a0d0b5c3514254bab49367ba067970f7551ea11c3a7b90b8eda434499
+  data.tar.gz: e8c267ccd410ccfc978ef873805696299b1a658d1894dccae2f63d39f8afd12595ced23fbce6c8f4173188cc0b1e6fa44957d37c29c32b8ea18c08fb9a8745e1

data/README.md

@@ -1,18 +1,21 @@
 # Word to Markdown converter
 
-A Ruby gem to liberate content from [the jail that is Word documents](http://ben.balter.com/2012/10/19/we-ve-been-trained-to-make-paper/#jailbreaking-content)
+> [!IMPORTANT]
+> **Looking for the latest and greatest?** Check out [**word-to-markdown-js**](https://github.com/benbalter/word-to-markdown-js), the newer, better successor to this project. It's a modern, actively maintained rewrite and is recommended for new projects. This Ruby gem remains available for existing users.
 
-[![CI](https://github.com/benbalter/word-to-markdown/actions/workflows/ci.yml/badge.svg)](https://github.com/benbalter/word-to-markdown/actions/workflows/ci.yml) [![Gem Version](https://badge.fury.io/rb/word-to-markdown.png)](http://badge.fury.io/rb/word-to-markdown) [![Inline docs](http://inch-ci.org/github/benbalter/word-to-markdown.png)](http://inch-ci.org/github/benbalter/word-to-markdown) [![Build status](https://ci.appveyor.com/api/projects/status/x2gnsfvli3q47a2e/branch/master?svg=true)](https://ci.appveyor.com/project/benbalter/word-to-markdown/branch/master) [![Maintainability](https://api.codeclimate.com/v1/badges/aae0d67ea7db185f1595/maintainability)](https://codeclimate.com/github/benbalter/word-to-markdown/maintainability) [![Test Coverage](https://api.codeclimate.com/v1/badges/aae0d67ea7db185f1595/test_coverage)](https://codeclimate.com/github/benbalter/word-to-markdown/test_coverage)
+A Ruby gem to liberate content from [the jail that is Word documents](https://ben.balter.com/2012/10/19/we-ve-been-trained-to-make-paper/#jailbreaking-content)
+
+[![CI](https://github.com/benbalter/word-to-markdown/actions/workflows/ci.yml/badge.svg)](https://github.com/benbalter/word-to-markdown/actions/workflows/ci.yml) [![Gem Version](https://img.shields.io/gem/v/word-to-markdown)](https://rubygems.org/gems/word-to-markdown)
 
 ## The problem
 
-> Our default content publishing workflow is terribly broken. [We've all been trained to make paper](http://ben.balter.com/2012/10/19/we-ve-been-trained-to-make-paper/), yet today, content authored once is more commonly consumed in multiple formats, and rarely, if ever, does it embody physical form. Put another way, our go-to content authoring workflow remains relatively unchanged since it was conceived in the early 80s.
+> Our default content publishing workflow is terribly broken. [We've all been trained to make paper](https://ben.balter.com/2012/10/19/we-ve-been-trained-to-make-paper/), yet today, content authored once is more commonly consumed in multiple formats, and rarely, if ever, does it embody physical form. Put another way, our go-to content authoring workflow remains relatively unchanged since it was conceived in the early 80s.
 >
-> I'm asked regularly by government employees — knowledge workers who fire up a desktop word processor as the first step to any project — for an automated pipeline to convert Microsoft Word documents to [Markdown](http://guides.github.com/overviews/mastering-markdown/), the *lingua franca* of the internet, but as my recent foray into building [just such a converter](http://word-to-markdown.herokuapp.com/) proves, it's not that simple.
+> I'm asked regularly by government employees — knowledge workers who fire up a desktop word processor as the first step to any project — for an automated pipeline to convert Microsoft Word documents to [Markdown](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax), the *lingua franca* of the internet, but as my recent foray into building [just such a converter](https://word2md.com/) proves, it's not that simple.
 >
 > Markdown isn't just an alternative format. Markdown forces you to write for the web.
 
-**[Read more](http://ben.balter.com/2014/03/31/word-versus-markdown-more-than-mere-semantics/)**
+**[Read more](https://ben.balter.com/2014/03/31/word-versus-markdown-more-than-mere-semantics/)**
 
 ## Just want to convert a Microsoft Word (or Google) document to Markdown?
 
@@ -20,7 +23,7 @@ You can use this **[hosted service](https://word2md.com/)** (or check out [its s
 
 ## Install
 
-You'll need to install [LibreOffice](http://www.libreoffice.org/). Then:
+You'll need to install [LibreOffice](https://www.libreoffice.org/). Then:
 
 ```bash
 gem install word-to-markdown
@@ -67,6 +70,14 @@ $ w2m path/to/document.docx
 
 Word-to-markdown requires `soffice` a command line interface to LibreOffice that works on Linux, Mac, and Windows. To install soffice, see [the LibreOffice documentation](https://www.libreoffice.org/get-help/install-howto/).
 
+Word-to-markdown only accepts Word documents (`.docx` and `.doc`), identified by their contents rather than their file extension. Other files, including those LibreOffice could otherwise open, such as HTML, ODT, or RTF, raise `WordToMarkdown::Document::UnsupportedFormatError`.
+
+LibreOffice is killed if a conversion takes longer than 60 seconds, raising `WordToMarkdown::TimeoutError`. To change the limit, set `WordToMarkdown.timeout = 120` or the `WORD_TO_MARKDOWN_TIMEOUT` environment variable.
+
+### Converting untrusted documents
+
+Word documents can link to remote resources, such as images, which LibreOffice fetches while converting the document. If you convert documents from untrusted sources (for example, files uploaded to a web service), run the conversion without network access, such as in a container or sandbox with no outbound network, so that a document can't make requests to internal services or other hosts on your behalf.
+
 ## Testing
 
 ```
@@ -75,18 +86,12 @@ script/cibuild
 
 ## Docker
 
-First, create the `Gemfile.lock` by installing the dependencies:
-
-```
-bundle install
-```
-
 Everything you need to run the executable locally:
 
 ```
-docker-compose build
-docker-compose run --rm app bundle exec w2m --help
-docker-compose run --rm app bundle exec w2m test/fixtures/em.docx
+docker compose build
+docker compose run --rm app bundle exec w2m --help
+docker compose run --rm app bundle exec w2m test/fixtures/em.docx
 ```
 
 ## Hosted service

data/bin/w2m

@@ -1,17 +1,35 @@
 #!/usr/bin/env ruby
 # frozen_string_literal: true
 
+require 'optparse'
 require 'word-to-markdown'
 
-if ARGV.size != 1 || ARGV[0] == '--help'
-  puts 'Usage: bundle exec w2m path/to/document.docx'
+parser = OptionParser.new do |opts|
+  opts.banner = 'Usage: w2m path/to/document.docx'
+
+  opts.on('-h', '--help', 'Show this help') do
+    puts opts
+    exit
+  end
+
+  opts.on('-v', '--version', 'Show the WordToMarkdown and LibreOffice versions') do
+    puts "WordToMarkdown v#{WordToMarkdown::VERSION}"
+    puts "LibreOffice v#{WordToMarkdown.soffice.version}" unless Gem.win_platform?
+    exit
+  end
+end
+
+begin
+  parser.parse!
+rescue OptionParser::ParseError => e
+  warn e.message
+  warn parser
   exit 1
 end
 
-if ARGV[0] == '--version'
-  puts "WordToMarkdown v#{WordToMarkdown::VERSION}"
-  puts "LibreOffice v#{WordToMarkdown.soffice.version}" unless Gem.win_platform?
-else
-  doc = WordToMarkdown.new ARGV[0]
-  puts doc.to_s
+if ARGV.size != 1
+  warn parser
+  exit 1
 end
+
+puts WordToMarkdown.new(ARGV[0])

data/lib/word-to-markdown/converter.rb

@@ -7,14 +7,22 @@ class WordToMarkdown
     # Number of headings to guess, e.g., h6
     HEADING_DEPTH = 6
 
-    # Percentile step for eaceh eheading
+    # Percentile step for each heading
     HEADING_STEP = 100 / HEADING_DEPTH
 
     # Minimum heading size
     MIN_HEADING_SIZE = 20
 
     # Unicode bullets to strip when processing
-    UNICODE_BULLETS = ['○', 'o', '●', "\u2022", '\\p{C}'].freeze
+    UNICODE_BULLETS = ['○', '●', "\u2022", '\\p{C}'].freeze
+
+    # Leading bullets to strip from list items. A plain "o" only counts as a
+    # bullet when followed by whitespace, so words like "orange" survive.
+    BULLET_REGEX = /\A(?:[#{UNICODE_BULLETS.join}]|o(?=[[:space:]]))+[[:space:]]*/
+
+    # Leading list numbering to strip, e.g., "1.", "a.", or "iv.", along with
+    # any whitespace (including non-breaking spaces) that follows it
+    NUMBERING_REGEX = /\A(?:\d+|[a-zA-Z]|[ivxlcdm]+|[IVXLCDM]+)\.(?:[[:space:]]+|\z)/
 
     # @param document [WordToMarkdown::Document] The document to convert
     def initialize(document)
@@ -58,13 +66,13 @@ class WordToMarkdown
         @document.tree.css('[style]').each do |element|
           sizes.push element.font_size.round(-1) unless element.font_size.nil?
         end
-        sizes.uniq.sort.extend(DescriptiveStatistics)
+        sizes.uniq.sort
       end
     end
 
     # Given a Nokogiri node, guess what heading it represents, if any
     #
-    # @param node [Nokigiri::Node] the nokigiri node
+    # @param node [Nokogiri::Node] the nokogiri node
     # @return [String, nil] the heading tag (e.g., H1), or nil
     def guess_heading(node)
       return nil if node.font_size.nil?
@@ -82,7 +90,23 @@ class WordToMarkdown
     #
     # @return [Integer] the minimum font size
     def h(num)
-      font_sizes.percentile(((HEADING_DEPTH - 1) - num) * HEADING_STEP)
+      self.class.percentile(font_sizes, ((HEADING_DEPTH - 1) - num) * HEADING_STEP)
+    end
+
+    # Linearly interpolated percentile, matching the algorithm used by the
+    # descriptive_statistics gem (v2.5.1) that this replaces
+    #
+    # @param values [Array<Numeric>] the values
+    # @param pct [Numeric] the percentile, from 0 to 100
+    #
+    # @return [Float, nil] the percentile, or nil if values is empty
+    def self.percentile(values, pct)
+      sorted = values.map(&:to_f).sort
+      rank = pct / 100.0 * (sorted.size - 1)
+      lower, upper = sorted[rank.floor, 2]
+      return lower if upper.nil? # empty or single-element collection, or the 100th percentile
+
+      lower + ((upper - lower) * (rank - rank.floor))
     end
 
     # Convert span-based font styles to `strong`s and `em`s
@@ -110,7 +134,7 @@ class WordToMarkdown
     def remove_unicode_bullets_from_list_items!
       path = WordToMarkdown.soffice.major_version == '5' ? 'li span span' : 'li span'
       @document.tree.search(path).each do |span|
-        span.inner_html = span.inner_html.gsub(/^([#{UNICODE_BULLETS.join}]+)/, '')
+        span.inner_html = span.inner_html.sub(BULLET_REGEX, '')
       end
     end
 
@@ -118,21 +142,21 @@ class WordToMarkdown
     def remove_numbering_from_list_items!
       path = WordToMarkdown.soffice.major_version == '5' ? 'li span span' : 'li span'
       @document.tree.search(path).each do |span|
-        span.inner_html = span.inner_html.gsub(/^[a-zA-Z0-9]+\./m, '')
+        span.inner_html = span.inner_html.sub(NUMBERING_REGEX, '')
       end
     end
 
-    # Remvoe whitespace from list items
+    # Remove whitespace from list items
     def remove_whitespace_from_list_items!
-      @document.tree.search('li span').each { |span| span.inner_html.strip! }
+      @document.tree.search('li span').each { |span| span.inner_html = span.inner_html.strip }
     end
 
-    # Convert table headers to `th`s2
+    # Convert table headers to `th`s
     def semanticize_table_headers!
       @document.tree.search('table tr:first td').each { |node| node.node_name = 'th' }
     end
 
-    # Try to guess heading where implicit bassed on font size
+    # Try to guess heading where implicit based on font size
     def semanticize_headings!
       implicit_headings.each do |element|
         heading = guess_heading element

data/lib/word-to-markdown/document.rb

@@ -6,14 +6,21 @@ class WordToMarkdown
 
     class ConversionError < StandardError; end
 
-    attr_reader :path, :tmpdir
+    class UnsupportedFormatError < StandardError; end
+
+    attr_reader :path, :tmpdir, :import_filter
 
     # @param path [string] Path to the Word document
     # @param tmpdir [string] Path to a working directory to use
     def initialize(path, tmpdir = nil)
       @path = File.expand_path path, Dir.pwd
-      @tmpdir = tmpdir || Dir.mktmpdir
       raise NotFoundError, "File #{@path} does not exist" unless File.exist?(@path)
+
+      @import_filter = InputFormat.filter_for(@path)
+      raise UnsupportedFormatError, "File #{@path} is not a Word document (.docx or .doc)" if @import_filter.nil?
+
+      @own_tmpdir = tmpdir.nil?
+      @tmpdir = tmpdir || Dir.mktmpdir
     end
 
     # @return [String] the document's extension
@@ -21,7 +28,7 @@ class WordToMarkdown
       File.extname path
     end
 
-    # @return [Nokigiri::Document]
+    # @return [Nokogiri::Document]
     def tree
       @tree ||= begin
         tree = Nokogiri::HTML(normalized_html)
@@ -32,6 +39,7 @@ class WordToMarkdown
 
     # @return [String] the html representation of the document
     def html
+      UrlScrubber.scrub!(tree)
       tree.to_html.gsub("</li>\n", '</li>')
     end
 
@@ -79,8 +87,8 @@ class WordToMarkdown
       string.gsub!('&nbsp;', ' ')       # HTML encoded spaces
       string.sub!(/\A[[:space:]]+/, '') # document leading whitespace
       string.sub!(/[[:space:]]+\z/, '') # document trailing whitespace
-      string.gsub!(/([ ]+)$/, '')       # line trailing whitespace
-      string.gsub!(/\n\n\n\n/, "\n\n")  # Quadruple line breaks
+      string.gsub!(/( +)$/, '')         # line trailing whitespace
+      string.gsub!("\n\n\n\n", "\n\n")  # Quadruple line breaks
       string.delete!(' ')               # Unicode non-breaking spaces, injected as tabs
       string.gsub!(/\*\*\ +(?!\*|_)([[:punct:]])/, '**\1') # Remove extra space after bold
       string
@@ -88,23 +96,31 @@ class WordToMarkdown
 
     # @return [String] the path to the intermediary HTML document
     def dest_path
-      dest_filename = File.basename(path).gsub(/#{Regexp.escape(extension)}$/, '.html')
-      File.expand_path(dest_filename, tmpdir)
+      File.expand_path("#{File.basename(path, '.*')}.html", tmpdir)
     end
 
     # @return [String] the unnormalized HTML representation
     def raw_html
       @raw_html ||= begin
-        WordToMarkdown.run_command '--headless', '--convert-to', filter, path, '--outdir', tmpdir
+        WordToMarkdown.run_command '--headless', "--infilter=#{import_filter}", '--convert-to', filter, path, '--outdir', tmpdir
         raise ConversionError, "Failed to convert #{path}" unless File.exist?(dest_path)
 
         html = File.read dest_path
         File.delete dest_path
         html
+      ensure
+        remove_tmpdir
       end
     end
 
-    # @return [String] the LibreOffice filter to use for conversion
+    # Remove the working directory if we created it and nothing else is in it.
+    # Non-empty directories are kept, since LibreOffice may have written
+    # images there that the markdown references.
+    def remove_tmpdir
+      Dir.rmdir(tmpdir) if @own_tmpdir && Dir.empty?(tmpdir)
+    end
+
+    # @return [String] the LibreOffice filter to use for export
     def filter
       if WordToMarkdown.soffice.major_version == '5'
         'html:XHTML Writer File:UTF8'

data/lib/word-to-markdown/input_format.rb

@@ -0,0 +1,86 @@
+# frozen_string_literal: true
+
+class WordToMarkdown
+  # Identifies Word documents by their content, rather than their extension,
+  # so that other formats LibreOffice would otherwise sniff and import (e.g.,
+  # HTML, which can reference remote resources) are never passed to it
+  module InputFormat
+    # Signature of an OLE compound file, used by .doc files
+    OLE_SIGNATURE = "\xD0\xCF\x11\xE0\xA1\xB1\x1A\xE1".b.freeze
+
+    # Signature of a ZIP local file header, used by .docx files
+    ZIP_SIGNATURE = "PK\x03\x04".b.freeze
+
+    # Signature of a ZIP end of central directory record
+    ZIP_EOCD_SIGNATURE = "PK\x05\x06".b.freeze
+
+    # Size of the end of central directory record, excluding the comment
+    ZIP_EOCD_SIZE = 22
+
+    # Maximum length of a ZIP archive comment
+    ZIP_MAX_COMMENT_SIZE = 0xFFFF
+
+    # The main document part every Word OOXML package contains
+    OOXML_DOCUMENT_PART = 'word/document.xml'.b.freeze
+
+    # LibreOffice import filters, by format
+    FILTERS = {
+      ooxml: 'MS Word 2007 XML',
+      ole: 'MS Word 97'
+    }.freeze
+
+    class << self
+      # @param path [String] path to the file
+      # @return [String, nil] the LibreOffice import filter for the file, or nil if it isn't a Word document
+      def filter_for(path)
+        FILTERS[detect(path)]
+      end
+
+      # @param path [String] path to the file
+      # @return [Symbol, nil] :ooxml for .docx, :ole for .doc, or nil if it isn't a Word document
+      def detect(path)
+        File.open(path, 'rb') do |file|
+          signature = file.read(OLE_SIGNATURE.bytesize).to_s
+          if signature == OLE_SIGNATURE
+            :ole
+          elsif signature.start_with?(ZIP_SIGNATURE) && ooxml_document?(file)
+            :ooxml
+          end
+        end
+      end
+
+      private
+
+      # @param file [File] an open ZIP file
+      # @return [Boolean] true if the ZIP's central directory lists the Word main document part
+      def ooxml_document?(file)
+        directory = central_directory(file)
+        !directory.nil? && directory.include?(OOXML_DOCUMENT_PART)
+      end
+
+      # Read the ZIP central directory, whose entries include each file name, uncompressed
+      #
+      # @param file [File] an open ZIP file
+      # @return [String, nil] the raw central directory, or nil if it can't be found
+      def central_directory(file)
+        size, offset = central_directory_location(file)
+        return if size.nil? || offset + size > file.size
+
+        file.seek(offset)
+        file.read(size)
+      end
+
+      # @param file [File] an open ZIP file
+      # @return [Array<Integer>, nil] the central directory's size and offset, or nil if not found
+      def central_directory_location(file)
+        tail_size = [file.size, ZIP_EOCD_SIZE + ZIP_MAX_COMMENT_SIZE].min
+        file.seek(-tail_size, IO::SEEK_END)
+        tail = file.read(tail_size)
+        index = tail.rindex(ZIP_EOCD_SIGNATURE)
+        return if index.nil? || tail.bytesize - index < ZIP_EOCD_SIZE
+
+        tail.byteslice(index + 12, 8).unpack('VV')
+      end
+    end
+  end
+end

data/lib/word-to-markdown/url_scrubber.rb

@@ -0,0 +1,110 @@
+# frozen_string_literal: true
+
+class WordToMarkdown
+  # Removes links and images with unsafe URL schemes (e.g., javascript: or
+  # vbscript:) so that they don't survive into the markdown output
+  module UrlScrubber
+    # URL schemes permitted in link targets. Links with any other scheme are
+    # unwrapped to their text. Relative URLs and fragments are always permitted.
+    SAFE_LINK_SCHEMES = %w[http https mailto].freeze
+
+    # URL schemes permitted in image sources, in addition to data:image/ URIs.
+    # Images with any other scheme are removed.
+    SAFE_IMAGE_SCHEMES = %w[http https].freeze
+
+    # Browsers remove ASCII tabs and newlines anywhere in a URL, and C0
+    # control characters and spaces at either end, before parsing the scheme
+    STRIPPED_CHARS = /[\t\n\r]/
+    EDGE_CHARS = /\A[\x00-\x20]+|[\x00-\x20]+\z/
+
+    # Matches a URL scheme, e.g., "https:"
+    SCHEME_REGEX = /\A([a-z][a-z0-9+.-]*):/i
+
+    # Matches a URL without a recognizable scheme whose first segment contains
+    # characters a Markdown renderer or browser may decode into a scheme
+    # separator, e.g., "javascript:", "javascript:", or "javascript\:"
+    AMBIGUOUS_REGEX = %r{\A[^/?#]*[:&\\]}
+
+    # Characters that could end or alter a Markdown link destination, which
+    # are percent-encoded in the URLs that are kept
+    DESTINATION_UNSAFE_CHARS = /[\x00-\x20\x7F()<>\[\]\\`"]/
+
+    # Matches a data URI for an image
+    DATA_IMAGE_REGEX = %r{\Adata:image/}i
+
+    class << self
+      # Unwrap links and remove images whose URL scheme isn't permitted
+      #
+      # @param tree [Nokogiri::HTML::Document] the document to scrub, in place
+      # @return [Nokogiri::HTML::Document] the scrubbed document
+      def scrub!(tree)
+        tree.css('a[href]').each do |node|
+          safe_link?(node['href']) ? escape_attributes!(node, 'href') : node.replace(node.children)
+        end
+
+        tree.css('img[src]').each do |node|
+          safe_image?(node['src']) ? escape_attributes!(node, 'src') : node.remove
+        end
+
+        tree
+      end
+
+      # Percent-encode characters in a URL that could end or alter a Markdown link destination
+      #
+      # @param url [String] the URL
+      # @return [String] the escaped URL
+      def escape_destination(url)
+        url.gsub(DESTINATION_UNSAFE_CHARS) { |char| format('%%%02X', char.ord) }
+      end
+
+      # @param url [String] a link target
+      # @return [Boolean] true if the URL is relative, a fragment, or has a permitted scheme
+      def safe_link?(url)
+        scheme = scheme(url)
+        scheme.nil? ? relative?(url) : SAFE_LINK_SCHEMES.include?(scheme)
+      end
+
+      # @param url [String] an image source
+      # @return [Boolean] true if the URL is relative, a data:image/ URI, or has a permitted scheme
+      def safe_image?(url)
+        scheme = scheme(url)
+        return relative?(url) if scheme.nil?
+
+        SAFE_IMAGE_SCHEMES.include?(scheme) || normalize(url).match?(DATA_IMAGE_REGEX)
+      end
+
+      # @param url [String] the URL
+      # @return [String, nil] the URL's lowercased scheme, or nil if it has none
+      def scheme(url)
+        match = normalize(url).match(SCHEME_REGEX)
+        match && match[1].downcase
+      end
+
+      private
+
+      # Escape a kept link or image's URL and title so they can't break out
+      # of the Markdown link that ReverseMarkdown writes for them
+      #
+      # @param node [Nokogiri::XML::Element] the link or image
+      # @param attribute [String] the name of the URL attribute
+      def escape_attributes!(node, attribute)
+        node[attribute] = escape_destination(node[attribute])
+        node['title'] = node['title'].tr('"', "'") if node['title']
+      end
+
+      # @param url [String] a URL without a scheme
+      # @return [Boolean] true if the URL is a relative path or fragment that can't be decoded into one with a scheme
+      def relative?(url)
+        !normalize(url).match?(AMBIGUOUS_REGEX)
+      end
+
+      # Normalize a URL the way a browser does before parsing its scheme
+      #
+      # @param url [String] the URL
+      # @return [String] the normalized URL
+      def normalize(url)
+        url.to_s.gsub(STRIPPED_CHARS, '').gsub(EDGE_CHARS, '')
+      end
+    end
+  end
+end

data/lib/word-to-markdown/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 class WordToMarkdown
-  VERSION = '1.1.9'
+  VERSION = '1.2.0'
 end

data/lib/word-to-markdown.rb

@@ -1,6 +1,5 @@
 # frozen_string_literal: true
 
-require 'descriptive_statistics/safe'
 require 'reverse_markdown'
 require 'nokogiri-styles'
 require 'premailer'
@@ -12,12 +11,16 @@ require 'cliver'
 require 'open3'
 
 require_relative 'word-to-markdown/version'
+require_relative 'word-to-markdown/input_format'
+require_relative 'word-to-markdown/url_scrubber'
 require_relative 'word-to-markdown/document'
 require_relative 'word-to-markdown/converter'
 require_relative 'nokogiri/xml/element'
 require_relative 'cliver/dependency_ext'
 
 class WordToMarkdown
+  class TimeoutError < StandardError; end
+
   attr_reader :document, :converter
 
   # Options to be passed to Reverse Markdown
@@ -26,6 +29,11 @@ class WordToMarkdown
     github_flavored: true
   }.freeze
 
+  # Default number of seconds to wait for LibreOffice to convert a document
+  # before giving up. Can be overridden with the WORD_TO_MARKDOWN_TIMEOUT
+  # environment variable, or by setting WordToMarkdown.timeout
+  DEFAULT_TIMEOUT = 60
+
   # Minimum version of LibreOffice Required
   SOFFICE_VERSION_REQUIREMENT = '> 4.0'
 
@@ -34,6 +42,7 @@ class WordToMarkdown
     '*', # Sub'd for ENV["PATH"]
     '~/Applications/LibreOffice.app/Contents/MacOS',
     '/Applications/LibreOffice.app/Contents/MacOS',
+    '/Program Files/LibreOffice/program',
     '/Program Files/LibreOffice 5/program',
     '/Program Files (x86)/LibreOffice 4/program'
   ].freeze
@@ -56,20 +65,46 @@ class WordToMarkdown
   end
 
   class << self
+    attr_writer :timeout
+
+    # @return [Numeric] seconds to wait for LibreOffice before giving up
+    def timeout
+      @timeout ||= Float(ENV.fetch('WORD_TO_MARKDOWN_TIMEOUT', DEFAULT_TIMEOUT))
+    end
+
     # Run an soffice command
     #
-    # @param args [string] one or more arguments to pass to the sofice command
+    # @param args [string] one or more arguments to pass to the soffice command
     # @return [string] the command output
     def run_command(*args)
       raise 'LibreOffice already running' if soffice.open?
 
-      output, status = Open3.capture2e(soffice.path, *args)
+      output, status = capture_with_timeout(soffice.path, *args, timeout: timeout)
       logger.debug output
       raise "Command `#{soffice.path} #{args.join(' ')}` failed: #{output}" if status.exitstatus != 0
 
       output
     end
 
+    # Run a command, capturing its combined output, and kill it (along with
+    # any child processes) if it runs longer than the timeout
+    #
+    # @param command [Array<String>] the command and its arguments
+    # @param timeout [Numeric] seconds to wait before killing the command
+    # @return [Array(String, Process::Status)] the command output and status
+    def capture_with_timeout(*command, timeout:)
+      Open3.popen2e(*command, process_group_option) do |stdin, output, waiter|
+        stdin.close
+        reader = read_in_background(output)
+        unless waiter.join(timeout)
+          kill_process_group(waiter.pid)
+          raise TimeoutError, "Command `#{command.join(' ')}` timed out after #{timeout} seconds"
+        end
+
+        [reader.value, waiter.value]
+      end
+    end
+
     # Returns a Cliver::Dependency object representing our soffice dependency
     #
     # Attempts to resolve by looking at PATH followed by paths in the PATHS constant
@@ -94,13 +129,38 @@ class WordToMarkdown
 
     private
 
+    # Start commands in their own process group, so that on timeout any
+    # processes they spawn (e.g., soffice.bin) are killed too
+    def process_group_option
+      Gem.win_platform? ? { new_pgroup: true } : { pgroup: true }
+    end
+
+    # Read a stream in a separate thread, so the command can't block on a full pipe
+    #
+    # @param io [IO] the stream to read
+    # @return [Thread] a thread whose value is the stream's contents
+    def read_in_background(io)
+      Thread.new do
+        io.read
+      rescue IOError
+        '' # The stream was closed after the command timed out
+      end
+    end
+
+    # @param pid [Integer] the pid of a process group leader
+    def kill_process_group(pid)
+      Process.kill('KILL', Gem.win_platform? ? pid : -pid)
+    rescue Errno::ESRCH
+      nil
+    end
+
     # Workaround for two upstream bugs:
-    # 1. `soffice.exe --version` on windows opens a popup and retuns a null string when manually closed
+    # 1. `soffice.exe --version` on windows opens a popup and returns a null string when manually closed
     # 2. Even if the second argument to Cliver is nil, Cliver thinks there's a requirement
     #    and will shell out to `soffice.exe --version`
     # In order to support Windows, don't pass *any* version requirement to Cliver
     def soffice_dependency_args
-      args = [path: PATHS.join(File::PATH_SEPARATOR)]
+      args = [{ path: PATHS.join(File::PATH_SEPARATOR) }]
       if Gem.win_platform?
         args
       else

metadata

@@ -1,14 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: word-to-markdown
 version: !ruby/object:Gem::Version
-  version: 1.1.9
+  version: 1.2.0
 platform: ruby
 authors:
 - Ben Balter
-autorequire:
 bindir: bin
 cert_chain: []
-date: 2025-01-08 00:00:00.000000000 Z
+date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: cliver
@@ -25,19 +24,19 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0.3'
 - !ruby/object:Gem::Dependency
-  name: descriptive_statistics
+  name: logger
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.5'
+        version: '1.4'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '2.5'
+        version: '1.4'
 - !ruby/object:Gem::Dependency
   name: nokogiri-styles
   requirement: !ruby/object:Gem::Requirement
@@ -92,42 +91,54 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.0'
+        version: '1.3'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.0'
+        version: '1.3'
 - !ruby/object:Gem::Dependency
   name: minitest
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '5'
+    - - "<"
       - !ruby/object:Gem::Version
-        version: '5.0'
+        version: '7'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '5'
+    - - "<"
       - !ruby/object:Gem::Version
-        version: '5.0'
+        version: '7'
 - !ruby/object:Gem::Dependency
   name: mocha
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '1.1'
+        version: '2'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '4'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '2'
+    - - "<"
       - !ruby/object:Gem::Version
-        version: '1.1'
+        version: '4'
 - !ruby/object:Gem::Dependency
   name: pry
   requirement: !ruby/object:Gem::Requirement
@@ -227,13 +238,17 @@ files:
 - lib/word-to-markdown.rb
 - lib/word-to-markdown/converter.rb
 - lib/word-to-markdown/document.rb
+- lib/word-to-markdown/input_format.rb
+- lib/word-to-markdown/url_scrubber.rb
 - lib/word-to-markdown/version.rb
 homepage: https://github.com/benbalter/word-to-markdown
 licenses:
 - MIT
 metadata:
   rubygems_mfa_required: 'true'
-post_install_message:
+  source_code_uri: https://github.com/benbalter/word-to-markdown
+  bug_tracker_uri: https://github.com/benbalter/word-to-markdown/issues
+  changelog_uri: https://github.com/benbalter/word-to-markdown/releases
 rdoc_options: []
 require_paths:
 - lib
@@ -241,15 +256,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '0'
+      version: '3.2'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.5.16
-signing_key:
+rubygems_version: 3.6.9
 specification_version: 4
 summary: Ruby Gem to convert Word documents to markdown
 test_files: []