readword 0.1.1 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2c151bf301945512c356cbaadc83fc03f939c712293015a052be64a62c0c56f1
-  data.tar.gz: 3d9f112fa2fff0582d9040b6e146d9b7e587b6933330773df746e41414ac0b2f
+  metadata.gz: b700bdd0e0d5132d8179dca381922454528ac0c2606ae6bd792990250569d57e
+  data.tar.gz: 4a17928587b6725fa57b2adeeaca3db33e375ed9ac27c21d9c6da1e2b1d18da0
 SHA512:
-  metadata.gz: 876795d221df033bbddf1b407e1cbf6969ac8d39e0956e7b81056b72bae81b90d2df7a887e729494a4ce5e83cce8fbf19db02a5a6e5d69887653d5836f868bf7
-  data.tar.gz: 5ed50d5820cb66dc8b529ea78ce09b1fee80b7783e3977b5239db1bb0d2372798ad8d0483fbbc1ed1963a4d4c96b0dd7369fbadbdbaaee00c6b79d1c18eb4a81
+  metadata.gz: 9fef45c6b23b951af4e7c1d6a27101f9526f3dc137528a6fb21d5e4a8cc02ad5a52bb1ded37c738cc3d3af51c7578b85569cfb5366d129f04c19199343142ce3
+  data.tar.gz: 302d28516c6f17151795e84a6d2c71d0f2b877d172c4364ef8eb5d52e4881015fb0f5dd8200db3fd94977ef575d5d64b432156b7fd582ec20a496764f638a65f

data/CHANGELOG.md

@@ -1,5 +1,10 @@
 # Changelog
 
+## 0.2.0 (2026-10-22)
+
+- Keeps bold, italics and links, as `**bold**`, `*italic*` and `[text](url)`. Only links to web pages and email addresses are kept. Headings, and header cells that are bold as a whole, stay plain.
+- The size limit is now an option: `max_size:` in Ruby and `--max-size` on the command line, in megabytes. The default went up from 10 to 32, which reads a 300-page report, and 0 turns the limit off.
+
 ## 0.1.1 (2026-10-22)
 
 Same code as 0.1.0. Only the author name in the gem's details changed.

data/README.md

@@ -43,8 +43,9 @@ End of report.
 - Tracked changes as if accepted: inserted text stays and deleted text goes.
 - Field results, such as page numbers, dates and merge fields. Field codes are left out.
 - Characters added with Insert Symbol, such as µ, ±, ≤ and °, as the real characters.
+- Bold, italics and links, as `**bold**`, `*italic*` and `[text](https://example.com)`. Only links to web pages and email addresses are kept. Headings, and header cells that are bold as a whole, stay plain.
 
-It does not keep bold, italics, links, images, charts, comments or embedded objects (an Excel table pasted into the document as an object, for example).
+It does not keep underlining, colors, images, charts, comments in `.docx` files, or embedded objects (an Excel table pasted into the document as an object, for example).
 
 ## Use it with a coding agent
 
@@ -57,13 +58,14 @@ To read a Word file (.doc or .docx), run `readword path/to/file` and read the Ma
 ## Command line
 
 ```console
-$ readword [--max-characters N] FILE...
+$ readword [--max-characters N] [--max-size MB] FILE...
 ```
 
 - With several files, each one starts with a `==> path <==` line.
 - Use `-` as the file name to read from standard input.
 - If a file cannot be read, the reason goes to standard error and the exit code is 1. The other files are still printed.
 - `--max-characters N` refuses documents longer than N characters, which is useful to stay inside a model's context window.
+- `--max-size MB` sets the size limit described under [Safe on untrusted files](#safe-on-untrusted-files). The default is 32, and 0 turns the limit off.
 
 ## Use it from Ruby
 
@@ -72,6 +74,7 @@ require 'readword'
 
 Readword.convert_file('report.doc')                      # => "# Test report\n\n..."
 Readword.convert(File.binread('report.docx'), max_characters: 150_000)
+Readword.convert_file('manual.docx', max_size: 100)      # a size limit of 100 MB instead of 32
 ```
 
 `convert` takes the file's bytes and works out the format from them, so a `.docx` saved with a `.doc` name still reads. Files that cannot be read raise `Readword::Error`. Its `reason` is one of:
@@ -80,23 +83,48 @@ Readword.convert(File.binread('report.docx'), max_characters: 150_000)
 | --- | --- |
 | `:unreadable` | Not a Word document (RTF or plain text named `.doc`, for example), or damaged |
 | `:no_text` | The document has no text, only images for example |
-| `:too_long` | Over `max_characters`, or over the reader's own size limits |
+| `:too_long` | Over `max_characters`, or over the size limit (`max_size`) |
 | `:password_protected` | Encrypted with a password |
 | `:old_format` | Saved by Word 6 or Word 95 |
 
 ## Safe on untrusted files
 
 - The `.doc` reader is plain Ruby, so a crafted file cannot cause the memory errors that C readers are prone to. The `.docx` reader uses Nokogiri and rubyzip.
-- Size limits on unpacked XML, XML nodes, text and formatting, so a small zip bomb or a crafted file cannot make it use gigabytes of memory.
+- A size limit on the unpacked XML, its elements, the text and the formatting, so a small zip bomb or a crafted file cannot make it use gigabytes of memory. It is 32 MB by default, which reads a 300-page report. Dense documents count as bigger, since the limit also counts XML elements. At the default, the worst file we could craft, an 11 KB zip holding 1.28 million empty paragraphs, takes about 10 seconds and 330 MB of memory. Lower `max_size` for tighter bounds, or raise it for bigger documents.
 - XML parts that declare their own entities are refused, which stops entity bombs.
+- Links are kept only when they point to a web page or an email address, so a document cannot put a `javascript:` link into the Markdown.
 - Loop guards in the `.doc` container reader, so a crafted sector chain cannot make it run forever.
-- Fuzzed with 3,000,000 broken copies of real Word files, and 300,000 more after headings and lists were added, with no crash and no hang. The worst crafted files found so far use under 50 MB of memory.
+- Fuzzed with 3,000,000 broken copies of real Word files, 300,000 more after headings and lists were added, and 50,000 more after bold, italics and links, with no crash and no hang.
 
 ## How it was checked
 
-readword was compared with Apache Tika, LibreOffice and macOS textutil on 650 Word files from the Apache POI and LibreOffice test suites.
+readword was compared with Apache Tika, LibreOffice and macOS textutil on 650 Word files from the Apache POI and LibreOffice test suites, and with other tools that read Word files.
 
-| | antiword 0.37 | docx gem 0.13 | readword |
+### Compared with anydoc
+
+[anydoc](https://github.com/firecrawl/anydoc) is Firecrawl's open-source converter, written in Rust. It reads many formats and also runs as an MCP server. Version 0.2.4 was run on the same 650 files. The counts below are for the 300 `.doc` and 212 `.docx` files where both Tika and textutil found text.
+
+| | anydoc 0.2.4 | readword 0.2 |
+| --- | --- | --- |
+| `.doc` files read | 260 | 281 |
+| `.doc` files missing words that both Tika and textutil found | 122 | 29, of which 13 are Word 6 or 95 files that neither tool reads |
+| Text left out of `.doc` files | page headers and footers, comments, some text boxes | field codes, embedded objects |
+| `.doc` files saved by macOS TextEdit | refused | read |
+| `.docx` files read | 208 | 210 |
+| `.docx` files missing words that both Tika and textutil found | 9, for example content controls in tables and deeply nested tables | 2: an entity test file refused on purpose, and a link that splits a name from its footnote number |
+| List numbers | ordinal styles shown as plain numbers, some continued and restarted lists numbered differently | match LibreOffice |
+| Lists inside table cells | numbers dropped | numbers kept |
+| Bold, italics and links | kept | kept |
+| Other formats | PDF, PowerPoint, Excel, OpenDocument, RTF, EPUB, CSV and more | none |
+| Median time per file | 0.2 ms | 0.5 ms |
+| 10,000 crafted `.doc` files | no crash or hang | no crash or hang |
+| Use from | Rust, Node, Python, WebAssembly, command line, MCP | Ruby, command line |
+
+For Word files, readword keeps more of the text and the list numbers Word shows. anydoc is the better pick for other formats or when speed matters most.
+
+### Compared with antiword and the docx gem
+
+| | antiword 0.37 | docx gem 0.13 | readword 0.1 |
 | --- | --- | --- | --- |
 | Reads `.doc` | yes | no | yes |
 | Reads `.docx` | no | yes | yes |
@@ -108,7 +136,7 @@ readword was compared with Apache Tika, LibreOffice and macOS textutil on 650 Wo
 | A 1 MB zip that unpacks to 1 GB | | 3.2 GB of memory used | refused in 0.17 s |
 | Install | system package (no Homebrew formula) | gem | gem |
 
-The words readword misses are field codes, embedded objects, charts and picture names.
+This comparison was made earlier, with readword 0.1 and a smaller set of the test files. The words readword misses are field codes, embedded objects, charts and picture names.
 
 List numbers were compared with LibreOffice's on the 45 test files that have numbered lists. All numbers match in 41 of them. Of the other four, two are deliberately corrupted fuzz files, one changed its numbering with Track Changes, and one numbers an empty paragraph that holds a section break, where LibreOffice drops the paragraph.
 

data/exe/readword

@@ -4,6 +4,8 @@
 require 'optparse'
 require 'readword'
 
+Signal.trap('PIPE', 'SYSTEM_DEFAULT')
+
 options = {}
 parser = OptionParser.new do |opts|
   opts.banner = 'Usage: readword [options] FILE...'
@@ -13,6 +15,10 @@ parser = OptionParser.new do |opts|
   opts.on('--max-characters N', Integer, 'Fail when a document has more than N characters') do |count|
     options[:max_characters] = count
   end
+  opts.on('--max-size MB', Integer,
+          "Fail when a document unpacks to more than MB megabytes (default #{Readword::DEFAULT_MAX_SIZE}, 0 for no limit)") do |megabytes|
+    options[:max_size] = megabytes
+  end
   opts.on('-v', '--version', 'Print the version') do
     puts Readword::VERSION
     exit

data/lib/readword/doc.rb

@@ -40,15 +40,18 @@ module Readword
     MAX_PAPX_RUNS_PER_PAGE = 0x1D
     MAX_CHPX_RUNS_PER_PAGE = 0x65
     # Limits against crafted files, so the reader cannot be made to work forever.
-    MAX_PROPERTY_BYTES = 8 * 1024 * 1024
-    MAX_TEXT_POSITIONS = 400_000
+    TEXT_POSITIONS_PER_MEGABYTE = 40_000
+    PROPERTY_BYTES_PER_MEGABYTE = 800 * 1024
     NO_FLAGS = {}.freeze
 
     # Paragraph sprms: in a table, end of a table row, and the same two for a table nested in a cell.
     PARAGRAPH_SPRMS = { 0x2416 => :in_table, 0x2417 => :row_end, 0x244B => :inner_cell, 0x244C => :inner_row_end }.freeze
-    # sprmCFRMarkDel: deleted with Track Changes. Word writes 1, or 0x81, a ToggleOperand meaning the opposite of the style.
+    # sprmCFRMarkDel (deleted with Track Changes), sprmCFBold and sprmCFItalic. Their ToggleOperand is 1, or 0x81 for the opposite of the style.
     DELETED = 0x0800
-    DELETED_VALUES = [0x01, 0x81].freeze
+    BOLD = 0x0835
+    ITALIC = 0x0836
+    ON_VALUES = [0x01, 0x81].freeze
+    PLAIN = [false, false, nil].freeze
     # sprmCSymbol: Insert Symbol stores "(" in the text, and the font number and real character here.
     SYMBOL = 0x6A09
     SYMBOL_PLACEHOLDER = '('
@@ -89,7 +92,9 @@ module Readword
       byte.chr.force_encoding(Encoding::Windows_1252).encode(Encoding::UTF_8, undef: :replace, replace: '')
     end.freeze
 
-    def initialize(bytes)
+    def initialize(bytes, max_size: DEFAULT_MAX_SIZE)
+      @max_text_positions = max_size ? max_size * TEXT_POSITIONS_PER_MEGABYTE : Float::INFINITY
+      @max_property_bytes = max_size ? max_size * PROPERTY_BYTES_PER_MEGABYTE : Float::INFINITY
       @file = CompoundFile.new(bytes)
       @word = @file.stream('WordDocument').to_s
       raise Error, PASSWORD_PROTECTED if @word.empty? && @file.stream?('EncryptedPackage')
@@ -102,7 +107,7 @@ module Readword
 
     def blocks
       pieces = text_pieces
-      raise Error, TOO_LONG if pieces.sum { |piece| [piece.finish - piece.start, 0].max } > MAX_TEXT_POSITIONS
+      raise Error, TOO_LONG if pieces.sum { |piece| [piece.finish - piece.start, 0].max } > @max_text_positions
 
       @property_bytes = 0
       @runs = paragraph_runs
@@ -112,12 +117,13 @@ module Readword
       @chains = {}
       @paragraph_start = 0
       @row = []
-      @text = +''
+      @text = []
+      @format = PLAIN
       read_characters(pieces)
       if @row.any?
         end_row
       else
-        @blocks << Paragraph.new(text: @text)
+        @blocks << Paragraph.new(text: Inline.render(@text))
       end
       @blocks
     end
@@ -128,28 +134,58 @@ module Readword
     def read_characters(pieces)
       fields = []
       code_depth = 0
+      link = nil
       each_character(pieces) do |character, offset, width|
-        marks = @marked.empty? ? {} : run_flags(@marked, offset)
+        marks = @marked.empty? ? NO_FLAGS : run_flags(@marked, offset)
         next if marks[:deleted]
 
         character = font_symbol(*marks[:symbol]) if marks[:symbol] && character == SYMBOL_PLACEHOLDER
         case character
         when FIELD_BEGIN
-          fields.push(:code)
+          fields.push({ state: :code, code: +'' })
           code_depth += 1
         when FIELD_SEPARATOR
-          if fields.last == :code
-            fields[-1] = :result
+          field = fields.last
+          if field && field[:state] == :code
+            field.merge!(state: :result, link: Readword.hyperlink(field[:code]))
             code_depth -= 1
           end
+          link = field_link(fields)
         when FIELD_END
-          code_depth -= 1 if fields.pop == :code
+          code_depth -= 1 if fields.pop&.fetch(:state) == :code
+          link = field_link(fields)
         else
-          read_character(character, offset, width) if code_depth.zero?
+          if code_depth.zero?
+            @format = text_format(marks, link)
+            read_character(character, offset, width)
+          else
+            fields.reverse_each.find { |field| field[:state] == :code }&.fetch(:code)&.<<(character)
+          end
         end
       end
     end
 
+    def field_link(fields)
+      fields.reverse_each { |field| return field[:link] if field[:link] }
+      nil
+    end
+
+    def text_format(marks, link)
+      bold = marks[:bold] == true
+      italic = marks[:italic] == true
+      bold == @format[0] && italic == @format[1] && link == @format[2] ? @format : [bold, italic, link]
+    end
+
+    def append(text, format = PLAIN)
+      last = @text.last
+      bold, italic, link = format
+      if last && @text.size > @paragraph_start && last.bold == bold && last.italic == italic && last.link == link
+        last.text << text
+      else
+        @text << Span.new(text.dup, bold, italic, link)
+      end
+    end
+
     # A page or column break ends its paragraph only when it closes the paragraph's run, as a section break does.
     def read_character(character, offset, width)
       if character == CELL_MARK
@@ -158,9 +194,9 @@ module Readword
       elsif character == PARAGRAPH_MARK || (PARAGRAPH_ENDS.include?(character) && paragraph_end?(offset, width))
         end_paragraph(paragraph_flags(offset))
       elsif PARAGRAPH_ENDS.include?(character)
-        @text << "\n"
+        append("\n", @format)
       elsif character >= ' ' || REPLACEMENTS.key?(character)
-        @text << REPLACEMENTS.fetch(character, character)
+        append(REPLACEMENTS.fetch(character, character), @format)
       end
     end
 
@@ -169,34 +205,35 @@ module Readword
       elsif flags[:inner_cell] then end_cell(flags)
       elsif flags[:in_table]
         label_paragraph(flags)
-        @text << ' '
+        append(' ')
         @paragraph_start = @text.size
       else
         end_row if @row.any?
-        @blocks << Paragraph.new(text: @text, heading: heading_level(flags), list: list_item(flags))
-        @text = +''
+        heading = heading_level(flags)
+        @blocks << Paragraph.new(text: Inline.render(@text, bold: !heading), heading: heading, list: list_item(flags))
+        @text = []
         @paragraph_start = 0
       end
     end
 
     def end_cell(flags = nil)
       label_paragraph(flags) if flags
-      @row << Readword.squish(@text)
-      @text = +''
+      @row << Readword.squish(Inline.render(@text))
+      @text = []
       @paragraph_start = 0
     end
 
     def end_row
-      end_cell unless Readword.blank?(@text)
+      end_cell unless Readword.blank?(@text.map(&:text).join)
       @blocks << Row.new(@row) unless @row.all? { |cell| Readword.blank?(cell) }
       @row = []
-      @text = +''
+      @text = []
       @paragraph_start = 0
     end
 
     def label_paragraph(flags)
       item = list_item(flags) or return
-      @text.insert(@paragraph_start, "#{item.bullet ? '•' : item.label} ")
+      @text.insert(@paragraph_start, Span.new("#{item.bullet ? '•' : item.label} ", false, false, nil))
     end
 
     def heading_level(flags)
@@ -351,7 +388,7 @@ module Readword
       table = clx.byteslice(position + 5, u32(clx, position + 1).to_i).to_s
       count = (table.bytesize - 4) / 12
       raise Error, UNREADABLE unless count.positive?
-      raise Error, TOO_LONG if count > MAX_TEXT_POSITIONS
+      raise Error, TOO_LONG if count > @max_text_positions
 
       positions = table.unpack("V#{count + 1}")
       Array.new(count) do |index|
@@ -366,7 +403,7 @@ module Readword
       runs = []
       property_pages(FIB_PAPX_TABLE, FIB_PAPX_TABLE_BYTES).each do |offset|
         runs.concat(page_runs(offset))
-        raise Error, TOO_LONG if runs.size > MAX_TEXT_POSITIONS
+        raise Error, TOO_LONG if runs.size > @max_text_positions
       end
       runs.sort_by(&:start)
     end
@@ -374,8 +411,8 @@ module Readword
     def marked_runs
       runs = []
       property_pages(FIB_CHPX_TABLE, FIB_CHPX_TABLE_BYTES).each do |offset|
-        runs.concat(character_runs(offset).select { |run| run.flags[:deleted] || run.flags[:symbol] })
-        raise Error, TOO_LONG if runs.size > MAX_TEXT_POSITIONS
+        runs.concat(character_runs(offset).select { |run| run.flags.values_at(:deleted, :symbol, :bold, :italic).any? })
+        raise Error, TOO_LONG if runs.size > @max_text_positions
       end
       runs.sort_by(&:start)
     end
@@ -436,7 +473,7 @@ module Readword
 
     def sprm_flags(grpprl, nested: false)
       @property_bytes += grpprl.bytesize
-      raise Error, UNREADABLE if @property_bytes > MAX_PROPERTY_BYTES
+      raise Error, UNREADABLE if @property_bytes > @max_property_bytes
 
       flags = {}
       position = 0
@@ -445,7 +482,9 @@ module Readword
         size = operand_size(sprm, grpprl, position + 2) or break
         operand = grpprl.byteslice(position + 2, size).to_s
         if PARAGRAPH_SPRMS.key?(sprm) then flags[PARAGRAPH_SPRMS[sprm]] = operand.getbyte(0) == 1
-        elsif sprm == DELETED then flags[:deleted] = DELETED_VALUES.include?(operand.getbyte(0))
+        elsif sprm == DELETED then flags[:deleted] = ON_VALUES.include?(operand.getbyte(0))
+        elsif sprm == BOLD then flags[:bold] = ON_VALUES.include?(operand.getbyte(0))
+        elsif sprm == ITALIC then flags[:italic] = ON_VALUES.include?(operand.getbyte(0))
         elsif sprm == SYMBOL then flags[:symbol] = [u16(operand, 0).to_i, u16(operand, 2).to_i]
         elsif sprm == OUTLINE_LEVEL then flags[:outline] = operand.getbyte(0)
         elsif sprm == LIST_LEVEL then flags[:list_level] = operand.getbyte(0)
@@ -496,10 +535,10 @@ module Readword
 
     def run_flags(runs, offset)
       index = (runs.bsearch_index { |run| run.start > offset } || runs.size) - 1
-      return {} if index.negative?
+      return NO_FLAGS if index.negative?
 
       run = runs[index]
-      offset < run.finish ? run.flags : {}
+      offset < run.finish ? run.flags : NO_FLAGS
     end
 
     def font_symbol(font, code)

data/lib/readword/docx.rb

@@ -5,14 +5,17 @@ require 'zip'
 module Readword
   # Reads the text of a Word 2007 or later .docx (ECMA-376 WordprocessingML) from its XML parts.
   class Docx
-    MAX_XML_BYTES = 10 * 1024 * 1024
-    MAX_XML_NODES = 400_000
+    NODES_PER_MEGABYTE = 40_000
     MAX_GRID_SPAN = 63
     MAX_STYLE_DEPTH = 16
     UTF16_STARTS = ["\xFF\xFE".b, "\xFE\xFF".b, "<\0".b, "\0<".b].freeze
     RELATIONSHIP = %r{/relationships/(officeDocument|header|footer|footnotes|endnotes|styles|numbering)\z}
     SKIPPED = %w[del moveFrom instrText delText rPr pPr sectPr tblPr trPr tcPr sdtPr].freeze
     CHARACTERS = %w[t tab ptab br cr noBreakHyphen sym].freeze
+    OFF = %w[0 false off].freeze
+    LINK = %r{/relationships/hyperlink\z}
+    Format = Struct.new(:bold, :italic, :link)
+    PLAIN = Format.new(false, false, nil).freeze
     SEPARATOR_NOTES = %w[separator continuationSeparator continuationNotice].freeze
     HEADING_STYLE = /\Aheading ([1-9])\z/i
     TITLE_STYLE = /\Atitle\z/i
@@ -21,8 +24,10 @@ module Readword
                 'lowerLetter' => :lower_letter, 'decimalZero' => :decimal_zero, 'ordinal' => :ordinal,
                 'bullet' => :bullet, 'none' => :none }.freeze
 
-    def initialize(bytes)
+    def initialize(bytes, max_size: DEFAULT_MAX_SIZE)
       @bytes = bytes
+      @max_xml_bytes = max_size ? max_size * MEGABYTE : Float::INFINITY
+      @max_xml_nodes = max_size ? max_size * NODES_PER_MEGABYTE : Float::INFINITY
     end
 
     def blocks
@@ -73,10 +78,10 @@ module Readword
 
     def xml(path, budget = :content)
       entry = @entries[path.downcase] or return
-      remaining = MAX_XML_BYTES - @budgets[budget][:bytes]
+      remaining = @max_xml_bytes - @budgets[budget][:bytes]
       raise Error, TOO_LONG if entry.size > remaining
 
-      data = entry.get_input_stream { |stream| stream.read(remaining + 1) }.to_s
+      data = entry.get_input_stream { |stream| stream.read(remaining.finite? ? remaining + 1 : nil) }.to_s
       raise Error, TOO_LONG if data.bytesize > remaining
 
       @budgets[budget][:bytes] += data.bytesize
@@ -93,6 +98,7 @@ module Readword
 
     def part(path)
       @counter.reset
+      @links = link_targets(path)
       content(xml(path))
     end
 
@@ -103,6 +109,7 @@ module Readword
     def notes(paths)
       paths.flat_map do |path|
         @counter.reset
+        @links = link_targets(path)
         found = Array(xml(path)&.xpath('//*[self::footnote or self::endnote]'))
         found.reject { |note| SEPARATOR_NOTES.include?(note['type']) }.flat_map { |note| content(note) }
       end
@@ -124,13 +131,25 @@ module Readword
 
     def paragraph_blocks(paragraph)
       boxes = []
-      text = inline(paragraph, boxes, [])
+      spans = []
       properties = child_named(paragraph, 'pPr')
       style = value(properties, 'pStyle')
-      block = Paragraph.new(text: text, heading: heading_level(properties, style), list: list_item(properties, style))
+      heading = heading_level(properties, style)
+      inline(paragraph, boxes, [], spans, PLAIN)
+      block = Paragraph.new(text: Inline.render(spans, bold: !heading), heading: heading, list: list_item(properties, style))
       [block, *boxes.flat_map { |box| content(box) }]
     end
 
+    def link_targets(path)
+      document = path && xml(File.join(File.dirname(path), '_rels', "#{File.basename(path)}.rels"))
+      return {} unless document
+
+      document.xpath('//Relationship').each_with_object({}) do |relationship, links|
+        target = relationship['Target'].to_s.scrub('')
+        links[relationship['Id']] = target if LINK.match?(relationship['Type'].to_s) && WEB_ADDRESS.match?(target)
+      end
+    end
+
     def heading_level(properties, style)
       chain = style_chain(style)
       outline = value(properties, 'outlineLvl') || chain.filter_map { |entry| entry[:outline] }.first
@@ -170,8 +189,13 @@ module Readword
     end
 
     def read_styles(document)
+      @character_styles = {}
       return {} unless document
 
+      @character_styles = document.xpath('//style[@type="character"]').to_h do |style|
+        properties = child_named(style, 'rPr')
+        [style['styleId'], { bold: flag(properties, 'b', false), italic: flag(properties, 'i', false) }]
+      end
       document.xpath('//style[@type="paragraph"]').to_h do |style|
         properties = child_named(style, 'pPr')
         numbering = child_named(properties, 'numPr')
@@ -212,19 +236,47 @@ module Readword
       text.split(/(%[1-9])/).reject(&:empty?).map { |part| LEVEL_REFERENCE.match?(part) ? part[1].to_i - 1 : part }
     end
 
-    def inline(node, boxes, fields)
-      node.element_children.map do |child|
+    def inline(node, boxes, fields, spans, format)
+      node.element_children.each do |child|
         case child.name
-        when *CHARACTERS then fields.include?(:code) ? '' : character(child)
+        when *CHARACTERS then spans << Span.new(character(child), format.bold, format.italic, format.link || field_link(fields)) unless code?(fields)
+        when 'instrText' then fields.last[:code] << child.text.scrub('') if code?(fields)
         when 'fldChar' then field_char(child['fldCharType'], fields)
-        when 'txbxContent'
-          boxes << child
-          ''
-        when 'AlternateContent' then inline(alternative(child), boxes, fields)
-        when *SKIPPED then ''
-        else inline(child, boxes, fields)
+        when 'r' then inline(child, boxes, fields, spans, run_format(child, format))
+        when 'hyperlink' then inline(child, boxes, fields, spans, link_format(child, format))
+        when 'txbxContent' then boxes << child
+        when 'AlternateContent' then inline(alternative(child), boxes, fields, spans, format)
+        when *SKIPPED then nil
+        else inline(child, boxes, fields, spans, format)
         end
-      end.join
+      end
+    end
+
+    def run_format(run, format)
+      properties = child_named(run, 'rPr')
+      return format unless properties
+
+      style = @character_styles[value(properties, 'rStyle')] || {}
+      Format.new(flag(properties, 'b', style[:bold]), flag(properties, 'i', style[:italic]), format.link)
+    end
+
+    def flag(properties, name, inherited)
+      node = child_named(properties, name)
+      node ? !OFF.include?(node['val'].to_s.downcase) : inherited || false
+    end
+
+    def link_format(node, format)
+      target = @links[node['id'].to_s]
+      target ? Format.new(format.bold, format.italic, target) : format
+    end
+
+    def code?(fields)
+      fields.any? { |field| field[:state] == :code }
+    end
+
+    def field_link(fields)
+      fields.reverse_each { |field| return field[:link] if field[:state] == :result && field[:link] }
+      nil
     end
 
     def character(node)
@@ -239,11 +291,12 @@ module Readword
 
     def field_char(type, fields)
       case type
-      when 'begin' then fields.push(:code)
-      when 'separate' then fields[-1] = :result if fields.last == :code
+      when 'begin' then fields.push({ state: :code, code: +'' })
+      when 'separate'
+        field = fields.last
+        field.merge!(state: :result, link: Readword.hyperlink(field[:code])) if field && field[:state] == :code
       when 'end' then fields.pop
       end
-      ''
     end
 
     def alternative(node)
@@ -271,7 +324,7 @@ module Readword
 
     def count_nodes(count, budget = :content)
       @budgets[budget][:nodes] += count
-      raise Error, TOO_LONG if @budgets[budget][:nodes] > MAX_XML_NODES
+      raise Error, TOO_LONG if @budgets[budget][:nodes] > @max_xml_nodes
     end
 
     def children_named(node, name)

data/lib/readword/inline.rb

@@ -0,0 +1,101 @@
+# frozen_string_literal: true
+
+module Readword
+  # Writes formatted pieces of a paragraph as Markdown: bold, italics and links, opening and closing markers only where the
+  # formatting changes, keeping spaces outside the markers and ignoring emphasis that starts or ends inside a word.
+  module Inline
+    EDGE_SPACE = /\A[[:space:]]+|[[:space:]]+\z/
+    LEADING_SPACE = /\A[[:space:]]*/
+    TRAILING_SPACE = /[[:space:]]*\z/
+    WORD_START = /\A[[:alnum:]]/
+    WORD_END = /[[:alnum:]]\z/
+
+    module_function
+
+    def render(spans, bold: true)
+      return spans.map(&:text).join unless spans.any? { |span| (bold && span.bold) || span.italic || span.link }
+
+      pieces = merge(spans.map { |span| [span.text, bold && span.bold, span.italic, span.link] })
+      pieces = merge(pieces.each_with_index.map { |piece, index| inside_word?(pieces, index) ? [piece[0], false, false, piece[3]] : piece })
+      state = [false, false]
+      held = +''
+      text = +''
+      pieces.each do |content, strong, emphasis, link|
+        if link
+          text << close(state) << held << decorate(content, strong, emphasis, link)
+          held = +''
+          next
+        end
+        core = content.gsub(EDGE_SPACE, '')
+        next held << content if core.empty?
+
+        core = core.gsub('*', '\\*') if strong || emphasis
+        text << switch(state, strong, emphasis, held + content[LEADING_SPACE]) << core
+        held = +content[TRAILING_SPACE]
+      end
+      text << close(state) << held
+    end
+
+    def merge(pieces)
+      pieces.each_with_object([]) do |piece, merged|
+        next merged.last[0] += piece[0] if merged.last && merged.last[1..] == piece[1..]
+
+        merged << [piece[0].dup, *piece[1..]]
+      end
+    end
+
+    def inside_word?(pieces, index)
+      content, strong, emphasis, link = pieces[index]
+      return false unless (strong || emphasis) && !link
+
+      before = pieces[index - 1]&.first if index.positive?
+      after = pieces[index + 1]&.first
+      (before&.match?(WORD_END) && content.match?(WORD_START)) || (after&.match?(WORD_START) && content.match?(WORD_END))
+    end
+
+    def switch(state, strong, emphasis, gap)
+      bold_open, italic_open = state
+      closing = +''
+      opening = +''
+      if italic_open && (!emphasis || (bold_open && !strong) || (strong && !bold_open))
+        closing << '*'
+        italic_open = false
+      end
+      if bold_open && !strong
+        closing << '**'
+        bold_open = false
+      end
+      if strong && !bold_open
+        opening << '**'
+        bold_open = true
+      end
+      if emphasis && !italic_open
+        opening << '*'
+        italic_open = true
+      end
+      state.replace([bold_open, italic_open])
+      "#{closing}#{gap}#{opening}"
+    end
+
+    def close(state)
+      markers = "#{'*' if state[1]}#{'**' if state[0]}"
+      state.replace([false, false])
+      markers
+    end
+
+    def decorate(text, strong, emphasis, link)
+      core = text.gsub(EDGE_SPACE, '')
+      return text if core.empty? || !(strong || emphasis || link)
+
+      core = core.gsub('*', '\\*') if strong || emphasis
+      marker = if strong && emphasis then '***'
+               elsif strong then '**'
+               elsif emphasis then '*'
+               else ''
+               end
+      core = "#{marker}#{core}#{marker}"
+      core = "[#{core}](#{link.match?(/[[:space:]()<>]/) ? "<#{link.delete('<>')}>" : link})" if link
+      "#{text[LEADING_SPACE]}#{core}#{text[TRAILING_SPACE]}"
+    end
+  end
+end

data/lib/readword/markdown.rb

@@ -4,6 +4,7 @@ module Readword
   Paragraph = Struct.new(:text, :heading, :list, keyword_init: true)
   ListItem = Struct.new(:level, :label, :bullet, :list, keyword_init: true)
   Row = Struct.new(:cells)
+  Span = Struct.new(:text, :bold, :italic, :link)
 
   # Writes the readers' paragraphs and table rows as Markdown: headings, nested lists and pipe tables.
   module Markdown
@@ -73,6 +74,8 @@ module Readword
 
     def table(rows)
       width = rows.map(&:size).max
+      header, *body = rows
+      rows = [header.map { |cell| unbold(cell) }, *body]
       lines = rows.map do |cells|
         padded = cells.map { |cell| cell.gsub('|', '\|') } + Array.new(width - cells.size, '')
         "| #{padded.join(' | ')} |"
@@ -80,6 +83,11 @@ module Readword
       lines.insert(1, "| #{Array.new(width, '---').join(' | ')} |").join("\n")
     end
 
+    def unbold(cell)
+      inner = cell[/\A\*\*(.*)\*\*\z/m, 1]
+      inner && !inner.include?('**') ? inner : cell
+    end
+
     def escape(text)
       LITERAL_START.match?(text) ? "\\#{text}" : text
     end

data/lib/readword/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Readword
-  VERSION = '0.1.1'
+  VERSION = '0.2.0'
 end

data/lib/readword.rb

@@ -15,6 +15,10 @@ module Readword
     0xB0 => '°', 0xB1 => '±', 0xB3 => '≥', 0xB4 => '×', 0xB7 => '•', 0xB9 => '≠', 0xBB => '≈'
   }.freeze
   ZIP_SIGNATURE = "PK\x03\x04".b.freeze
+  DEFAULT_MAX_SIZE = 32
+  LINK_SWITCHES = ['\\l', '\\o', '\\t'].freeze
+  WEB_ADDRESS = %r{\A((https?|ftp)://|mailto:)[^[:cntrl:]]*\z}i
+  MEGABYTE = 1024 * 1024
 
   UNREADABLE = :unreadable
   NO_TEXT = :no_text
@@ -44,8 +48,9 @@ module Readword
     convert(File.binread(path), **options)
   end
 
-  def self.convert(bytes, max_characters: nil)
-    text = Markdown.render(reader(bytes.b).blocks).split("\n").map(&:rstrip).join("\n").gsub(/\n{3,}/, "\n\n").strip
+  def self.convert(bytes, max_characters: nil, max_size: DEFAULT_MAX_SIZE)
+    max_size = nil unless max_size.to_i.positive?
+    text = Markdown.render(reader(bytes.b, max_size).blocks).split("\n").map(&:rstrip).join("\n").gsub(/\n{3,}/, "\n\n").strip
     raise Error, NO_TEXT if text.empty?
     raise Error, TOO_LONG if max_characters && text.size > max_characters
 
@@ -56,9 +61,9 @@ module Readword
     raise Error, UNREADABLE
   end
 
-  def self.reader(bytes)
-    if bytes.start_with?(CompoundFile::SIGNATURE) then Doc.new(bytes)
-    elsif bytes.start_with?(ZIP_SIGNATURE) then Docx.new(bytes)
+  def self.reader(bytes, max_size)
+    if bytes.start_with?(CompoundFile::SIGNATURE) then Doc.new(bytes, max_size: max_size)
+    elsif bytes.start_with?(ZIP_SIGNATURE) then Docx.new(bytes, max_size: max_size)
     else raise Error, UNREADABLE
     end
   end
@@ -70,6 +75,22 @@ module Readword
     code.between?(0x20, 0xD7FF) ? code.chr(Encoding::UTF_8) : ''
   end
 
+  def self.hyperlink(code)
+    words = code.to_s.scan(/"[^"]*"|\S+/)
+    return unless words.shift&.casecmp?('HYPERLINK')
+
+    while (word = words.shift)
+      if word.start_with?('\\')
+        words.shift if LINK_SWITCHES.include?(word.downcase)
+        next
+      end
+      url = word.delete('"')
+      return url if WEB_ADDRESS.match?(url)
+
+      break
+    end
+  end
+
   def self.blank?(text)
     text.nil? || text.match?(/\A[[:space:]]*\z/)
   end
@@ -80,6 +101,7 @@ module Readword
 end
 
 require_relative 'readword/markdown'
+require_relative 'readword/inline'
 require_relative 'readword/numbering'
 require_relative 'readword/binary'
 require_relative 'readword/compound_file'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: readword
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.2.0
 platform: ruby
 authors:
 - urbullet
@@ -63,6 +63,7 @@ files:
 - lib/readword/compound_file.rb
 - lib/readword/doc.rb
 - lib/readword/docx.rb
+- lib/readword/inline.rb
 - lib/readword/markdown.rb
 - lib/readword/numbering.rb
 - lib/readword/version.rb