markdownator 0.1.1 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +21 -1
- data/Gemfile +8 -14
- data/README.md +14 -18
- data/lib/markdownator/converters/html.rb +2 -2
- data/lib/markdownator/converters/xlsx.rb +87 -20
- data/lib/markdownator/{converters → renderers}/html_renderer.rb +1 -1
- data/lib/markdownator/version.rb +1 -1
- data/markdownator.gemspec +5 -4
- metadata +60 -4
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: f3bb73d9071265c3e01fa7103402c07a8a7ebbcf48e765d52e81af03c5214279
|
|
4
|
+
data.tar.gz: 2a62788cffbb31953bdf6a9026da04fc6b34a2d96cf8b1c66ea7bf999e9409ce
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 806ca5e6247cdd8a3dc0d38d5c976c58adb40cfa5da9670df5c2f91fd0caec8fa73876dbbc77b7a27eb95ae1a275ed749c6c3ab239cb2067dcedf1eacd592738
|
|
7
|
+
data.tar.gz: 839c22421473002fd50a838a474bca86fcf3506e42fe5d742ccd444954f286babd993ba023043fc8087bf6733cbf70bddce79bcd45f1cd0f6bf1e6ba88ff89cf
|
data/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.1.3] - 2026-06-13
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
|
|
14
|
+
- `nokogiri`, `rubyzip`, `pdf-reader`, and `exifr` are now runtime dependencies,
|
|
15
|
+
so every supported format works out of the box instead of requiring callers to
|
|
16
|
+
install the format gems themselves.
|
|
17
|
+
|
|
18
|
+
## [0.1.2] - 2026-06-13
|
|
19
|
+
|
|
20
|
+
### Changed
|
|
21
|
+
|
|
22
|
+
- XLSX conversion now reads the workbook directly with rubyzip and Nokogiri
|
|
23
|
+
instead of `roo`, so every Office format (DOCX, XLSX, PPTX, EPUB) shares one
|
|
24
|
+
approach and the `roo` dependency is dropped.
|
|
25
|
+
- Moved the HTML renderer to `Markdownator::Renderers::HtmlRenderer` (it renders
|
|
26
|
+
Markdown, it does not convert a source file).
|
|
27
|
+
|
|
10
28
|
## [0.1.1] - 2026-06-13
|
|
11
29
|
|
|
12
30
|
### Changed
|
|
@@ -29,6 +47,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
29
47
|
dependencies.
|
|
30
48
|
- Pluggable LLM image-captioner hook (off by default).
|
|
31
49
|
|
|
32
|
-
[Unreleased]: https://github.com/alexrupom/markdownator/compare/v0.1.
|
|
50
|
+
[Unreleased]: https://github.com/alexrupom/markdownator/compare/v0.1.3...HEAD
|
|
51
|
+
[0.1.3]: https://github.com/alexrupom/markdownator/compare/v0.1.2...v0.1.3
|
|
52
|
+
[0.1.2]: https://github.com/alexrupom/markdownator/compare/v0.1.1...v0.1.2
|
|
33
53
|
[0.1.1]: https://github.com/alexrupom/markdownator/compare/v0.1.0...v0.1.1
|
|
34
54
|
[0.1.0]: https://github.com/alexrupom/markdownator/releases/tag/v0.1.0
|
data/Gemfile
CHANGED
|
@@ -5,17 +5,11 @@ source "https://rubygems.org"
|
|
|
5
5
|
# Specify your gem's dependencies in markdownator.gemspec
|
|
6
6
|
gemspec
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
gem "
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
# gem itself requires them lazily and never depends on them at runtime.
|
|
17
|
-
gem "exifr", "~> 1.3"
|
|
18
|
-
gem "nokogiri", "~> 1.15"
|
|
19
|
-
gem "pdf-reader", "~> 2.12"
|
|
20
|
-
gem "roo", "~> 2.10"
|
|
21
|
-
gem "rubyzip", "~> 2.3"
|
|
8
|
+
# The format libraries are runtime dependencies declared in the gemspec.
|
|
9
|
+
|
|
10
|
+
# Development and test tooling.
|
|
11
|
+
group :development, :test do
|
|
12
|
+
gem "rake", "~> 13.0"
|
|
13
|
+
gem "rspec", "~> 3.0"
|
|
14
|
+
gem "rubocop", "~> 1.21"
|
|
15
|
+
end
|
data/README.md
CHANGED
|
@@ -3,28 +3,28 @@
|
|
|
3
3
|
Convert files into clean, LLM-friendly **Markdown**. Point Markdownator at a PDF,
|
|
4
4
|
Office document, web page, archive, or image and get Markdown back.
|
|
5
5
|
|
|
6
|
-
It uses a pluggable converter-registry architecture and
|
|
7
|
-
|
|
6
|
+
It uses a pluggable converter-registry architecture, and ships with everything it
|
|
7
|
+
needs — every supported format works out of the box.
|
|
8
8
|
|
|
9
9
|
## Supported formats
|
|
10
10
|
|
|
11
|
-
| Format | Extensions |
|
|
12
|
-
|
|
13
|
-
| Plain text / Markdown | `.txt`, `.md` |
|
|
14
|
-
| CSV | `.csv` |
|
|
15
|
-
| JSON | `.json` |
|
|
11
|
+
| Format | Extensions | Backed by |
|
|
12
|
+
|--------|------------|-----------|
|
|
13
|
+
| Plain text / Markdown | `.txt`, `.md` | stdlib |
|
|
14
|
+
| CSV | `.csv` | stdlib |
|
|
15
|
+
| JSON | `.json` | stdlib |
|
|
16
16
|
| HTML | `.html`, `.htm` | `nokogiri` |
|
|
17
17
|
| XML | `.xml` | `nokogiri` |
|
|
18
18
|
| Word | `.docx` | `rubyzip`, `nokogiri` |
|
|
19
|
-
| Excel | `.xlsx` | `
|
|
19
|
+
| Excel | `.xlsx` | `rubyzip`, `nokogiri` |
|
|
20
20
|
| PowerPoint | `.pptx` | `rubyzip`, `nokogiri` |
|
|
21
21
|
| PDF | `.pdf` | `pdf-reader` |
|
|
22
22
|
| EPUB | `.epub` | `rubyzip`, `nokogiri` |
|
|
23
23
|
| ZIP (recurses) | `.zip` | `rubyzip` |
|
|
24
|
-
| Images (metadata) | `.jpg`, `.png`, `.tiff`, … | `exifr`
|
|
24
|
+
| Images (metadata) | `.jpg`, `.png`, `.tiff`, … | `exifr` |
|
|
25
25
|
|
|
26
|
-
|
|
27
|
-
|
|
26
|
+
These libraries (`nokogiri`, `rubyzip`, `pdf-reader`, `exifr`) are runtime
|
|
27
|
+
dependencies, so they are installed with the gem automatically.
|
|
28
28
|
|
|
29
29
|
## Installation
|
|
30
30
|
|
|
@@ -32,14 +32,10 @@ telling you exactly what to add to your `Gemfile`.
|
|
|
32
32
|
gem "markdownator"
|
|
33
33
|
```
|
|
34
34
|
|
|
35
|
-
|
|
35
|
+
or:
|
|
36
36
|
|
|
37
|
-
```
|
|
38
|
-
gem
|
|
39
|
-
gem "roo" # XLSX
|
|
40
|
-
gem "rubyzip" # DOCX, PPTX, EPUB, ZIP
|
|
41
|
-
gem "nokogiri" # HTML, XML, DOCX, PPTX, EPUB
|
|
42
|
-
gem "exifr" # image EXIF
|
|
37
|
+
```sh
|
|
38
|
+
gem install markdownator
|
|
43
39
|
```
|
|
44
40
|
|
|
45
41
|
## Usage
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
require_relative "html_renderer"
|
|
3
|
+
require_relative "../renderers/html_renderer"
|
|
4
4
|
|
|
5
5
|
module Markdownator
|
|
6
6
|
module Converters
|
|
@@ -20,7 +20,7 @@ module Markdownator
|
|
|
20
20
|
Markdownator.require_optional("nokogiri", feature: "HTML conversion")
|
|
21
21
|
doc = Nokogiri::HTML(html)
|
|
22
22
|
root = doc.at_css("body") || doc.root || doc
|
|
23
|
-
HtmlRenderer.new.render(root)
|
|
23
|
+
Renderers::HtmlRenderer.new.render(root)
|
|
24
24
|
end
|
|
25
25
|
|
|
26
26
|
def self.extract_title(html)
|
|
@@ -1,11 +1,12 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
require "tempfile"
|
|
4
|
-
|
|
5
3
|
module Markdownator
|
|
6
4
|
module Converters
|
|
7
5
|
# Converts an Excel .xlsx workbook into Markdown: one `## SheetName` heading
|
|
8
|
-
# and a Markdown table per sheet
|
|
6
|
+
# and a Markdown table per sheet.
|
|
7
|
+
#
|
|
8
|
+
# Reads the OOXML zip directly with rubyzip and Nokogiri, the same approach
|
|
9
|
+
# used by the DOCX, PPTX, and EPUB converters.
|
|
9
10
|
class Xlsx < Base
|
|
10
11
|
def accepts?(_io, stream_info)
|
|
11
12
|
matches?(
|
|
@@ -16,12 +17,16 @@ module Markdownator
|
|
|
16
17
|
end
|
|
17
18
|
|
|
18
19
|
def convert(io, _stream_info, **_options)
|
|
19
|
-
Markdownator.require_optional("
|
|
20
|
+
Markdownator.require_optional("zip", feature: "XLSX conversion")
|
|
21
|
+
Markdownator.require_optional("nokogiri", feature: "XLSX conversion")
|
|
20
22
|
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
sections =
|
|
24
|
-
|
|
23
|
+
::Zip::File.open_buffer(io) do |zip|
|
|
24
|
+
shared = shared_strings(zip)
|
|
25
|
+
sections = sheets(zip).filter_map do |name, path|
|
|
26
|
+
table = markdown_table(parse_sheet(read(zip, path), shared))
|
|
27
|
+
"## #{name}\n\n#{table}" unless table.empty?
|
|
28
|
+
end
|
|
29
|
+
return Result.new(markdown: sections.join("\n\n"))
|
|
25
30
|
end
|
|
26
31
|
rescue StandardError => e
|
|
27
32
|
raise FileConversionError, "Could not read XLSX: #{e.message}"
|
|
@@ -29,23 +34,85 @@ module Markdownator
|
|
|
29
34
|
|
|
30
35
|
private
|
|
31
36
|
|
|
32
|
-
def
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
37
|
+
def read(zip, path)
|
|
38
|
+
zip.find_entry(path)&.get_input_stream&.read
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
# Ordered [[sheet_name, worksheet_path], ...] resolved via the workbook
|
|
42
|
+
# relationships.
|
|
43
|
+
def sheets(zip)
|
|
44
|
+
workbook = parse(read(zip, "xl/workbook.xml"))
|
|
45
|
+
return [] if workbook.nil?
|
|
46
|
+
|
|
47
|
+
rels = relationships(zip)
|
|
48
|
+
workbook.xpath("//sheets/sheet").filter_map do |sheet|
|
|
49
|
+
path = resolve_target(rels[sheet["id"]])
|
|
50
|
+
[sheet["name"].to_s, path] if path
|
|
36
51
|
end
|
|
37
|
-
table = markdown_table(rows)
|
|
38
|
-
table.empty? ? nil : "## #{name}\n\n#{table}"
|
|
39
52
|
end
|
|
40
53
|
|
|
41
|
-
def
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
54
|
+
def relationships(zip)
|
|
55
|
+
doc = parse(read(zip, "xl/_rels/workbook.xml.rels"))
|
|
56
|
+
return {} if doc.nil?
|
|
57
|
+
|
|
58
|
+
doc.xpath("//Relationship").to_h { |rel| [rel["Id"], rel["Target"]] }
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
def resolve_target(target)
|
|
62
|
+
return nil if target.nil? || target.empty?
|
|
63
|
+
|
|
64
|
+
target = target.delete_prefix("/")
|
|
65
|
+
target.start_with?("xl/") ? target : "xl/#{target}"
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
# The shared string table: index -> text.
|
|
69
|
+
def shared_strings(zip)
|
|
70
|
+
doc = parse(read(zip, "xl/sharedStrings.xml"))
|
|
71
|
+
return [] if doc.nil?
|
|
72
|
+
|
|
73
|
+
doc.xpath("//si").map { |si| si.xpath(".//t").map(&:text).join }
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
def parse_sheet(xml, shared)
|
|
77
|
+
doc = parse(xml)
|
|
78
|
+
return [] if doc.nil?
|
|
79
|
+
|
|
80
|
+
doc.xpath("//sheetData/row").map do |row|
|
|
81
|
+
values = {}
|
|
82
|
+
width = 0
|
|
83
|
+
row.xpath("./c").each_with_index do |cell, position|
|
|
84
|
+
column = column_index(cell["r"]) || (position + 1)
|
|
85
|
+
width = column if column > width
|
|
86
|
+
values[column] = cell_value(cell, shared)
|
|
87
|
+
end
|
|
88
|
+
(1..width).map { |i| values[i] || "" }
|
|
89
|
+
end
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
def cell_value(cell, shared)
|
|
93
|
+
case cell["t"]
|
|
94
|
+
when "s" then shared[cell.at_xpath("./v")&.text.to_i].to_s
|
|
95
|
+
when "inlineStr" then cell.xpath("./is//t").map(&:text).join
|
|
96
|
+
when "b" then cell.at_xpath("./v")&.text == "1" ? "TRUE" : "FALSE"
|
|
97
|
+
else cell.at_xpath("./v")&.text.to_s
|
|
47
98
|
end
|
|
48
99
|
end
|
|
100
|
+
|
|
101
|
+
# Converts a cell reference like "B2" into a 1-based column index (2).
|
|
102
|
+
def column_index(ref)
|
|
103
|
+
letters = ref.to_s[/\A[A-Z]+/i]
|
|
104
|
+
return nil if letters.nil?
|
|
105
|
+
|
|
106
|
+
letters.upcase.each_char.reduce(0) { |acc, char| (acc * 26) + (char.ord - 64) }
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
def parse(xml)
|
|
110
|
+
return nil if xml.nil?
|
|
111
|
+
|
|
112
|
+
doc = Nokogiri::XML(xml)
|
|
113
|
+
doc.remove_namespaces!
|
|
114
|
+
doc
|
|
115
|
+
end
|
|
49
116
|
end
|
|
50
117
|
end
|
|
51
118
|
end
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
module Markdownator
|
|
4
|
-
module
|
|
4
|
+
module Renderers
|
|
5
5
|
# Walks a Nokogiri HTML node tree and renders Markdown. A focused,
|
|
6
6
|
# dependency-free replacement for reverse_markdown: HTML conversion needs
|
|
7
7
|
# only Nokogiri (which reverse_markdown depended on anyway).
|
data/lib/markdownator/version.rb
CHANGED
data/markdownator.gemspec
CHANGED
|
@@ -32,8 +32,9 @@ Gem::Specification.new do |spec|
|
|
|
32
32
|
spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
|
|
33
33
|
spec.require_paths = ["lib"]
|
|
34
34
|
|
|
35
|
-
#
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
35
|
+
# Format libraries used by the converters.
|
|
36
|
+
spec.add_dependency "exifr", "~> 1.3" # image metadata
|
|
37
|
+
spec.add_dependency "nokogiri", "~> 1.15" # HTML, XML, DOCX, XLSX, PPTX, EPUB
|
|
38
|
+
spec.add_dependency "pdf-reader", "~> 2.12" # PDF
|
|
39
|
+
spec.add_dependency "rubyzip", "~> 2.3" # DOCX, XLSX, PPTX, EPUB, ZIP
|
|
39
40
|
end
|
metadata
CHANGED
|
@@ -1,15 +1,71 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: markdownator
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.3
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- alexrupom
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: exe
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-06-
|
|
12
|
-
dependencies:
|
|
11
|
+
date: 2026-06-18 00:00:00.000000000 Z
|
|
12
|
+
dependencies:
|
|
13
|
+
- !ruby/object:Gem::Dependency
|
|
14
|
+
name: exifr
|
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
|
16
|
+
requirements:
|
|
17
|
+
- - "~>"
|
|
18
|
+
- !ruby/object:Gem::Version
|
|
19
|
+
version: '1.3'
|
|
20
|
+
type: :runtime
|
|
21
|
+
prerelease: false
|
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
23
|
+
requirements:
|
|
24
|
+
- - "~>"
|
|
25
|
+
- !ruby/object:Gem::Version
|
|
26
|
+
version: '1.3'
|
|
27
|
+
- !ruby/object:Gem::Dependency
|
|
28
|
+
name: nokogiri
|
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
|
30
|
+
requirements:
|
|
31
|
+
- - "~>"
|
|
32
|
+
- !ruby/object:Gem::Version
|
|
33
|
+
version: '1.15'
|
|
34
|
+
type: :runtime
|
|
35
|
+
prerelease: false
|
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
37
|
+
requirements:
|
|
38
|
+
- - "~>"
|
|
39
|
+
- !ruby/object:Gem::Version
|
|
40
|
+
version: '1.15'
|
|
41
|
+
- !ruby/object:Gem::Dependency
|
|
42
|
+
name: pdf-reader
|
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
|
44
|
+
requirements:
|
|
45
|
+
- - "~>"
|
|
46
|
+
- !ruby/object:Gem::Version
|
|
47
|
+
version: '2.12'
|
|
48
|
+
type: :runtime
|
|
49
|
+
prerelease: false
|
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
51
|
+
requirements:
|
|
52
|
+
- - "~>"
|
|
53
|
+
- !ruby/object:Gem::Version
|
|
54
|
+
version: '2.12'
|
|
55
|
+
- !ruby/object:Gem::Dependency
|
|
56
|
+
name: rubyzip
|
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
|
58
|
+
requirements:
|
|
59
|
+
- - "~>"
|
|
60
|
+
- !ruby/object:Gem::Version
|
|
61
|
+
version: '2.3'
|
|
62
|
+
type: :runtime
|
|
63
|
+
prerelease: false
|
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
65
|
+
requirements:
|
|
66
|
+
- - "~>"
|
|
67
|
+
- !ruby/object:Gem::Version
|
|
68
|
+
version: '2.3'
|
|
13
69
|
description: Markdownator converts PDF, Word, Excel, PowerPoint, EPUB, HTML, CSV,
|
|
14
70
|
JSON, XML, ZIP archives and images into clean Markdown suitable for large language
|
|
15
71
|
models, using a pluggable converter architecture.
|
|
@@ -33,7 +89,6 @@ files:
|
|
|
33
89
|
- lib/markdownator/converters/docx.rb
|
|
34
90
|
- lib/markdownator/converters/epub.rb
|
|
35
91
|
- lib/markdownator/converters/html.rb
|
|
36
|
-
- lib/markdownator/converters/html_renderer.rb
|
|
37
92
|
- lib/markdownator/converters/image.rb
|
|
38
93
|
- lib/markdownator/converters/json.rb
|
|
39
94
|
- lib/markdownator/converters/pdf.rb
|
|
@@ -44,6 +99,7 @@ files:
|
|
|
44
99
|
- lib/markdownator/converters/zip.rb
|
|
45
100
|
- lib/markdownator/engine.rb
|
|
46
101
|
- lib/markdownator/errors.rb
|
|
102
|
+
- lib/markdownator/renderers/html_renderer.rb
|
|
47
103
|
- lib/markdownator/result.rb
|
|
48
104
|
- lib/markdownator/stream_info.rb
|
|
49
105
|
- lib/markdownator/version.rb
|