pd4ml 1.1.1 → 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: eada212500d1ee924117e4c9b8f0ad4f8d29f7b69fb4ff6b0b1775caf3c23bfe
4
- data.tar.gz: c3f70c2b02f24e52517d95b4bb10ee2c81116230b9b8d356ccae0f7619b4bf74
3
+ metadata.gz: 7ace1bb99f70827a33726094f1da1657116b1b9b87c6819a228ce7210d265771
4
+ data.tar.gz: 7ff634c9640497ec5e62d91f173c51acee311016cf36ab46c0117dd6f6ee6fc7
5
5
  SHA512:
6
- metadata.gz: f6f7e9b5e9ff21c9db8a8cbae74770c766444afd2c9ce6123a08817691c9dfac6c941104a12cf56f2614d4274e858df788baaeb986a8ecd88d5c246128f0a8c1
7
- data.tar.gz: ad146b82b276a72480749a4f2f25e3254b982e8d1a3dfd4d065e202e06b61496c6b4e250abddf5aac44a5f29150259cdb0f57853ea132b3be0d75dc5740bafc9
6
+ metadata.gz: 98cf806f1c3ceb41355644d7b6644f3dd1977229a426bb5d77d7b8b3d5494da9597fb9509e095c24d2ec369e7b906fcc30092dcc794150c79141185de268ff04
7
+ data.tar.gz: 648ca15fc0bc61442c4761c7e0c5fdaf9e4feacde877a953cc85914788cfff34bd12299f5ec261a979ccd6f540111e76d43e6112f5d4409d74d02667bd5cd96e
data/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # pd4ml — PD4ML for Ruby
2
2
 
3
- HTML and CSS to **PDF**, **PDF/A**, **PDF/UA**, **ZUGFeRD/Factur-X**, **DOCX**, **RTF** and
3
+ HTML and CSS to **PDF**, **PDF/A**, **PDF/UA**, **ZUGFeRD/Factur-X**, **DOCX**, **RTF**, **Markdown** and
4
4
  **images** from Ruby, with the complete public [PD4ML](https://pd4ml.com) API. The v42 layout
5
5
  engine is the default.
6
6
 
@@ -57,7 +57,8 @@ The gem adds Ruby I/O on top: paths, URLs, binary Strings and IO objects in and
57
57
  for progress listeners. Several processes can share one bridge server: run `pd4ml serve` and set
58
58
  `PD4ML_BRIDGE=tcp://127.0.0.1:8723`.
59
59
 
60
- With PD4ML 4.2, the gem also reads e-mail messages and saved web pages (`read_eml`, `read_mht`),
60
+ With PD4ML 4.2, the gem also reads e-mail messages and saved web pages (`read_eml`, `read_mht`; DMS
61
+ and UA editions),
61
62
  and gives access to the parsed document before it is written (`get_document`, `layout`: the DOM
62
63
  API of the DMS and UA editions, whose lists come back as Ruby Arrays). See the
63
64
  [guide](docs/guide.md#516-e-mail-messages-and-saved-web-pages).
data/docs/api.md CHANGED
@@ -164,6 +164,8 @@ async_resource_loader(thread_pool_size: Integer) -> nil # asyncResourceLoader
164
164
  | `Pd4ml::Constants::PDFA` | String | static read-only |
165
165
  | `Pd4ml::Constants::PDFUA` | String | static read-only |
166
166
  | `Pd4ml::Constants::DOCX` | String | static read-only |
167
+ | `Pd4ml::Constants::MARKDOWN` | String | static read-only |
168
+ | `Pd4ml::Constants::PD4ML_DOCX_EMBED_FONTS` | String | static read-only |
167
169
  | `Pd4ml::Constants::RTF` | String | static read-only |
168
170
  | `Pd4ml::Constants::RTF_WMF` | String | static read-only |
169
171
  | `Pd4ml::Constants::PNG8` | String | static read-only |
data/docs/guide.md CHANGED
@@ -1,7 +1,7 @@
1
1
  # PD4ML for Ruby — User Guide
2
2
 
3
3
  `pd4ml` brings [PD4ML](https://pd4ml.com) to Ruby: it converts HTML and CSS into **PDF**,
4
- **PDF/A**, **PDF/UA**, **ZUGFeRD/Factur-X** invoices, **DOCX**, **RTF** and **page images**, and
4
+ **PDF/A**, **PDF/UA**, **ZUGFeRD/Factur-X** invoices, **DOCX**, **RTF**, **Markdown** and **page images**, and
5
5
  gives Ruby applications the complete public PD4ML API.
6
6
 
7
7
  PD4ML is a Java library. The gem runs it in a small Java *bridge* process that Ruby talks to.
@@ -192,7 +192,7 @@ puts pdf[0, 5] # %PDF-
192
192
  A document is **read once and written any number of times**:
193
193
 
194
194
  - `read_html` parses the document and lays it out;
195
- - each `write_pdf`, `write_docx`, `write_rtf` or `render_as_images` call then produces one output
195
+ - each `write_pdf`, `write_docx`, `write_rtf`, `write_markdown` or `render_as_images` call then produces one output
196
196
  from that same layout.
197
197
 
198
198
  Settings (page size, headers, fonts...) go before `read_html`. See
@@ -328,7 +328,7 @@ pd.read_html_string(html)
328
328
  pdf = pd.write_pdf # the PDF as a binary String
329
329
  ```
330
330
 
331
- `write_pdf`, `write_docx` and `write_rtf` return the document as a binary String when given no
331
+ `write_pdf`, `write_docx`, `write_rtf` and `write_markdown` return the document as a binary String when given no
332
332
  target. Given a path (a String or Pathname) they write the file, given an IO (a `File`, a
333
333
  `StringIO`, a Rack body) they write into it.
334
334
 
@@ -553,6 +553,25 @@ The v42 engine does not support the multi-page TIFF formats yet; use the v40 eng
553
553
  With the v42 engine, DOCX/RTF output is *editable*: positioned content becomes text boxes
554
554
  and form fields, not pictures.
555
555
 
556
+ **Markdown** for LLM, RAG and search pipelines (PD4ML 4.2.0 or newer, v42 engine): only the
557
+ text a reader sees, in reading order, with headings, lists, tables, links, footnotes and image
558
+ references. With `page_markers: true` each page starts with `<!-- page N -->`, the page of the
559
+ PDF the same conversion writes, so a chunk of text can be cited by page:
560
+
561
+ ```ruby
562
+ require 'pd4ml'
563
+
564
+ pd = Pd4ml::PD4ML.new
565
+ pd.read_html_string('<h1>Report</h1><p>Revenue grew by <b>12%</b>.</p>')
566
+ pd.write_pdf('report.pdf')
567
+ md = pd.to_markdown(page_markers: true) # a String: "<!-- page 1 -->\n\n# Report\n\n..."
568
+ pd.write_markdown('report.md') # or to an IO, or a binary String
569
+ ```
570
+
571
+ Markdown output is a feature of the DMS and UA editions: with another license, or none, the
572
+ Markdown carries evaluation notes. See
573
+ [HTML to Markdown for LLM and RAG Pipelines](https://pd4ml.com/docs/features/html-to-markdown-for-llm-and-rag-pipelines.html).
574
+
556
575
  ### 5.12 Interactive forms and multi-column layout
557
576
 
558
577
  ```ruby
@@ -652,6 +671,10 @@ and is reachable through `Pd4ml::Java.type(...)`.
652
671
 
653
672
  ### 5.16 E-mail messages and saved web pages
654
673
 
674
+ > **Editions.** E-mail and saved web page input — `read_eml`/`read_mht`, and `read_html` for input it
675
+ > recognizes as one of them — is part of the PD4ML **DMS and UA editions**. With an STD or PRO license it
676
+ > works, but the output carries the "PD4ML EVALUATION MODE" banners.
677
+
655
678
  PD4ML 4.2 reads e-mail messages (`.eml`) and web pages saved as a single file (`.mht`, `.mhtml`)
656
679
  and writes them to any output format:
657
680
 
@@ -853,7 +876,7 @@ Without a license, PD4ML is fully functional but marks its output. See
853
876
  always caused by unresolved relative links ([details](https://pd4ml.com/docs/#6.2-reading-the-source)).
854
877
  - **Settings go before `read_html`.** Page size, headers, fonts and styles apply to the layout
855
878
  `read_html` makes; writes afterwards only choose the output format.
856
- - **Write several formats from one read.** PDF, DOCX and page images from a single `read_html`
879
+ - **Write several formats from one read.** PDF, DOCX, Markdown and page images from a single `read_html`
857
880
  cost one layout.
858
881
  - **Binary in, binary out.** Pass content as a binary String (`File.binread`, `.b`) or an IO;
859
882
  results are binary Strings, ready for `File.binwrite` or `send_data`.
@@ -908,9 +931,10 @@ Without a license, PD4ML is fully functional but marks its output. See
908
931
  |---|---|
909
932
  | `read_html(source, base_url = nil, encoding = nil)` | `source`: a path or URL (String, Pathname, URI), a binary String or IO, or a Java `URL`/`InputStream` |
910
933
  | `read_html_string(html, base_url = nil)` | the document as a String |
911
- | `read_eml(source)`, `read_mht(source)` | an e-mail message, a saved web page ([5.16](#516-e-mail-messages-and-saved-web-pages)); `source` as for `read_html` |
934
+ | `read_eml(source)`, `read_mht(source)` | an e-mail message, a saved web page ([5.16](#516-e-mail-messages-and-saved-web-pages)), DMS and UA editions; `source` as for `read_html` |
912
935
  | `write_pdf(target = nil, spec: nil)` | `target`: a path or IO; none: returns a binary String. `spec`: a `PdfSpec`, a keyword (`'pdf/a-2b'`) or `'PDF'`/`'PDFA'`/`'PDFUA'` |
913
936
  | `write_docx(target = nil)`, `write_rtf(target = nil, rtf_wmf: false)` | as `write_pdf` |
937
+ | `write_markdown(target = nil, page_markers: false)`, `to_markdown(page_markers: false)` | Markdown, as `write_pdf` / as a String |
914
938
  | `render_as_images(format)`, `render_png` | pages as encoded images (binary Strings); `render_as_images(dir, name, format)` writes files |
915
939
  | `monitor_progress_with { \|message_id, progress, note, msec\| ... }` | a block, or a Java `ProgressListener` |
916
940
  | `use_html_renderer(engine, html_width: nil, params: nil)` | the layout engine by name; `pd.engine` tells which one is in use |
data/exe/pd4ml CHANGED
@@ -19,7 +19,7 @@ USAGE = <<~TEXT
19
19
  the versions in the PD4ML repository (* the installed one)
20
20
  version the PD4ML version the gem runs (downloads it on first use)
21
21
  serve [--port N] run a bridge server (default port 8723) for PD4ML_BRIDGE=tcp://127.0.0.1:N
22
- convert INPUT OUTPUT [--format pdf|docx|rtf|png] [--page-size A4] [--landscape]
22
+ convert INPUT OUTPUT [--format pdf|docx|rtf|md|png] [--page-size A4] [--landscape]
23
23
  [--margins 10,10,10,10] [--width PX] [--engine v42|v40] [--spec pdf/a-2b]
24
24
  [--footer HTML] [--bookmarks]
25
25
  convert a file or URL; the format follows OUTPUT's extension by default
@@ -73,6 +73,7 @@ def convert(args)
73
73
  when 'pdf' then pd.write_pdf(output, spec: opts[:spec])
74
74
  when 'docx' then pd.write_docx(output)
75
75
  when 'rtf' then pd.write_rtf(output)
76
+ when 'md', 'markdown' then pd.write_markdown(output, page_markers: true)
76
77
  when 'png'
77
78
  pages = pd.render_png
78
79
  if pages.size == 1
@@ -81,7 +82,7 @@ def convert(args)
81
82
  stem = output.sub(/\.png\z/i, '')
82
83
  pages.each_with_index { |p, i| File.binwrite("#{stem}-#{i + 1}.png", p) }
83
84
  end
84
- else raise ArgumentError, "unknown format '#{format}' (pdf, docx, rtf, png)"
85
+ else raise ArgumentError, "unknown format '#{format}' (pdf, docx, rtf, md, png)"
85
86
  end
86
87
  end
87
88
 
Binary file
data/lib/pd4ml/pd4ml.rb CHANGED
@@ -1,7 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Pd4ml
4
- # An HTML to PDF/DOCX/RTF/image converter: com.pd4ml.PD4ML with Ruby I/O and the v42 layout
4
+ # An HTML to PDF/DOCX/RTF/Markdown/image converter: com.pd4ml.PD4ML with Ruby I/O and the v42 layout
5
5
  # engine by default. Every Java method is available by its Java name or in Ruby style
6
6
  # (`set_page_size`, `add_style`, `use_ttf`); these take Ruby I/O: read_html (a path, URL or
7
7
  # binary String/IO), read_html_string, write_pdf/write_docx/write_rtf (the document returned as
@@ -166,6 +166,31 @@ module Pd4ml
166
166
  end
167
167
  alias writeRTF write_rtf
168
168
 
169
+ # Writes the document as Markdown (GitHub-flavored, UTF-8) for LLM, RAG and search pipelines:
170
+ # the text a reader sees, in reading order, with headings, lists, tables, links, footnotes and
171
+ # image references (v42 engine, PD4ML 4.2.0 or newer). page_markers: each page starts with
172
+ # `<!-- page N -->`, N being the page of the PDF the same conversion writes. A feature of the
173
+ # DMS and UA editions: with another license, or none, the Markdown carries evaluation notes.
174
+ # See write_pdf for `target`; to_markdown returns it as a String.
175
+ def write_markdown(target = nil, page_markers: false)
176
+ outs = []
177
+ begin
178
+ Bridge.get.call(@java, 'writeMarkdown', [Output.new(0), page_markers ? true : false], outs)
179
+ rescue StandardError => e
180
+ raise unless e.message.to_s.include?('NoSuchMethodException')
181
+
182
+ raise "write_markdown needs PD4ML 4.2.0 or newer (this is #{@java.getVersion}): " \
183
+ "run 'pd4ml install' to update"
184
+ end
185
+ Pd4ml.deliver(outs[0] || ''.b, target)
186
+ end
187
+ alias writeMarkdown write_markdown
188
+
189
+ # The document as a Markdown String, UTF-8 (see write_markdown).
190
+ def to_markdown(page_markers: false)
191
+ write_markdown(nil, page_markers: page_markers).force_encoding(Encoding::UTF_8)
192
+ end
193
+
169
194
  # render_as_images(format): the pages as encoded images (Constants::PNG24...), binary Strings;
170
195
  # render_as_images(dir, name, format): image files written to a directory;
171
196
  # render_as_images: Java BufferedImage objects.
data/lib/pd4ml/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Pd4ml
4
- VERSION = '1.1.1'
4
+ VERSION = '1.2.0'
5
5
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pd4ml
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.1.1
4
+ version: 1.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - PD4ML
@@ -53,6 +53,6 @@ required_rubygems_version: !ruby/object:Gem::Requirement
53
53
  requirements: []
54
54
  rubygems_version: 3.6.9
55
55
  specification_version: 4
56
- summary: 'PD4ML for Ruby: HTML and CSS to PDF, PDF/A, PDF/UA, ZUGFeRD, DOCX, RTF and
57
- images'
56
+ summary: 'PD4ML for Ruby: HTML and CSS to PDF, PDF/A, PDF/UA, ZUGFeRD, DOCX, RTF,
57
+ Markdown and images'
58
58
  test_files: []