docvortex 0.2.4__tar.gz → 0.3.0__tar.gz

{docvortex-0.2.4/src/docvortex.egg-info → docvortex-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: docvortex
-Version: 0.2.4
+Version: 0.3.0
 Summary: A fast, multi-format document parsing and conversion engine
 License-Expression: MIT
 Project-URL: Repository, https://github.com/myhloli/DocVortex
@@ -38,7 +38,6 @@ Requires-Dist: mathml2omml==0.0.2
 Requires-Dist: resvg_py<0.6,>=0.5.0
 Requires-Dist: reportlab
 Requires-Dist: ziamath<0.14,>=0.13
-Requires-Dist: fast-langdetect<0.3.0,>=0.2.3
 Requires-Dist: magika<1.1.0,>=0.6.2
 Requires-Dist: opencv-python>=4.11.0.86
 Provides-Extra: test
@@ -125,6 +124,33 @@ PDF page selections use `1-5`, `r1` and `all`; other native formats are parsed a
 whole documents. A caller-owned `PDFDocument` can be passed to `analyze` or `parse`
 and remains open afterward.
 
+## Page images and embedded assets
+
+```python
+from docvortex.assets import parse_image_data_uri_strict, transcode_image
+from docvortex.content.tree import iter_image_payloads
+from docvortex.document.pdf import PDFDocument
+
+with PDFDocument("report.pdf") as document:
+    image = document.render_image(0, bbox=(0.1, 0.2, 0.8, 0.7), image_format="png")
+# image.data, image.width, image.height, image.mime_type and image.extension
+# remain available after the document closes.
+```
+
+`render_image` accepts zero-based page indices, optional normalized bounding boxes
+and `jpeg` (default), `png` or `webp` output. Omitting `bbox` renders the whole page.
+Crops are encoded directly to the requested format. `crop_image` continues to
+return JPEG bytes. Image sources opened with `PDFDocument.from_image` retain the
+existing image-to-PDF conversion behavior.
+
+`docvortex.assets` exposes immutable `ImageArtifact` and `ImageFormat` contracts.
+`parse_image_data_uri_strict(uri)` validates embedded image bytes and returns
+`(data, extension)`; `transcode_image(data, image_format="png")` returns an
+`ImageArtifact`. These operations do not fetch URLs or resolve filesystem paths,
+and transcoding does not add SVG rasterization support.
+`iter_image_payloads(block)` yields the current node, if it carries an image,
+then traverses its children depth first without modifying the document tree.
+
 ## Explicit PDF classification
 
 ```python
@@ -168,3 +194,24 @@ DocVortex project code is licensed under the [MIT License](LICENSE.md).
 
 See [rendering ownership](docs/RENDER_OWNERSHIP.md) for the seven engine targets,
 MinerU Content List integration and public fragment helpers.
+
+## Read source metadata without parsing the body
+
+```python
+from docvortex import extract_metadata
+
+inspection = extract_metadata("report.pdf")
+print(inspection.metadata.document.title)
+print(inspection.metadata.document.authors)
+```
+
+All 15 native document formats support this API. Normal parsing also carries these properties in
+`metadata.document`, preserving the original PDF properties across page selections. See the
+[field definitions, format matrix, and compatibility notes](docs/METADATA.md).
+
+
+### Python 3.14 and text joining (0.3.0)
+
+DocVortex supports ordinary CPython 3.10–3.14. Physical text lines are joined by deterministic Unicode boundary rules, with URL and dehyphenation safeguards. Language detection models and the fast-langdetect / fasttext-predict dependencies are no longer required.
+
+The shared `resolve_text_line_boundary` and `merge_text_line_contents` helpers no longer accept `block_language`; `detect_lang` has been removed. Hosts should call the shared boundary helpers directly. OCR language selection and Magika code-language recognition are unchanged.

{docvortex-0.2.4 → docvortex-0.3.0}/README.md

@@ -71,6 +71,33 @@ PDF page selections use `1-5`, `r1` and `all`; other native formats are parsed a
 whole documents. A caller-owned `PDFDocument` can be passed to `analyze` or `parse`
 and remains open afterward.
 
+## Page images and embedded assets
+
+```python
+from docvortex.assets import parse_image_data_uri_strict, transcode_image
+from docvortex.content.tree import iter_image_payloads
+from docvortex.document.pdf import PDFDocument
+
+with PDFDocument("report.pdf") as document:
+    image = document.render_image(0, bbox=(0.1, 0.2, 0.8, 0.7), image_format="png")
+# image.data, image.width, image.height, image.mime_type and image.extension
+# remain available after the document closes.
+```
+
+`render_image` accepts zero-based page indices, optional normalized bounding boxes
+and `jpeg` (default), `png` or `webp` output. Omitting `bbox` renders the whole page.
+Crops are encoded directly to the requested format. `crop_image` continues to
+return JPEG bytes. Image sources opened with `PDFDocument.from_image` retain the
+existing image-to-PDF conversion behavior.
+
+`docvortex.assets` exposes immutable `ImageArtifact` and `ImageFormat` contracts.
+`parse_image_data_uri_strict(uri)` validates embedded image bytes and returns
+`(data, extension)`; `transcode_image(data, image_format="png")` returns an
+`ImageArtifact`. These operations do not fetch URLs or resolve filesystem paths,
+and transcoding does not add SVG rasterization support.
+`iter_image_payloads(block)` yields the current node, if it carries an image,
+then traverses its children depth first without modifying the document tree.
+
 ## Explicit PDF classification
 
 ```python
@@ -114,3 +141,24 @@ DocVortex project code is licensed under the [MIT License](LICENSE.md).
 
 See [rendering ownership](docs/RENDER_OWNERSHIP.md) for the seven engine targets,
 MinerU Content List integration and public fragment helpers.
+
+## Read source metadata without parsing the body
+
+```python
+from docvortex import extract_metadata
+
+inspection = extract_metadata("report.pdf")
+print(inspection.metadata.document.title)
+print(inspection.metadata.document.authors)
+```
+
+All 15 native document formats support this API. Normal parsing also carries these properties in
+`metadata.document`, preserving the original PDF properties across page selections. See the
+[field definitions, format matrix, and compatibility notes](docs/METADATA.md).
+
+
+### Python 3.14 and text joining (0.3.0)
+
+DocVortex supports ordinary CPython 3.10–3.14. Physical text lines are joined by deterministic Unicode boundary rules, with URL and dehyphenation safeguards. Language detection models and the fast-langdetect / fasttext-predict dependencies are no longer required.
+
+The shared `resolve_text_line_boundary` and `merge_text_line_contents` helpers no longer accept `block_language`; `detect_lang` has been removed. Hosts should call the shared boundary helpers directly. OCR language selection and Magika code-language recognition are unchanged.

{docvortex-0.2.4 → docvortex-0.3.0}/docs/JSON_PROTOCOL.md

@@ -80,3 +80,14 @@ current document. Generic ParseResult, HTTP/ZIP, Gradio and DocVortex codecs rem
 strict. Successfully converted batches can be cached and compacted when their
 metadata/extensions/full-document flags agree; ordinary reads never rewrite files.
 Unknown or damaged formats still require source reparse.
+
+## Optional source properties (DocVortex 0.2.5)
+
+`metadata.document` optionally carries a typed `DocumentProperties` object. Its descriptive,
+publication, date, authoring-software, and qualified count fields are shared by the lightweight
+`extract_metadata()` API and native analysis. It is distinct from the parser identity in
+`metadata.producer` and application data in `extensions`.
+
+The schema version remains 2.0. New readers accept absent source properties in existing documents;
+old strict readers require an upgrade to accept this addition. No legacy data is rewritten or
+silently enriched. See [source metadata](METADATA.md) for field and format semantics.

docvortex-0.3.0/docs/METADATA.md

@@ -0,0 +1,80 @@
+# Source document metadata
+
+`docvortex.extract_metadata()` reads the same source properties that native analysis places in
+`ModelJson.metadata.document` and `MiddleJson.metadata.document`. It does not run body prediction,
+OCR, VLM, PDF classification, scripts, or resource downloads.
+
+```python
+from docvortex import extract_metadata, parse
+
+inspection = extract_metadata("report.pdf")
+print(inspection.metadata.document)
+print(inspection.diagnostics)
+result = parse("report.pdf", page_range="2-3", keep_model_json=True)
+assert result.middle_json.metadata.document == inspection.metadata.document
+```
+
+The source can be a path, `bytes`, or a caller-owned `PDFDocument`. Bytes without a recognizable
+signature can use `file_suffix=`. HTML can additionally receive `source_context=` for its declared
+transport encoding. A caller-owned PDF remains open; internally opened documents are closed.
+PDF source properties are captured before page selection, so the count refers to the whole input.
+
+## Properties
+
+`DocumentProperties` is defined in `docvortex.schema`. Optional strings are `title`, `subject`,
+`description`, `publisher`, `created_at`, `modified_at`, `creator_application`, and
+`producer_application`. `authors`, `keywords`, `languages`, and `identifiers` are ordered lists:
+empty values are removed and duplicates are removed without splitting individual source strings.
+Dates are ISO 8601 strings retaining source precision and explicit timezone information. OLE
+FILETIME values are UTC. Missing or invalid dates are not replaced with filesystem or current time.
+
+`page_count` is a nonnegative source count or null; `page_count_kind` describes its meaning:
+
+| Native format | Property sources | Count kind |
+| --- | --- | --- |
+| PDF | Info first, XMP fills missing properties, catalog language | `physical` |
+| DOCX | OOXML core/extended properties | `declared`, if stored |
+| PPTX | OOXML properties and slide directory, including hidden slides | `slide` |
+| XLSX | OOXML properties and sheet directory, including hidden sheets | `sheet` |
+| DOC / PPT / XLS | OLE SummaryInformation / DocumentSummaryInformation | `declared`, if stored |
+| ODT | `meta.xml` | `declared`, if stored |
+| ODS / ODP | `meta.xml` and visible sheet/slide structure | `sheet` / `slide` |
+| RTF | `info`, generator, declared language | `declared`, if stored |
+| EPUB | OPF Dublin Core and recognized meta properties | `spine` |
+| OFD | DocInfo; first nonempty scalar across DocBody entries | `physical`, summed across documents |
+| HTML | title, standard meta, Dublin Core, Open Graph, document language | `logical`, 1 |
+| CSV | no embedded descriptive property convention | `logical`, 1 |
+
+A declared count may be stale and is not a measured layout. Counts are never inferred from the
+number of pages selected for parsing. Source author strings are not authors inferred from body text.
+Publication dates are not relabeled as creation dates. Image EXIF and arbitrary custom properties
+are outside this interface.
+
+## Errors and protocol
+
+`MetadataResult` contains `metadata: DocumentMetadata` and `diagnostics: tuple[Diagnostic, ...]`.
+An unreadable source raises `DocumentError` with `open_failed`; unsupported inputs use
+`file_type_unsupported`. Optional property-part failures produce `read_metadata_failed` diagnostics
+and retain other successfully extracted fields. Ordinary native analysis can continue if optional
+metadata extraction fails and its body reader can still parse the document.
+
+`metadata.producer` identifies the parser producing the JSON. Original authoring/conversion software
+belongs inside `metadata.document`, and never replaces that parser identity. Postprocessing deep
+copies the properties. JSON, Structured Content, and bundles retain them without needing the source.
+The new fields do not change rendering layout or the existing exported file-property defaults.
+
+This is an optional addition to shared JSON **2.0**, introduced in DocVortex **0.2.5**. Existing 2.0
+JSON without `metadata.document` remains readable and no source facts are invented. Older strict
+readers must be upgraded before consuming new fields. Schema and bundle version numbers remain 2.0.
+
+## MinerU integration
+
+Doclib calls the public lightweight API during ingestion. It maps existing database columns and
+applies its own truncation limits. It does not store all expanded properties in new database columns;
+complete properties are retained in subsequent parse JSON. Authors join with `; `, keywords with
+`, `, and the language column takes the first declared language. Reflow scheduling remains a product
+policy and does not change the source count recorded in JSON.
+
+After parsing, nonempty source properties update existing columns and title/author search fields
+without downgrading the indexed body tier. Old caches are not rewritten or forcibly reparsed;
+compaction still declines batches with different metadata envelopes.

{docvortex-0.2.4 → docvortex-0.3.0}/pyproject.toml

@@ -27,7 +27,7 @@ dependencies = [
     "mammoth>=1.11.0,<2", "openpyxl>=3.1.5,<4", "olefile>=0.47,<1",
     "pylatexenc>=2.10,<3", "latex2mathml>=3.81.0,<4", "mathml2omml==0.0.2",
     "resvg_py>=0.5.0,<0.6",
-    "reportlab", "ziamath>=0.13,<0.14", "fast-langdetect>=0.2.3,<0.3.0",
+    "reportlab", "ziamath>=0.13,<0.14",
     "magika>=0.6.2,<1.1.0", "opencv-python>=4.11.0.86",
 ]
 

{docvortex-0.2.4 → docvortex-0.3.0}/schemas/middle-2.0.json

@@ -774,6 +774,17 @@
         },
         "producer": {
           "$ref": "#/$defs/Producer"
+        },
+        "document": {
+          "anyOf": [
+            {
+              "$ref": "#/$defs/DocumentProperties"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null
         }
       },
       "required": [
@@ -783,6 +794,171 @@
       "title": "DocumentMetadata",
       "type": "object"
     },
+    "DocumentProperties": {
+      "additionalProperties": false,
+      "description": "保存源文件声明的属性；计数属于完整源文件而非当前解析选页。",
+      "properties": {
+        "title": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null,
+          "title": "Title"
+        },
+        "authors": {
+          "items": {
+            "type": "string"
+          },
+          "title": "Authors",
+          "type": "array"
+        },
+        "subject": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null,
+          "title": "Subject"
+        },
+        "keywords": {
+          "items": {
+            "type": "string"
+          },
+          "title": "Keywords",
+          "type": "array"
+        },
+        "description": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null,
+          "title": "Description"
+        },
+        "languages": {
+          "items": {
+            "type": "string"
+          },
+          "title": "Languages",
+          "type": "array"
+        },
+        "identifiers": {
+          "items": {
+            "type": "string"
+          },
+          "title": "Identifiers",
+          "type": "array"
+        },
+        "publisher": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null,
+          "title": "Publisher"
+        },
+        "created_at": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null,
+          "title": "Created At"
+        },
+        "modified_at": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null,
+          "title": "Modified At"
+        },
+        "creator_application": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null,
+          "title": "Creator Application"
+        },
+        "producer_application": {
+          "anyOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null,
+          "title": "Producer Application"
+        },
+        "page_count": {
+          "anyOf": [
+            {
+              "minimum": 0,
+              "type": "integer"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null,
+          "title": "Page Count"
+        },
+        "page_count_kind": {
+          "anyOf": [
+            {
+              "enum": [
+                "physical",
+                "declared",
+                "slide",
+                "sheet",
+                "spine",
+                "logical"
+              ],
+              "type": "string"
+            },
+            {
+              "type": "null"
+            }
+          ],
+          "default": null,
+          "title": "Page Count Kind"
+        }
+      },
+      "title": "DocumentProperties",
+      "type": "object"
+    },
     "EquationBlock": {
       "additionalProperties": false,
       "properties": {