exstruct 0.3.1__tar.gz → 0.3.5__tar.gz

{exstruct-0.3.1 → exstruct-0.3.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: exstruct
-Version: 0.3.1
+Version: 0.3.5
 Summary: Excel to structured JSON (tables, shapes, charts) for LLM/RAG pipelines
 Keywords: excel,structure,data,exstruct
 Author: harumiWeb
@@ -57,16 +57,16 @@ Description-Content-Type: text/markdown
 
 [![PyPI version](https://badge.fury.io/py/exstruct.svg)](https://pypi.org/project/exstruct/) [![PyPI Downloads](https://static.pepy.tech/personalized-badge/exstruct?period=total&units=INTERNATIONAL_SYSTEM&left_color=BLACK&right_color=GREEN&left_text=downloads)](https://pepy.tech/projects/exstruct) ![Licence: BSD-3-Clause](https://img.shields.io/badge/license-BSD--3--Clause-blue?style=flat-square) [![pytest](https://github.com/harumiWeb/exstruct/actions/workflows/pytest.yml/badge.svg)](https://github.com/harumiWeb/exstruct/actions/workflows/pytest.yml) [![Codacy Badge](https://app.codacy.com/project/badge/Grade/e081cb4f634e4175b259eb7c34f54f60)](https://app.codacy.com/gh/harumiWeb/exstruct/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade) [![codecov](https://codecov.io/gh/harumiWeb/exstruct/graph/badge.svg?token=2XI1O8TTA9)](https://codecov.io/gh/harumiWeb/exstruct)
 
-![ExStruct Image](/docs/assets/icon.webp)
+![ExStruct Image](docs/assets/icon.webp)
 
-ExStruct reads Excel workbooks and outputs structured data (cells, table candidates, shapes, charts, smartart, print areas/views, auto page-break areas, hyperlinks) as JSON by default, with optional YAML/TOON formats. It targets both COM/Excel environments (rich extraction) and non-COM environments (cells + table candidates + print areas), with tunable detection heuristics and multiple output modes to fit LLM/RAG pipelines.
+ExStruct reads Excel workbooks and outputs structured data (cells, table candidates, shapes, charts, smartart, merged cell ranges, print areas/views, auto page-break areas, hyperlinks) as JSON by default, with optional YAML/TOON formats. It targets both COM/Excel environments (rich extraction) and non-COM environments (cells + table candidates + print areas), with tunable detection heuristics and multiple output modes to fit LLM/RAG pipelines.
 
 [日本版 README](README.ja.md)
 
 ## Features
 
 - **Excel → Structured JSON**: cells, shapes, charts, smartart, table candidates, print areas/views, and auto page-break areas per sheet.
-- **Output modes**: `light` (cells + table candidates + print areas; no COM, shapes/charts empty), `standard` (texted shapes + arrows, charts, smartart, print areas), `verbose` (all shapes with width/height, charts with size, print areas). Verbose also emits cell hyperlinks and `colors_map`. Size output is flag-controlled.
+- **Output modes**: `light` (cells + table candidates + print areas; no COM, shapes/charts empty), `standard` (texted shapes + arrows, charts, smartart, merged cell ranges, print areas), `verbose` (all shapes with width/height, charts with size, merged cell ranges, print areas). Verbose also emits cell hyperlinks and `colors_map`. Size output is flag-controlled.
 - **Auto page-break export (COM only)**: capture Excel-computed auto page breaks and write per-area JSON/YAML/TOON when requested (CLI option appears only when COM is available).
 - **Formats**: JSON (compact by default, `--pretty` available), YAML, TOON (optional dependencies).
 - **Table detection tuning**: adjust heuristics at runtime via API.
@@ -98,8 +98,8 @@ exstruct input.xlsx -o out.json --pretty   # pretty JSON to a file
 exstruct input.xlsx --format yaml          # YAML (needs pyyaml)
 exstruct input.xlsx --format toon          # TOON (needs python-toon)
 exstruct input.xlsx --sheets-dir sheets/   # split per sheet in chosen format
-exstruct input.xlsx --print-areas-dir areas/  # split per print area (if any)
 exstruct input.xlsx --auto-page-breaks-dir auto_areas/  # COM only; option appears when available
+exstruct input.xlsx --print-areas-dir areas/  # split per print area (if any)
 exstruct input.xlsx --mode light           # cells + table candidates only
 exstruct input.xlsx --pdf --image          # PDF and PNGs (Excel required)
 ```
@@ -147,9 +147,9 @@ engine = ExStructEngine(
     ),
 )
 wb2 = engine.extract("input.xlsx")
-engine.export(wb2, Path("out_filtered.json"))  # drops shapes via filters
+engine.export(wb2, Path("out_filtered.json"))
 
-# Enable hyperlinks in other modes
+# Enable hyperlinks in standard mode
 engine_links = ExStructEngine(options=StructOptions(mode="standard", include_cell_links=True))
 with_links = engine_links.extract("input.xlsx")
 
@@ -188,8 +188,8 @@ Use higher thresholds to reduce false positives; lower them if true tables are m
 ## Output Modes
 
 - **light**: cells + table candidates (no COM needed).
-- **standard**: texted shapes + arrows, charts (COM if available), table candidates. Hyperlinks are off unless `include_cell_links=True`.
-- **verbose**: all shapes (with width/height), charts, table candidates, cell hyperlinks, and `colors_map`.
+- **standard**: texted shapes + arrows, charts (COM if available), merged cell ranges, table candidates. Hyperlinks are off unless `include_cell_links=True`.
+- **verbose**: all shapes (with width/height), charts, merged cell ranges, table candidates, cell hyperlinks, and `colors_map`.
 
 ## Error Handling / Fallbacks
 
@@ -207,7 +207,7 @@ exstruct input.xlsx --pdf --image --dpi 144
 
 Creates `<output>.pdf` and `<output>_images/` PNGs per sheet.
 
-## Benchmark: Excel Structuring Demo
+## Example 1: Excel Structuring Demo
 
 To show how well exstruct can structure Excel, we parse a workbook that combines three elements on one sheet and share an AI reasoning benchmark that uses the JSON output.
 
@@ -216,7 +216,7 @@ To show how well exstruct can structure Excel, we parse a workbook that combines
 - Flowchart built only with shapes
 
 (Screenshot below is the actual sample Excel sheet)
-![Sample Excel](/docs/assets/demo_sheet.png)
+![Sample Excel](docs/assets/demo_sheet.png)
 Sample workbook: `sample/sample.xlsx`
 
 ### 1. Input: Excel Sheet Overview
@@ -277,6 +277,7 @@ Below is a **shortened JSON output example** from parsing this Excel workbook.
           "text": "開始",
           "l": 148,
           "t": 220,
+          "kind": "shape",
           "type": "AutoShape-FlowchartProcess"
         },
         {
@@ -284,12 +285,13 @@ Below is a **shortened JSON output example** from parsing this Excel workbook.
           "text": "入力データ読み込み",
           "l": 132,
           "t": 282,
+          "kind": "shape",
           "type": "AutoShape-FlowchartProcess"
         },
         {
           "l": 193,
           "t": 246,
-          "type": "AutoShape-Mixed",
+          "kind": "arrow",
           "begin_arrow_style": 1,
           "end_arrow_style": 2,
           "begin_id": 1,
@@ -389,6 +391,255 @@ flowchart TD
 ```
 ````
 
+
+## Example 2: General Application Form
+
+### Excel Sheet
+
+![General Application Form Excel](docs/assets/demo_form_en.png)
+
+### ExStruct JSON
+
+(Truncated for brevity)
+
+```json
+{
+  "book_name": "en_sf425.xlsx",
+  "sheets": {
+    "FFR": {
+      "rows": [
+        { "r": 1, "c": { "0": "FEDERAL FINANCIAL REPORT" } },
+        { "r": 3, "c": { "0": "(Follow form instructions)" } },
+        {
+          "r": 4,
+          "c": {
+            "0": "1.  Federal Agency and Organizational Element",
+            "5": "2.  Federal Grant or Other Identifying Number Assigned by Federal Agency",
+            "20": "Page",
+            "23": " of"
+          }
+        },
+        ...
+      ],
+      "shapes": [
+        {
+          "id": 1,
+          "text": "Not Required by EPA",
+          "l": 575,
+          "t": 396,
+          "kind": "shape",
+          "type": "AutoShape-Rectangle"
+        }
+      ],
+      "print_areas": [{ "r1": 1, "c1": 0, "r2": 66, "c2": 23 }],
+      "merged_cells": {
+        "schema": ["r1", "c1", "r2", "c2", "v"],
+        "items": [
+          [34, 15, 34, 23, " "],
+          [56, 10, 57, 17, "Federal Share Calculation"],
+          [18, 10, 18, 23, " "],
+          [15, 0, 15, 1, " "],
+          ...
+        ]
+      }
+    }
+  }
+}
+
+```
+
+### Migration note (v0.3.5): merged_cells format change
+
+`merged_cells` changed from a list of objects to a schema/items structure in v0.3.5 (breaking change for JSON consumers).
+
+Old format (<= v0.3.2):
+
+```json
+"merged_cells": [
+  { "r1": 34, "c1": 15, "r2": 34, "c2": 23, "v": " " },
+  { "r1": 56, "c1": 10, "r2": 57, "c2": 17, "v": "Federal Share Calculation" }
+]
+```
+
+New format (v0.3.5+):
+
+```json
+"merged_cells": {
+  "schema": ["r1", "c1", "r2", "c2", "v"],
+  "items": [
+    [34, 15, 34, 23, " "],
+    [56, 10, 57, 17, "Federal Share Calculation"]
+  ]
+}
+```
+
+Migration example (support both during transition):
+
+```python
+def normalize_merged_cells(raw):
+    schema = ["r1", "c1", "r2", "c2", "v"]
+    if isinstance(raw, list):
+        items = [[d.get(k, " ") for k in schema] for d in raw]
+        return {"schema": schema, "items": items}
+    if isinstance(raw, dict) and "schema" in raw and "items" in raw:
+        return raw
+    return None
+```
+
+### LLM reconstruction example
+
+```md
+<!-- This is a structured interpretation, not an official reproduction. -->
+
+# **Federal Financial Report (FFR)**
+
+_(Follow form instructions)_
+
+---
+
+## **1. Federal Agency and Organizational Element**
+
+**United States Environmental Protection Agency**
+
+## **2. Federal Grant or Other Identifying Number Assigned by Federal Agency**
+
+**Page 1 of pages**
+
+---
+
+## **3. Recipient Organization**
+
+_(Name and complete address including Zip code)_
+
+---
+
+## **4. Recipient Identifiers**
+
+- **4a. DUNS Number**
+- **4b. EIN**
+- **5. Recipient Account Number or Identifying Number**  
+  _(To report multiple grants, use FFR Attachment)_
+- **6. Report Type**
+  - □ Quarterly
+  - □ Semi-Annual
+  - □ Annual
+  - □ Final
+- **7. Basis of Accounting**
+  - □ Cash
+  - □ Accrual
+
+---
+
+## **8. Project/Grant Period**
+
+- **From:** (Month, Day, Year)
+- **To:** (Month, Day, Year)
+
+## **9. Reporting Period End Date**
+
+(Month, Day, Year)
+
+---
+
+# **10. Transactions**
+
+_(Use lines a–c for single or multiple grant reporting)_
+
+### **Federal Cash**
+
+_(To report multiple grants, also use FFR Attachment)_
+
+- **a. Cash Receipts**
+- **b. Cash Disbursements**
+- **c. Cash on Hand** (line a minus b)
+
+_(Use lines d–o for single grant reporting)_
+
+### **Federal Expenditures and Unobligated Balance**
+
+- **d. Total Federal funds authorized**
+- **e. Federal share of expenditures**
+- **f. Federal share of unliquidated obligations** — 0
+- **g. Total Federal share** (sum of lines e and f) — 0
+- **h. Unobligated balance of Federal funds** (line d minus g) — 0
+
+### **Recipient Share**
+
+- **i. Total recipient share required**
+- **j. Recipient share of expenditures**
+- **k. Remaining recipient share to be provided** (line i minus j) — 0
+
+### **Program Income**
+
+- **l. Total Federal program income earned**
+- **m. Program income expended (deduction alternative)**
+- **n. Program income expended (addition alternative)**
+- **o. Unexpended program income** (line l minus line m or line n)
+
+---
+
+# **11. Indirect Expense**
+
+| Type           | Rate | Period From | Period To | Base | Amount Charged | Federal Share |
+| -------------- | ---- | ----------- | --------- | ---- | -------------- | ------------- |
+|                |      |             |           |      |                |               |
+| **g. Totals:** |      |             |           | 0    | 0              | 0             |
+
+---
+
+# **12. Remarks**
+
+Attach any explanations deemed necessary or information required by the Federal sponsoring agency in compliance with governing legislation.
+
+---
+
+# **13. Certification**
+
+By signing this report, I certify that it is true, complete, and accurate to the best of my knowledge.  
+I am aware that any false, fictitious, or fraudulent information may subject me to criminal, civil, or administrative penalties.  
+_(U.S. Code, Title 18, Section 1001)_
+
+- **a. Typed or Printed Name and Title of Authorized Certifying Official**
+- **b. Signature of Authorized Certifying Official**
+- **c. Telephone** (Area code, number, extension)
+- **d. Email address**
+- **e. Date Report Submitted** (Month, Day, Year)
+
+---
+
+# **14. Agency Use Only**
+
+**Federal Share Calculation**
+
+---
+
+### **Standard Form 425**
+
+**OMB Approval Number:** 0348-0061  
+**Expiration Date:** 02/28/2015
+
+---
+
+# **Paperwork Burden Statement**
+
+According to the Paperwork Reduction Act, as amended, no persons are required to respond to a collection of information unless it displays a valid OMB Control Number.  
+The valid OMB control number for this information collection is **0348-0061**.
+
+Public reporting burden for this collection of information is estimated to average **1.5 hours per response**, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.
+
+Send comments regarding the burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to:
+
+**Office of Management and Budget**  
+Paperwork Reduction Project (0348-0060)  
+Washington, DC 20503
+
+---
+
+### **Shape in Original Document**
+
+- _Not Required by EPA_
+```
+
 From this we can see:
 
 **exstruct's JSON is already in a format that AI can read and reason over directly.**
@@ -398,6 +649,7 @@ Other LLM inference samples using this library can be found in the following dir
 - [Basic Excel](sample/basic/)
 - [Flowchart](sample/flowchart/)
 - [Gantt Chart](sample/gantt_chart/)
+- [Application forms with many merged cells](sample/forms_with_many_merged_cells/)
 
 ### 4. Summary
 
@@ -435,6 +687,11 @@ This project is suitable for teams that:
 - Use CLI `--auto-page-breaks-dir` (COM only), `DestinationOptions.auto_page_breaks_dir` (preferred), or `export_auto_page_breaks(...)` to write per-auto-page-break files; the API raises `ValueError` if no auto page breaks exist.
 - `PrintAreaView` includes rows and table candidates inside the area, plus shapes/charts that overlap the area (size-less shapes are treated as points). `normalize=True` rebases row/col indices to the area origin.
 
+## Documentation build
+
+- Update generated model docs before building the site: `python scripts/gen_model_docs.py`.
+- Build locally with mkdocs + mkdocstrings (dev deps required): `uv run mkdocs serve` or `uv run mkdocs build`.
+
 ## Architecture
 
 ExStruct uses a pipeline-based architecture that separates