json2ubl 1.0.0__tar.gz → 1.0.1__tar.gz

json2ubl-1.0.1/CHANGELOG.md

@@ -0,0 +1,108 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+---
+
+## [1.0.1] - 2026-02-03
+
+### 🐛 Fixed
+
+- **Multi-page document merging**: Fixed critical issue where only last occurrence of array fields (e.g., InvoiceLine, AllowanceCharge) was retained when same invoice ID appeared multiple times
+- **Hardcoded field names**: Replaced hardcoded array field list with schema-driven detection via `maxOccurs="unbounded"`
+- **Case-insensitive field matching**: Fixed merging when input JSON has mixed case field names (e.g., `InvoiceLine` vs `invoiceline`)
+
+### ✨ Added
+
+- **Schema-driven array detection**: `_merge_pages()` now dynamically identifies array fields from schema cache instead of hardcoded list
+- **Unified merge behavior across all APIs**: All 3 APIs (dict, file-to-dict, file-to-files) now use `_group_and_merge_documents()` helper for consistent merging
+- **Support for all document types**: Merge logic now works dynamically with all 60+ UBL document types, not just Invoice-specific fields
+- **Nested array merging**: Properly merges nested array fields (e.g., TaxTotal > TaxSubtotal, AllowanceCharge arrays)
+- **Comprehensive test coverage**: 
+  - Schema-driven array detection tests
+  - Mixed case field name merging tests
+  - Multi-invoice batch merging tests
+  - Nested array field merging tests
+  - Cross-API consistency tests
+  - Different document type tests (Invoice, CreditNote)
+
+### 🔄 Changed
+
+- `Json2UblConverter._merge_pages()`: Now accepts `schema_cache` parameter for dynamic array field detection
+- `Json2UblConverter.convert_json_file_to_xml_dict()`: Refactored to use new `_group_and_merge_documents()` helper
+- `json_dict_to_ubl_xml()` in `__init__.py`: Now calls `_group_and_merge_documents()` to ensure consistent merge behavior
+- Error handling: Improved error messages for merge failures with proper logging
+
+### 🧪 Testing
+
+**New test cases added:**
+- `test_merge_pages_multiple_pages` - Schema-driven array detection with Invoice
+- `test_merge_pages_mixed_case_field_names` - Case-insensitive field matching
+- `test_group_and_merge_documents_standalone` - Direct testing of merge helper
+- `test_group_and_merge_documents_skips_no_id` - Filtering documents without ID
+- `test_merge_pages_schema_driven_different_doc_types` - Different document type schema detection (CreditNote)
+- `test_merge_same_invoice_across_all_apis` - Integration test across all 3 APIs
+- `test_merge_multiple_invoices_same_batch` - Multi-invoice batch merging (4 pages, 2 invoices)
+- `test_merge_with_missing_document_type` - Error handling during merge
+- `test_merge_nested_arrays_in_complex_structure` - TaxTotal nested array merging
+- `test_merge_allowancecharge_arrays` - AllowanceCharge array merging
+
+### 🚀 Data Integrity
+
+**Important**: This release fixes a critical data loss issue:
+- **Before v1.0.1**: When invoice ID "728621" appeared twice in input JSON, only line items from last occurrence were in output XML (other pages' items lost)
+- **After v1.0.1**: All line items from all pages with same invoice ID are merged into single valid XML file
+
+### 📝 Documentation
+
+- Updated README with multi-page document merging feature
+- Added inline code documentation for schema-driven merging
+
+### 🔧 Technical Details
+
+**Key files modified:**
+- `src/json2ubl/converter.py`:
+  - Added `_group_and_merge_documents()` helper method (lines 609-654)
+  - Enhanced `_merge_pages()` with schema_cache parameter (lines 656-680)
+  - Refactored `convert_json_file_to_xml_dict()` (lines 354-407)
+  
+- `src/json2ubl/__init__.py`:
+  - Updated `json_dict_to_ubl_xml()` to use merge helper (lines 79-135)
+
+- `tests/test_components.py`:
+  - Enhanced merge test cases with schema_cache
+
+- `tests/test_error_handling.py`:
+  - Updated error handling validation
+
+- `tests/test_pipeline.py`:
+  - Added integration tests for multi-API consistency
+  - Added nested array merging tests
+
+---
+
+## [1.0.0] - 2026-01-30
+
+### ✨ Initial Release
+
+- Full UBL 2.1 support (60+ document types)
+- Schema-driven JSON-to-XML mapping
+- Three conversion APIs (dict, file-to-dict, file-to-files)
+- XML validation against UBL schemas
+- Comprehensive error handling with rollback
+- Thread-safe batch processing
+- Pre-built schema caches for performance
+- Extensive test coverage
+- Production-ready logging with loguru
+
+---
+
+## Versioning
+
+This project follows [Semantic Versioning](https://semver.org/):
+- **MAJOR**: Incompatible API changes
+- **MINOR**: New features (backward compatible)
+- **PATCH**: Bug fixes (backward compatible)

{json2ubl-1.0.0 → json2ubl-1.0.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: json2ubl
-Version: 1.0.0
+Version: 1.0.1
 Summary: Production-grade JSON to UBL 2.1 XML converter with schema-driven mapping
 Author-email: SherozShaikh <shaikh.sheroz07@gmail.com>
 License: MIT
@@ -22,16 +22,16 @@ Classifier: Topic :: Office/Business :: Financial
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: lxml<5.0,>=4.9.4
-Requires-Dist: pydantic<3.0,>=2.7.0
-Requires-Dist: pyyaml<7.0,>=6.0.1
-Requires-Dist: loguru<1.0,>=0.7.2
+Requires-Dist: lxml~=6.0
+Requires-Dist: pydantic~=2.7
+Requires-Dist: pyyaml~=6.0
+Requires-Dist: loguru~=0.7
 Provides-Extra: dev
-Requires-Dist: pytest<9.0,>=8.3.0; extra == "dev"
-Requires-Dist: pytest-cov<6.0,>=5.0.0; extra == "dev"
-Requires-Dist: black<25.0,>=24.10.0; extra == "dev"
-Requires-Dist: ruff<1.0,>=0.8.0; extra == "dev"
-Requires-Dist: mypy<2.0,>=1.13.0; extra == "dev"
+Requires-Dist: pytest~=8.3; extra == "dev"
+Requires-Dist: pytest-cov~=5.0; extra == "dev"
+Requires-Dist: black~=24.10; extra == "dev"
+Requires-Dist: ruff~=0.8; extra == "dev"
+Requires-Dist: mypy~=1.13; extra == "dev"
 Dynamic: license-file
 
 # json2ubl
@@ -41,6 +41,7 @@ Dynamic: license-file
 [![PyPI version](https://badge.fury.io/py/json2ubl.svg)](https://badge.fury.io/py/json2ubl)
 [![Python Versions](https://img.shields.io/pypi/pyversions/json2ubl.svg)](https://pypi.org/project/json2ubl/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 
 [json2ubl](https://pypi.org/project/json2ubl/) is a production-ready converter that transforms JSON documents into UBL 2.1-compliant XML. It works with all 60+ UBL document types using automatic schema-driven mapping—no hardcoded field definitions required.
 
@@ -211,6 +212,8 @@ Convert JSON file and write XML files to disk.
 - Rolls back on partial failure
 - Atomic file operations with temp file staging
 
+**For detailed API documentation with input/output examples and error handling, see [API.md](docs/API.md)**
+
 ---
 
 ## 🛡️ Error Handling

{json2ubl-1.0.0 → json2ubl-1.0.1}/README.md

@@ -5,6 +5,7 @@
 [![PyPI version](https://badge.fury.io/py/json2ubl.svg)](https://badge.fury.io/py/json2ubl)
 [![Python Versions](https://img.shields.io/pypi/pyversions/json2ubl.svg)](https://pypi.org/project/json2ubl/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 
 [json2ubl](https://pypi.org/project/json2ubl/) is a production-ready converter that transforms JSON documents into UBL 2.1-compliant XML. It works with all 60+ UBL document types using automatic schema-driven mapping—no hardcoded field definitions required.
 
@@ -175,6 +176,8 @@ Convert JSON file and write XML files to disk.
 - Rolls back on partial failure
 - Atomic file operations with temp file staging
 
+**For detailed API documentation with input/output examples and error handling, see [API.md](docs/API.md)**
+
 ---
 
 ## 🛡️ Error Handling

json2ubl-1.0.1/docs/API.md

@@ -0,0 +1,451 @@
+# JSON to UBL 2.1 XML Converter - API Documentation
+
+**Package:** `json2ubl>=1.0.0`  
+**Installation:** `pip install json2ubl>=1.0.0`
+
+---
+
+## Overview
+
+The json2ubl package provides three APIs to convert JSON documents to UBL 2.1-compliant XML format. All APIs support all 60+ UBL document types using schema-driven automatic mapping.
+
+---
+
+## API 1: `json_dict_to_ubl_xml()`
+
+Convert a list of JSON dictionaries to UBL XML strings in memory.
+
+### Function Signature
+
+```python
+from json2ubl import json_dict_to_ubl_xml
+
+response = json_dict_to_ubl_xml(
+    list_of_dicts: List[Dict[str, Any]],
+    config_path: str | None = None
+) -> Dict[str, Any]
+```
+
+### Parameters
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `list_of_dicts` | `List[Dict]` | Yes | List of document dictionaries to convert |
+| `config_path` | `str` | No | Path to optional `ubl_converter.yaml` config file |
+
+### Input Format
+
+Each dictionary must contain:
+- `id` - Document identifier (required)
+- `document_type` - Numeric code (required). Examples: 380=Invoice, 381=CreditNote, 382=DebitNote
+- Schema fields matching UBL 2.1 structure (case-insensitive)
+
+**Example Input:**
+```python
+invoices = [
+    {
+        "id": "INV-2026-001",
+        "document_type": 380,  # Invoice
+        "issue_date": "2026-01-30",
+        "accounting_supplier_party": {
+            "party_name": "Supplier Inc",
+            "party_identification": {"id": "12345"}
+        },
+        "accounting_customer_party": {
+            "party_name": "Customer Ltd"
+        },
+        "invoice_lines": [
+            {
+                "id": "1",
+                "invoiced_quantity": 10,
+                "invoiced_quantity_unit_code": "EA",
+                "line_extension_amount": 1000.00
+            }
+        ]
+    }
+]
+
+response = json_dict_to_ubl_xml(invoices)
+```
+
+### Response Format
+
+```python
+{
+    "documents": [
+        {
+            "id": "INV-2026-001",
+            "xml": "<ubl:Invoice xmlns:ubl='urn:...'> ... </ubl:Invoice>",
+            "unmapped_fields": ["custom_field_1", "custom_field_2"]
+        }
+    ],
+    "summary": {
+        "total_inputs": 1,
+        "files_created": 0,
+        "document_types": {"Invoice": 1}
+    },
+    "error_response": None
+}
+```
+
+### Response Fields
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `documents` | `List[Dict]` | List of converted documents |
+| `documents[].id` | `str` | Document identifier from input |
+| `documents[].xml` | `str` | Generated UBL 2.1 XML string |
+| `documents[].unmapped_fields` | `List[str]` | JSON fields not found in UBL schema |
+| `summary` | `Dict` | Conversion statistics |
+| `summary.total_inputs` | `int` | Number of documents processed |
+| `summary.files_created` | `int` | Always 0 for this API |
+| `summary.document_types` | `Dict` | Count by document type (e.g., `{"Invoice": 1}`) |
+| `error_response` | `Dict\|None` | Error details if conversion failed |
+
+### Error Response
+
+On failure, `error_response` contains:
+
+```python
+{
+    "error_code": "MISSING_DOCUMENT_TYPE",  # or INVALID_DOCUMENT_TYPE, etc
+    "message": "Missing required field: document_type",
+    "details": {
+        "doc_id": "INV-2026-001",
+        "required_field": "document_type"
+    }
+}
+```
+
+### Usage Example
+
+```python
+from json2ubl import json_dict_to_ubl_xml
+
+invoices = [
+    {
+        "id": "INV-001",
+        "document_type": 380,
+        "issue_date": "2026-01-31",
+        "accounting_supplier_party": {"party_name": "ABC Corp"},
+        "accounting_customer_party": {"party_name": "XYZ Ltd"},
+        "invoice_lines": [{"id": "1", "line_extension_amount": 500.00}]
+    }
+]
+
+response = json_dict_to_ubl_xml(invoices)
+
+if response.get("error_response"):
+    print(f"Error: {response['error_response']['message']}")
+else:
+    for doc in response["documents"]:
+        print(f"✓ {doc['id']} converted")
+        print(f"  Unmapped: {doc['unmapped_fields']}")
+        # Save XML or process further
+        with open(f"{doc['id']}.xml", "w") as f:
+            f.write(doc["xml"])
+```
+
+### Common Errors
+
+| Error Code | Cause | Solution |
+|------------|-------|----------|
+| `MISSING_DOCUMENT_TYPE` | Missing `document_type` field | Add numeric document type code |
+| `INVALID_DOCUMENT_TYPE` | Wrong `document_type` value | Use valid code (380=Invoice, 381=CreditNote, etc) |
+| `MappingError` | Invalid JSON structure | Verify field names match UBL schema |
+
+---
+
+## API 2: `json_file_to_ubl_xml_dict()`
+
+Convert JSON file to UBL XML strings and return in memory (batch processing).
+
+### Function Signature
+
+```python
+from json2ubl import json_file_to_ubl_xml_dict
+
+response = json_file_to_ubl_xml_dict(
+    json_file_path: str,
+    config_path: str | None = None
+) -> Dict[str, Any]
+```
+
+### Parameters
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `json_file_path` | `str` | Yes | Path to JSON file containing list of documents |
+| `config_path` | `str` | No | Path to optional config file |
+
+### Input Format
+
+JSON file must contain a list array:
+
+**example.json:**
+```json
+[
+  {
+    "id": "INV-001",
+    "document_type": 380,
+    "issue_date": "2026-01-31",
+    "accounting_supplier_party": {"party_name": "Supplier"},
+    "accounting_customer_party": {"party_name": "Customer"},
+    "invoice_lines": [{"id": "1", "line_extension_amount": 1000}]
+  },
+  {
+    "id": "INV-002",
+    "document_type": 380,
+    "issue_date": "2026-02-01",
+    ...
+  }
+]
+```
+
+### Response Format
+
+Same as API 1:
+
+```python
+{
+    "documents": [
+        {"id": "INV-001", "xml": "...", "unmapped_fields": []},
+        {"id": "INV-002", "xml": "...", "unmapped_fields": []}
+    ],
+    "summary": {
+        "total_inputs": 2,
+        "files_created": 0,
+        "document_types": {"Invoice": 2}
+    },
+    "error_response": None
+}
+```
+
+### Usage Example
+
+```python
+from json2ubl import json_file_to_ubl_xml_dict
+
+response = json_file_to_ubl_xml_dict("invoices.json")
+
+if response.get("error_response"):
+    print(f"Error: {response['error_response']['message']}")
+    print(f"Details: {response['error_response']['details']}")
+else:
+    print(f"✓ Converted {len(response['documents'])} documents")
+    print(f"  Summary: {response['summary']}")
+    
+    for doc in response["documents"]:
+        print(f"  - {doc['id']}: {len(doc['unmapped_fields'])} unmapped fields")
+```
+
+### Common Errors
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| `FileNotFoundError` | JSON file not found | Verify file path exists |
+| `JSONDecodeError` | Invalid JSON format | Validate JSON syntax |
+| `MISSING_DOCUMENT_TYPE` | Document missing `document_type` | Add type to all documents |
+
+---
+
+## API 3: `json_file_to_ubl_xml_files()`
+
+Convert JSON file and write XML files directly to disk.
+
+### Function Signature
+
+```python
+from json2ubl import json_file_to_ubl_xml_files
+
+response = json_file_to_ubl_xml_files(
+    json_file_path: str,
+    output_dir: str,
+    config_path: str | None = None
+) -> Dict[str, Any]
+```
+
+### Parameters
+
+| Parameter | Type | Required | Description |
+|-----------|------|----------|-------------|
+| `json_file_path` | `str` | Yes | Path to input JSON file |
+| `output_dir` | `str` | Yes | Directory where XML files will be written |
+| `config_path` | `str` | No | Path to optional config file |
+
+### Response Format
+
+Same structure as API 1 & 2, but **without** XML content:
+
+```python
+{
+    "documents": [
+        {
+            "id": "INV-001",
+            "unmapped_fields": []
+        },
+        {
+            "id": "INV-002",
+            "unmapped_fields": []
+        }
+    ],
+    "summary": {
+        "total_inputs": 2,
+        "files_created": 2,
+        "document_types": {"Invoice": 2}
+    },
+    "error_response": None
+}
+```
+
+### Response Fields
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `documents` | `List[Dict]` | List of processed documents (no XML) |
+| `documents[].id` | `str` | Document identifier |
+| `documents[].unmapped_fields` | `List[str]` | Fields not in UBL schema |
+| `summary.files_created` | `int` | Number of XML files written to disk |
+| `error_response` | `Dict\|None` | Error if conversion failed |
+
+**Note:** `xml` field is omitted to reduce response payload when writing to disk.
+
+### Output Files
+
+Generated files are named: `{document_id}.xml`
+
+```
+output_dir/
+├── INV-001.xml
+├── INV-002.xml
+└── CR-001.xml
+```
+
+### Usage Example
+
+```python
+from json2ubl import json_file_to_ubl_xml_files
+import os
+
+response = json_file_to_ubl_xml_files(
+    json_file_path="batch_invoices.json",
+    output_dir="./output_xml"
+)
+
+if response.get("error_response"):
+    print(f"❌ Error: {response['error_response']['message']}")
+else:
+    files_created = response["summary"]["files_created"]
+    print(f"✓ Successfully created {files_created} XML files")
+    print(f"  Location: {os.path.abspath('./output_xml')}")
+    
+    for doc in response["documents"]:
+        unmapped = len(doc["unmapped_fields"])
+        print(f"  - {doc['id']}: {unmapped} unmapped fields")
+```
+
+### Common Errors
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| `FileNotFoundError` | Input file not found | Verify input JSON path |
+| `PermissionError` | Cannot write to output_dir | Check directory permissions |
+| `INVALID_DOCUMENT_TYPE` | Document type invalid | Fix document_type values |
+
+---
+
+## Supported Document Types
+
+The converter supports all 60+ UBL 2.1 document types. Common ones:
+
+| Code | Document Type |
+|------|----------------|
+| 380 | Invoice |
+| 381 | Credit Note |
+| 382 | Debit Note |
+| 220 | Order |
+| 225 | Order Change |
+| 230 | Order Cancellation |
+
+For full list, see [UBL 2.1 Specification](https://docs.oasis-open.org/ubl/os-UBL-2.1/)
+
+---
+
+## Field Mapping
+
+- Input JSON keys are **case-insensitive** (automatically normalized to lowercase)
+- Fields not in UBL schema are tracked in `unmapped_fields`
+- Null values are preserved as empty XML elements
+- Nested structures follow UBL hierarchy automatically
+
+---
+
+## Configuration (Optional)
+
+Create `ubl_converter.yaml` for custom settings:
+
+```yaml
+schema_root: "./custom_schemas"
+max_recursion_depth: 20
+enable_logging: true
+log_file: "converter.log"
+```
+
+Pass to any API:
+```python
+response = json_dict_to_ubl_xml(invoices, config_path="ubl_converter.yaml")
+```
+
+---
+
+## Best Practices
+
+1. **Validate input before conversion** - Ensure `id` and `document_type` are present
+2. **Check `unmapped_fields`** - Review fields not in UBL schema
+3. **Handle errors gracefully** - Always check `error_response` before processing
+4. **Use API 3 for large batches** - Better for 1000+ documents (writes to disk directly)
+5. **Log conversions** - Track which documents failed for auditing
+
+---
+
+## Error Handling Pattern
+
+```python
+from json2ubl import json_dict_to_ubl_xml
+
+def convert_safely(documents):
+    response = json_dict_to_ubl_xml(documents)
+    
+    # Check for errors
+    if response.get("error_response"):
+        error = response["error_response"]
+        print(f"Error Code: {error['error_code']}")
+        print(f"Message: {error['message']}")
+        print(f"Details: {error['details']}")
+        return None
+    
+    # Process successful conversions
+    for doc in response["documents"]:
+        if doc["unmapped_fields"]:
+            print(f"Warning: {doc['id']} has unmapped fields: {doc['unmapped_fields']}")
+        
+        # Use XML or save to file
+        print(f"✓ {doc['id']} converted successfully")
+    
+    return response
+
+# Usage
+result = convert_safely(invoices)
+```
+
+---
+
+## Summary
+
+| API | Use Case | Output | Best For |
+|-----|----------|--------|----------|
+| **API 1** | Single/few documents | XML in memory | Real-time API responses |
+| **API 2** | Batch processing | XML in memory | Reports, archives |
+| **API 3** | Bulk files | XML on disk | High-volume batch jobs |
+
+All three APIs return consistent response structure with documents, summary, and error_response fields.

{json2ubl-1.0.0 → json2ubl-1.0.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "json2ubl"
-version = "1.0.0"
+version = "1.0.1"
 description = "Production-grade JSON to UBL 2.1 XML converter with schema-driven mapping"
 readme = "README.md"
 requires-python = ">=3.10"
@@ -27,19 +27,19 @@ classifiers = [
 ]
 
 dependencies = [
-    "lxml>=4.9.4,<5.0",
-    "pydantic>=2.7.0,<3.0",
-    "pyyaml>=6.0.1,<7.0",
-    "loguru>=0.7.2,<1.0",
+    "lxml~=6.0",
+    "pydantic~=2.7",
+    "pyyaml~=6.0",
+    "loguru~=0.7",
 ]
 
 [project.optional-dependencies]
 dev = [
-    "pytest>=8.3.0,<9.0",
-    "pytest-cov>=5.0.0,<6.0",
-    "black>=24.10.0,<25.0",
-    "ruff>=0.8.0,<1.0",
-    "mypy>=1.13.0,<2.0",
+    "pytest~=8.3",
+    "pytest-cov~=5.0",
+    "black~=24.10",
+    "ruff~=0.8",
+    "mypy~=1.13",
 ]
 
 [project.urls]