prevectorchunks-core 0.1.24__tar.gz → 0.1.25__tar.gz

{prevectorchunks_core-0.1.24 → prevectorchunks_core-0.1.25}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: prevectorchunks-core
-Version: 0.1.24
+Version: 0.1.25
 Summary: A Python module that allows conversion of a document into chunks to be inserted into Pinecone vector database
 Author-email: Zul Al-Kabir <zul.developer.2023@gmail.com>
 Project-URL: Homepage, https://github.com/yourusername/mydep
@@ -187,6 +187,41 @@ Updates existing chunks in the Vector Database by document name.
 - Keeping VDB chunks up to date when documents change
 - Re-ingesting revised or corrected content
 
+---
+### 5. ``markdown_and_chunk_documents``
+```python
+from prevectorchunks_core.services.markdown_and_chunk_documents import MarkdownAndChunkDocuments
+
+markdown_processor = MarkdownAndChunkDocuments()
+mapped_chunks = markdown_processor.markdown_and_chunk_documents("example.pdf")
+```
+
+**Description**  
+This new function automatically:
+1. Converts a document (PDF, DOCX, etc.) into images using `DocuToImageConverter`.
+2. Extracts **Markdown and text** content from those images using `DocuToMarkdownExtractor` (powered by GPT).
+3. Converts the extracted markdown text into **RL-based chunks** using `ChunkMapper` and `chunk_documents`.
+4. Merges unmatched markdown segments into the final structured output.
+
+**Parameters**
+- `file_path` (*str*): Path to the document (PDF, DOCX, or image) you want to process.
+
+**Returns**
+- `mapped_chunks` (*list[dict]*): A list of markdown-based chunks with both markdown and chunked text content.
+
+**Example**
+```python
+if __name__ == "__main__":
+    markdown_processor = MarkdownAndChunkDocuments()
+    mapped_chunks = markdown_processor.markdown_and_chunk_documents("421307-nz-au-top-loading-washer-guide-shorter.pdf")
+    print(mapped_chunks)
+```
+
+**Use Cases**
+- End-to-end document-to-markdown-to-chunks pipeline
+- Automating preprocessing for RAG/LLM ingestion
+- Extracting structured markdown for semantic search or content indexing
+
 ---
 
 ## 🚀 Example Workflow

{prevectorchunks_core-0.1.24 → prevectorchunks_core-0.1.25}/README.md

@@ -142,6 +142,41 @@ Updates existing chunks in the Vector Database by document name.
 - Keeping VDB chunks up to date when documents change
 - Re-ingesting revised or corrected content
 
+---
+### 5. ``markdown_and_chunk_documents``
+```python
+from prevectorchunks_core.services.markdown_and_chunk_documents import MarkdownAndChunkDocuments
+
+markdown_processor = MarkdownAndChunkDocuments()
+mapped_chunks = markdown_processor.markdown_and_chunk_documents("example.pdf")
+```
+
+**Description**  
+This new function automatically:
+1. Converts a document (PDF, DOCX, etc.) into images using `DocuToImageConverter`.
+2. Extracts **Markdown and text** content from those images using `DocuToMarkdownExtractor` (powered by GPT).
+3. Converts the extracted markdown text into **RL-based chunks** using `ChunkMapper` and `chunk_documents`.
+4. Merges unmatched markdown segments into the final structured output.
+
+**Parameters**
+- `file_path` (*str*): Path to the document (PDF, DOCX, or image) you want to process.
+
+**Returns**
+- `mapped_chunks` (*list[dict]*): A list of markdown-based chunks with both markdown and chunked text content.
+
+**Example**
+```python
+if __name__ == "__main__":
+    markdown_processor = MarkdownAndChunkDocuments()
+    mapped_chunks = markdown_processor.markdown_and_chunk_documents("421307-nz-au-top-loading-washer-guide-shorter.pdf")
+    print(mapped_chunks)
+```
+
+**Use Cases**
+- End-to-end document-to-markdown-to-chunks pipeline
+- Automating preprocessing for RAG/LLM ingestion
+- Extracting structured markdown for semantic search or content indexing
+
 ---
 
 ## 🚀 Example Workflow

{prevectorchunks_core-0.1.24 → prevectorchunks_core-0.1.25}/prevectorchunks_core.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: prevectorchunks-core
-Version: 0.1.24
+Version: 0.1.25
 Summary: A Python module that allows conversion of a document into chunks to be inserted into Pinecone vector database
 Author-email: Zul Al-Kabir <zul.developer.2023@gmail.com>
 Project-URL: Homepage, https://github.com/yourusername/mydep
@@ -187,6 +187,41 @@ Updates existing chunks in the Vector Database by document name.
 - Keeping VDB chunks up to date when documents change
 - Re-ingesting revised or corrected content
 
+---
+### 5. ``markdown_and_chunk_documents``
+```python
+from prevectorchunks_core.services.markdown_and_chunk_documents import MarkdownAndChunkDocuments
+
+markdown_processor = MarkdownAndChunkDocuments()
+mapped_chunks = markdown_processor.markdown_and_chunk_documents("example.pdf")
+```
+
+**Description**  
+This new function automatically:
+1. Converts a document (PDF, DOCX, etc.) into images using `DocuToImageConverter`.
+2. Extracts **Markdown and text** content from those images using `DocuToMarkdownExtractor` (powered by GPT).
+3. Converts the extracted markdown text into **RL-based chunks** using `ChunkMapper` and `chunk_documents`.
+4. Merges unmatched markdown segments into the final structured output.
+
+**Parameters**
+- `file_path` (*str*): Path to the document (PDF, DOCX, or image) you want to process.
+
+**Returns**
+- `mapped_chunks` (*list[dict]*): A list of markdown-based chunks with both markdown and chunked text content.
+
+**Example**
+```python
+if __name__ == "__main__":
+    markdown_processor = MarkdownAndChunkDocuments()
+    mapped_chunks = markdown_processor.markdown_and_chunk_documents("421307-nz-au-top-loading-washer-guide-shorter.pdf")
+    print(mapped_chunks)
+```
+
+**Use Cases**
+- End-to-end document-to-markdown-to-chunks pipeline
+- Automating preprocessing for RAG/LLM ingestion
+- Extracting structured markdown for semantic search or content indexing
+
 ---
 
 ## 🚀 Example Workflow

{prevectorchunks_core-0.1.24 → prevectorchunks_core-0.1.25}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "prevectorchunks-core"
-version = "0.1.24"
+version = "0.1.25"
 description = "A Python module that allows conversion of a document into chunks to be inserted into Pinecone vector database"
 readme = "README.md"
 license = { file = "LICENSE" }