chunkr-ai 0.0.2__tar.gz → 0.0.4__tar.gz

chunkr_ai-0.0.4/PKG-INFO

@@ -0,0 +1,204 @@
+Metadata-Version: 2.2
+Name: chunkr-ai
+Version: 0.0.4
+Summary: Python client for Chunkr: open source document intelligence
+Author-email: Ishaan Kapoor <ishaan@lumina.sh>
+Project-URL: Homepage, https://chunkr.ai
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: httpx>=0.28.1
+Requires-Dist: pillow>=11.1.0
+Requires-Dist: pydantic>=2.10.4
+Requires-Dist: python-dotenv>=1.0.1
+Requires-Dist: requests>=2.32.3
+Provides-Extra: test
+Requires-Dist: pytest>=8.3.4; extra == "test"
+Requires-Dist: pytest-xdist>=3.6.1; extra == "test"
+
+# Chunkr Python Client
+
+This provides a simple interface to interact with the Chunkr API.
+
+## Getting Started
+
+You can get an API key from [Chunkr](https://chunkr.ai) or deploy your own Chunkr instance. For self-hosted deployment options, check out our [deployment guide](https://github.com/lumina-ai-inc/chunkr/tree/main?tab=readme-ov-file#self-hosted-deployment-options).
+
+For more information about the API and its capabilities, visit the [Chunkr API docs](https://docs.chunkr.ai).
+
+## Installation
+
+```bash
+pip install chunkr-ai
+```
+
+## Usage
+
+We provide two clients: `Chunkr` for synchronous operations and `ChunkrAsync` for asynchronous operations.
+
+### Synchronous Usage
+
+```python
+from chunkr_ai import Chunkr
+
+# Initialize client
+chunkr = Chunkr()
+
+# Upload a file and wait for processing
+task = chunkr.upload("document.pdf")
+
+# Print the response
+print(task)
+
+# Get output from task
+output = task.output
+
+# If you want to upload without waiting for processing
+task = chunkr.start_upload("document.pdf")
+# ... do other things ...
+task.poll()  # Check status when needed
+```
+
+### Asynchronous Usage
+
+```python
+from chunkr_ai import ChunkrAsync
+
+async def process_document():
+    # Initialize client
+    chunkr = ChunkrAsync()
+
+    # Upload a file and wait for processing
+    task = await chunkr.upload("document.pdf")
+
+    # Print the response
+    print(task)
+
+    # Get output from task
+    output = task.output
+
+    # If you want to upload without waiting for processing
+    task = await chunkr.start_upload("document.pdf")
+    # ... do other things ...
+    await task.poll_async()  # Check status when needed
+```
+
+### Additional Features
+
+Both clients support various input types:
+
+```python
+# Upload from file path
+chunkr.upload("document.pdf")
+
+# Upload from opened file
+with open("document.pdf", "rb") as f:
+    chunkr.upload(f)
+
+# Upload from URL
+chunkr.upload("https://example.com/document.pdf")
+
+# Upload from base64 string
+chunkr.upload("data:application/pdf;base64,JVBERi0xLjcKCjEgMCBvYmo...")
+
+# Upload an image
+from PIL import Image
+img = Image.open("photo.jpg")
+chunkr.upload(img)
+```
+
+### Configuration
+
+You can customize the processing behavior by passing a `Configuration` object:
+
+```python
+from chunkr_ai.models import Configuration, OcrStrategy, SegmentationStrategy, GenerationStrategy
+
+# Basic configuration
+config = Configuration(
+    ocr_strategy=OcrStrategy.AUTO,
+    segmentation_strategy=SegmentationStrategy.LAYOUT_ANALYSIS,
+    high_resolution=True,
+    expires_in=3600,  # seconds
+)
+
+# Upload with configuration
+task = chunkr.upload("document.pdf", config)
+```
+
+#### Available Configuration Examples
+
+- **Chunk Processing**
+  ```python
+  from chunkr_ai.models import ChunkProcessing
+  config = Configuration(
+      chunk_processing=ChunkProcessing(target_length=1024)
+  )
+  ```
+- **Expires In**
+  ```python
+  config = Configuration(expires_in=3600)
+  ```
+
+- **High Resolution**
+  ```python
+  config = Configuration(high_resolution=True)
+  ```
+
+- **JSON Schema**
+  ```python
+  config = Configuration(json_schema=JsonSchema(
+      title="Sales Data",
+      properties=[
+          Property(name="Person with highest sales", prop_type="string", description="The person with the highest sales"),
+          Property(name="Person with lowest sales", prop_type="string", description="The person with the lowest sales"),
+      ]
+  ))
+  ```
+
+- **OCR Strategy**
+  ```python
+  config = Configuration(ocr_strategy=OcrStrategy.AUTO)
+  ```
+
+- **Segment Processing**
+  ```python
+  from chunkr_ai.models import SegmentProcessing, GenerationConfig, GenerationStrategy
+  config = Configuration(
+      segment_processing=SegmentProcessing(
+          page=GenerationConfig(
+              html=GenerationStrategy.LLM,
+              markdown=GenerationStrategy.LLM
+          )
+      )
+  )
+  ```
+
+- **Segmentation Strategy**
+  ```python
+  config = Configuration(
+      segmentation_strategy=SegmentationStrategy.LAYOUT_ANALYSIS  # or SegmentationStrategy.PAGE
+  )
+  ```
+
+## Environment setup
+
+You can provide your API key and URL in several ways:
+1. Environment variables: `CHUNKR_API_KEY` and `CHUNKR_URL`
+2. `.env` file
+3. Direct initialization:
+```python
+chunkr = Chunkr(
+    api_key="your-api-key",
+    url="https://api.chunkr.ai"
+)
+```
+
+## Run tests
+
+```python
+# Install dependencies
+uv pip install -e ".[test]"
+
+# Run tests
+uv run pytest
+```

chunkr_ai-0.0.4/README.md

@@ -0,0 +1,187 @@
+# Chunkr Python Client
+
+This provides a simple interface to interact with the Chunkr API.
+
+## Getting Started
+
+You can get an API key from [Chunkr](https://chunkr.ai) or deploy your own Chunkr instance. For self-hosted deployment options, check out our [deployment guide](https://github.com/lumina-ai-inc/chunkr/tree/main?tab=readme-ov-file#self-hosted-deployment-options).
+
+For more information about the API and its capabilities, visit the [Chunkr API docs](https://docs.chunkr.ai).
+
+## Installation
+
+```bash
+pip install chunkr-ai
+```
+
+## Usage
+
+We provide two clients: `Chunkr` for synchronous operations and `ChunkrAsync` for asynchronous operations.
+
+### Synchronous Usage
+
+```python
+from chunkr_ai import Chunkr
+
+# Initialize client
+chunkr = Chunkr()
+
+# Upload a file and wait for processing
+task = chunkr.upload("document.pdf")
+
+# Print the response
+print(task)
+
+# Get output from task
+output = task.output
+
+# If you want to upload without waiting for processing
+task = chunkr.start_upload("document.pdf")
+# ... do other things ...
+task.poll()  # Check status when needed
+```
+
+### Asynchronous Usage
+
+```python
+from chunkr_ai import ChunkrAsync
+
+async def process_document():
+    # Initialize client
+    chunkr = ChunkrAsync()
+
+    # Upload a file and wait for processing
+    task = await chunkr.upload("document.pdf")
+
+    # Print the response
+    print(task)
+
+    # Get output from task
+    output = task.output
+
+    # If you want to upload without waiting for processing
+    task = await chunkr.start_upload("document.pdf")
+    # ... do other things ...
+    await task.poll_async()  # Check status when needed
+```
+
+### Additional Features
+
+Both clients support various input types:
+
+```python
+# Upload from file path
+chunkr.upload("document.pdf")
+
+# Upload from opened file
+with open("document.pdf", "rb") as f:
+    chunkr.upload(f)
+
+# Upload from URL
+chunkr.upload("https://example.com/document.pdf")
+
+# Upload from base64 string
+chunkr.upload("data:application/pdf;base64,JVBERi0xLjcKCjEgMCBvYmo...")
+
+# Upload an image
+from PIL import Image
+img = Image.open("photo.jpg")
+chunkr.upload(img)
+```
+
+### Configuration
+
+You can customize the processing behavior by passing a `Configuration` object:
+
+```python
+from chunkr_ai.models import Configuration, OcrStrategy, SegmentationStrategy, GenerationStrategy
+
+# Basic configuration
+config = Configuration(
+    ocr_strategy=OcrStrategy.AUTO,
+    segmentation_strategy=SegmentationStrategy.LAYOUT_ANALYSIS,
+    high_resolution=True,
+    expires_in=3600,  # seconds
+)
+
+# Upload with configuration
+task = chunkr.upload("document.pdf", config)
+```
+
+#### Available Configuration Examples
+
+- **Chunk Processing**
+  ```python
+  from chunkr_ai.models import ChunkProcessing
+  config = Configuration(
+      chunk_processing=ChunkProcessing(target_length=1024)
+  )
+  ```
+- **Expires In**
+  ```python
+  config = Configuration(expires_in=3600)
+  ```
+
+- **High Resolution**
+  ```python
+  config = Configuration(high_resolution=True)
+  ```
+
+- **JSON Schema**
+  ```python
+  config = Configuration(json_schema=JsonSchema(
+      title="Sales Data",
+      properties=[
+          Property(name="Person with highest sales", prop_type="string", description="The person with the highest sales"),
+          Property(name="Person with lowest sales", prop_type="string", description="The person with the lowest sales"),
+      ]
+  ))
+  ```
+
+- **OCR Strategy**
+  ```python
+  config = Configuration(ocr_strategy=OcrStrategy.AUTO)
+  ```
+
+- **Segment Processing**
+  ```python
+  from chunkr_ai.models import SegmentProcessing, GenerationConfig, GenerationStrategy
+  config = Configuration(
+      segment_processing=SegmentProcessing(
+          page=GenerationConfig(
+              html=GenerationStrategy.LLM,
+              markdown=GenerationStrategy.LLM
+          )
+      )
+  )
+  ```
+
+- **Segmentation Strategy**
+  ```python
+  config = Configuration(
+      segmentation_strategy=SegmentationStrategy.LAYOUT_ANALYSIS  # or SegmentationStrategy.PAGE
+  )
+  ```
+
+## Environment setup
+
+You can provide your API key and URL in several ways:
+1. Environment variables: `CHUNKR_API_KEY` and `CHUNKR_URL`
+2. `.env` file
+3. Direct initialization:
+```python
+chunkr = Chunkr(
+    api_key="your-api-key",
+    url="https://api.chunkr.ai"
+)
+```
+
+## Run tests
+
+```python
+# Install dependencies
+uv pip install -e ".[test]"
+
+# Run tests
+uv run pytest
+```

{chunkr_ai-0.0.2 → chunkr_ai-0.0.4}/pyproject.toml

@@ -4,22 +4,22 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "chunkr-ai"
-version = "0.0.2"
+version = "0.0.4"
 authors = [{"name" = "Ishaan Kapoor", "email" = "ishaan@lumina.sh"}]
-description = "Python client for chunkr: open source document intelligence"
+description = "Python client for Chunkr: open source document intelligence"
 readme = "README.md"
 license = {"file" = "LICENSE"}
+urls = {Homepage = "https://chunkr.ai"} 
 dependencies = [
-    "build>=1.2.2.post1",
     "httpx>=0.28.1",
     "pillow>=11.1.0",
     "pydantic>=2.10.4",
     "python-dotenv>=1.0.1",
     "requests>=2.32.3",
-    "twine>=6.0.1",
 ]
 
 [project.optional-dependencies]
 test = [
     "pytest>=8.3.4",
+    "pytest-xdist>=3.6.1",
 ]

{chunkr_ai-0.0.2 → chunkr_ai-0.0.4}/src/chunkr_ai/api/auth.py

@@ -1,5 +1,3 @@
-from typing import Optional
-
 class HeadersMixin:
     """Mixin class for handling authorization headers"""
     

chunkr_ai-0.0.4/src/chunkr_ai/api/base.py

@@ -0,0 +1,173 @@
+from .config import Configuration
+from .task import TaskResponse
+from .auth import HeadersMixin
+from abc import abstractmethod
+from dotenv import load_dotenv
+import io
+import json
+import os
+from pathlib import Path
+from PIL import Image
+import requests
+from typing import BinaryIO, Tuple, Union
+
+class ChunkrBase(HeadersMixin):
+    """Base class with shared functionality for Chunkr API clients."""
+
+    def __init__(self, url: str = None, api_key: str = None):
+        load_dotenv()
+        self.url = (
+            url or 
+            os.getenv('CHUNKR_URL') or 
+            'https://api.chunkr.ai' 
+        )
+        self._api_key = (
+            api_key or 
+            os.getenv('CHUNKR_API_KEY')
+        )
+        if not self._api_key:
+            raise ValueError("API key must be provided either directly, in .env file, or as CHUNKR_API_KEY environment variable. You can get an api key at: https://www.chunkr.ai")
+            
+        self.url = self.url.rstrip("/")
+
+    def _prepare_file(
+        self,
+        file: Union[str, Path, BinaryIO, Image.Image]
+    ) -> Tuple[str, BinaryIO]:
+        """Convert various file types into a tuple of (filename, file-like object).
+
+        Args:
+            file: Input file, can be:
+                - String or Path to a file
+                - URL string starting with http:// or https://
+                - Base64 string
+                - Opened binary file (mode='rb')
+                - PIL/Pillow Image object
+
+        Returns:
+            Tuple[str, BinaryIO]: (filename, file-like object) ready for upload
+
+        Raises:
+            FileNotFoundError: If the file path doesn't exist
+            TypeError: If the file type is not supported
+            ValueError: If the URL is invalid or unreachable
+            ValueError: If the MIME type is unsupported
+        """
+        # Handle URLs
+        if isinstance(file, str) and (file.startswith('http://') or file.startswith('https://')):
+            response = requests.get(file)
+            response.raise_for_status()
+            file_obj = io.BytesIO(response.content)
+            filename = Path(file.split('/')[-1]).name or 'downloaded_file'
+            return filename, file_obj
+
+        # Handle base64 strings
+        if isinstance(file, str) and ',' in file and ';base64,' in file:
+            try:
+                # Split header and data
+                header, base64_data = file.split(',', 1)
+                import base64
+                file_bytes = base64.b64decode(base64_data)
+                file_obj = io.BytesIO(file_bytes)
+                
+                # Try to determine format from header
+                format = 'bin'
+                mime_type = header.split(':')[-1].split(';')[0].lower()
+                
+                # Map MIME types to file extensions
+                mime_to_ext = {
+                    'application/pdf': 'pdf',
+                    'application/vnd.openxmlformats-officedocument.wordprocessingml.document': 'docx',
+                    'application/msword': 'doc',
+                    'application/vnd.openxmlformats-officedocument.presentationml.presentation': 'pptx',
+                    'application/vnd.ms-powerpoint': 'ppt',
+                    'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet': 'xlsx',
+                    'application/vnd.ms-excel': 'xls',
+                    'image/jpeg': 'jpg',
+                    'image/png': 'png',
+                    'image/jpg': 'jpg'
+                }
+                
+                if mime_type in mime_to_ext:
+                    format = mime_to_ext[mime_type]
+                else:
+                    raise ValueError(f"Unsupported MIME type: {mime_type}")
+
+                return f"file.{format}", file_obj
+            except Exception as e:
+                raise ValueError(f"Invalid base64 string: {str(e)}")
+
+        # Handle file paths
+        if isinstance(file, (str, Path)):
+            path = Path(file).resolve()
+            if not path.exists():
+                raise FileNotFoundError(f"File not found: {file}")
+            return path.name, open(path, 'rb')
+
+        # Handle PIL Images
+        if isinstance(file, Image.Image):
+            img_byte_arr = io.BytesIO()
+            format = file.format or 'PNG'
+            file.save(img_byte_arr, format=format)
+            img_byte_arr.seek(0)
+            return f"image.{format.lower()}", img_byte_arr
+
+        # Handle file-like objects
+        if hasattr(file, 'read') and hasattr(file, 'seek'):
+            # Try to get the filename from the file object if possible
+            name = getattr(file, 'name', 'document') if hasattr(file, 'name') else 'document'
+            return Path(name).name, file
+
+        raise TypeError(f"Unsupported file type: {type(file)}")
+
+    def _prepare_upload_data(
+        self,
+        file: Union[str, Path, BinaryIO, Image.Image],
+        config: Configuration = None
+    ) -> Tuple[dict, dict]:
+        """Prepare files and data dictionaries for upload.
+        
+        Args:
+            file: The file to upload
+            config: Optional configuration settings
+            
+        Returns:
+            Tuple[dict, dict]: (files dict, data dict) ready for upload
+        """
+        filename, file_obj = self._prepare_file(file)
+        files = {"file": (filename, file_obj)}
+        data = {}
+        
+        if config:
+            config_dict = config.model_dump(mode="json", exclude_none=True)
+            for key, value in config_dict.items():
+                if isinstance(value, dict):
+                    files[key] = (None, json.dumps(value), 'application/json')
+                else:
+                    data[key] = value
+                    
+        return files, data
+    
+    @abstractmethod
+    def upload(self, file: Union[str, Path, BinaryIO, Image.Image], config: Configuration = None) -> TaskResponse:
+        """Upload a file and wait for processing to complete.
+        
+        Must be implemented by subclasses.
+        """
+        pass
+
+    @abstractmethod
+    def start_upload(self, file: Union[str, Path, BinaryIO, Image.Image], config: Configuration = None) -> TaskResponse:
+        """Upload a file for processing and immediately return the task response.
+        
+        Must be implemented by subclasses.
+        """
+        pass
+
+    @abstractmethod
+    def get_task(self, task_id: str) -> TaskResponse:
+        """Get a task response by its ID.
+        
+        Must be implemented by subclasses.
+        """
+        pass

chunkr_ai-0.0.4/src/chunkr_ai/api/chunkr.py

@@ -0,0 +1,108 @@
+from .base import ChunkrBase
+from .config import Configuration
+from .task import TaskResponse
+from pathlib import Path
+from PIL import Image
+import requests
+from typing import Union, BinaryIO
+
+class Chunkr(ChunkrBase):
+    """Chunkr API client"""
+
+    def __init__(self, url: str = None, api_key: str = None):
+        super().__init__(url, api_key)
+        self._session = requests.Session()
+
+    def upload(self, file: Union[str, Path, BinaryIO, Image.Image], config: Configuration = None) -> TaskResponse:
+        """Upload a file and wait for processing to complete.
+
+        Args:
+            file: The file to upload. 
+            config: Configuration options for processing. Optional.
+
+        Examples:
+        ```
+        # Upload from file path
+        chunkr.upload("document.pdf")
+
+        # Upload from URL
+        chunkr.upload("https://example.com/document.pdf")
+
+        # Upload from base64 string (must include MIME type header)
+        chunkr.upload("data:application/pdf;base64,JVBERi0xLjcKCjEgMCBvYmo...")
+
+        # Upload from opened file
+        with open("document.pdf", "rb") as f:
+            chunkr.upload(f)
+
+        # Upload an image
+        from PIL import Image
+        img = Image.open("photo.jpg")
+        chunkr.upload(img)
+        ```
+        Returns:
+            TaskResponse: The completed task response
+        """
+        task = self.start_upload(file, config)
+        return task.poll()
+
+    def start_upload(self, file: Union[str, Path, BinaryIO, Image.Image], config: Configuration = None) -> TaskResponse:
+        """Upload a file for processing and immediately return the task response. It will not wait for processing to complete. To wait for the full processing to complete, use `task.poll()`
+
+        Args:
+            file: The file to upload.
+            config: Configuration options for processing. Optional.
+
+        Examples:
+        ```
+        # Upload from file path
+        task = chunkr.start_upload("document.pdf")
+
+        # Upload from opened file
+        with open("document.pdf", "rb") as f:
+            task = chunkr.start_upload(f)
+
+        # Upload from URL
+        task = chunkr.start_upload("https://example.com/document.pdf")
+
+        # Upload from base64 string (must include MIME type header)
+        task = chunkr.start_upload("data:application/pdf;base64,JVBERi0xLjcKCjEgMCBvYmo...")
+
+        # Upload an image
+        from PIL import Image
+        img = Image.open("photo.jpg")
+        task = chunkr.start_upload(img)
+
+        # Wait for the task to complete - this can be done when needed
+        task.poll()
+        ```
+
+        Returns:
+            TaskResponse: The initial task response
+        """
+        files, data = self._prepare_upload_data(file, config)
+        r = self._session.post(
+            f"{self.url}/api/v1/task",
+            files=files,
+            data=data,  
+            headers=self._headers()
+        )
+        r.raise_for_status()
+        return TaskResponse(**r.json()).with_client(self)
+
+    def get_task(self, task_id: str) -> TaskResponse:
+        """Get a task response by its ID.
+        
+        Args:
+            task_id: The ID of the task to get
+
+        Returns:
+            TaskResponse: The task response
+        """
+        r = self._session.get(
+            f"{self.url}/api/v1/task/{task_id}",
+            headers=self._headers()
+        )
+        r.raise_for_status()
+        return TaskResponse(**r.json()).with_client(self)
+