parqv 0.1.0__tar.gz → 0.2.0__tar.gz

parqv-0.2.0/PKG-INFO

@@ -0,0 +1,104 @@
+Metadata-Version: 2.4
+Name: parqv
+Version: 0.2.0
+Summary: An interactive Python TUI for visualizing, exploring, and analyzing files directly in your terminal.
+Author-email: Sangmin Yoon <sanspareilsmyn@gmail.com>
+License-Expression: Apache-2.0
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: textual>=1.0.0
+Requires-Dist: pyarrow>=16.0.0
+Requires-Dist: pandas>=2.0.0
+Requires-Dist: numpy>=1.20.0
+Requires-Dist: duckdb>=1.2.0
+Dynamic: license-file
+
+# parqv
+
+[![Python Version](https://img.shields.io/badge/Python-3.10+-blue.svg)](https://www.python.org/)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
+[![PyPI version](https://badge.fury.io/py/parqv.svg)](https://badge.fury.io/py/parqv) <!-- TODO: Link after first PyPI release -->
+[![Built with Textual](https://img.shields.io/badge/Built%20with-Textual-blueviolet.svg)](https://textual.textualize.io/)
+
+---
+
+**Supported File Formats:** ✅ **Parquet** | ✅ **JSON** / **JSON Lines (ndjson)** | *(More planned!)*
+
+---
+
+**`parqv` is a Python-based interactive TUI (Text User Interface) tool designed to explore, analyze, and understand various data file formats directly within your terminal.** Initially supporting Parquet and JSON, `parqv` aims to provide a unified, visual experience for quick data inspection without leaving your console.
+
+## 💻 Demo (Showing Parquet)
+
+![parqv.gif](assets/parqv.gif)
+*(Demo shows Parquet features; UI adapts for other formats)*
+
+## 🤔 Why `parqv`?
+1.  **Unified Interface:** Launch `parqv <your_data_file>` to access **metadata, schema, data preview, and column statistics** all within a single, navigable terminal window. No more juggling different commands for different file types.
+2.  **Interactive Exploration:**
+    *   **🖱️ Keyboard & Mouse Driven:** Navigate using familiar keys (arrows, `hjkl`, Tab) or even your mouse (thanks to `Textual`).
+    *   **📜 Scrollable Views:** Easily scroll through large schemas, data tables, or column lists.
+    *   **🌲 Clear Schema View:** Understand column names, data types, and nullability at a glance. (Complex nested structures visualization might vary by format).
+    *   **📊 Dynamic Stats:** Select a column and instantly see its detailed statistics (counts, nulls, min/max, mean, distinct values, etc.).
+3.  **Cross-Format Consistency:**
+    *   **🎨 Rich Display:** Leverages `rich` and `Textual` for colorful, readable tables and text across supported formats.
+    *   **📈 Quick Stats:** Get key statistical insights consistently, regardless of the underlying file type.
+    *   **🔌 Extensible:** Designed with a handler interface to easily add support for more file formats in the future (like CSV, Arrow IPC, etc.).
+
+## ✨ Features (TUI Mode)
+*   **Multi-Format Support:** Currently supports **Parquet** (`.parquet`) and **JSON/JSON Lines** (`.json`, `.ndjson`). Run `parqv <your_file.{parquet,json,ndjson}>`.
+*   **Metadata Panel:** Displays key file information (path, format, size, total rows, column count, etc.). *Fields may vary slightly depending on the file format.*
+*   **Schema Explorer:**
+    *   Interactive list view of columns.
+    *   Clearly shows column names, data types, and nullability.
+*   **Data Table Viewer:**
+    *   Scrollable table preview of the file's data.
+    *   Attempts to preserve data types for better representation.
+*   **Column Statistics Viewer:**
+    *   Select a column in the Schema tab to view detailed statistics.
+    *   Shows counts (total, valid, null), percentages, and type-specific stats (min/max, mean, stddev, distinct counts, length stats, boolean value counts where applicable).
+*   **Row Group Inspector (Parquet Specific):**
+    *   *This panel only appears when viewing Parquet files.*
+    *   Lists row groups with stats (row count, compressed/uncompressed size).
+    *   (Planned) Select a row group for more details.
+
+## 🚀 Getting Started
+
+**1. Prerequisites:**
+*   **Python:** Version 3.10 or higher.
+*   **pip:** The Python package installer.
+
+**2. Install `parqv`:**
+*   Open your terminal and run:
+    ```bash
+    pip install parqv
+    ```
+    *(This will also install dependencies like `textual`, `pyarrow`, `pandas`, and `duckdb`)*
+*   **Updating `parqv`:**
+    ```bash
+    pip install --upgrade parqv
+    ```
+
+**3. Run `parqv`:**
+*   Point `parqv` to your data file:
+    ```bash
+    #parquet
+    parqv /path/to/your/data.parquet
+    
+    # json
+    parqv /path/to/your/data.json
+*   The interactive TUI will launch. Use your keyboard (and mouse, if supported by your terminal) to navigate:
+    *   **Arrow Keys / `j`,`k` (in lists):** Move selection up/down.
+    *   **`Tab` / `Shift+Tab`:** Cycle focus between the main tab content and potentially other areas. (Focus handling might evolve).
+    *   **`Enter` (in column list):** Select a column to view statistics.
+    *   **View Switching:** Use `Ctrl+N` (Next Tab) and `Ctrl+P` (Previous Tab) or click on the tabs (Metadata, Schema, Data Preview).
+    *   **Scrolling:** Use `PageUp` / `PageDown` / `Home` / `End` or arrow keys/mouse wheel within scrollable areas (like Schema stats or Data Preview).
+    *   **`q` / `Ctrl+C`:** Quit `parqv`.
+    *   *(Help Screen `?` is planned)*
+
+---
+
+## 📄 License
+
+Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for the full license text.

parqv-0.2.0/README.md

@@ -0,0 +1,88 @@
+# parqv
+
+[![Python Version](https://img.shields.io/badge/Python-3.10+-blue.svg)](https://www.python.org/)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
+[![PyPI version](https://badge.fury.io/py/parqv.svg)](https://badge.fury.io/py/parqv) <!-- TODO: Link after first PyPI release -->
+[![Built with Textual](https://img.shields.io/badge/Built%20with-Textual-blueviolet.svg)](https://textual.textualize.io/)
+
+---
+
+**Supported File Formats:** ✅ **Parquet** | ✅ **JSON** / **JSON Lines (ndjson)** | *(More planned!)*
+
+---
+
+**`parqv` is a Python-based interactive TUI (Text User Interface) tool designed to explore, analyze, and understand various data file formats directly within your terminal.** Initially supporting Parquet and JSON, `parqv` aims to provide a unified, visual experience for quick data inspection without leaving your console.
+
+## 💻 Demo (Showing Parquet)
+
+![parqv.gif](assets/parqv.gif)
+*(Demo shows Parquet features; UI adapts for other formats)*
+
+## 🤔 Why `parqv`?
+1.  **Unified Interface:** Launch `parqv <your_data_file>` to access **metadata, schema, data preview, and column statistics** all within a single, navigable terminal window. No more juggling different commands for different file types.
+2.  **Interactive Exploration:**
+    *   **🖱️ Keyboard & Mouse Driven:** Navigate using familiar keys (arrows, `hjkl`, Tab) or even your mouse (thanks to `Textual`).
+    *   **📜 Scrollable Views:** Easily scroll through large schemas, data tables, or column lists.
+    *   **🌲 Clear Schema View:** Understand column names, data types, and nullability at a glance. (Complex nested structures visualization might vary by format).
+    *   **📊 Dynamic Stats:** Select a column and instantly see its detailed statistics (counts, nulls, min/max, mean, distinct values, etc.).
+3.  **Cross-Format Consistency:**
+    *   **🎨 Rich Display:** Leverages `rich` and `Textual` for colorful, readable tables and text across supported formats.
+    *   **📈 Quick Stats:** Get key statistical insights consistently, regardless of the underlying file type.
+    *   **🔌 Extensible:** Designed with a handler interface to easily add support for more file formats in the future (like CSV, Arrow IPC, etc.).
+
+## ✨ Features (TUI Mode)
+*   **Multi-Format Support:** Currently supports **Parquet** (`.parquet`) and **JSON/JSON Lines** (`.json`, `.ndjson`). Run `parqv <your_file.{parquet,json,ndjson}>`.
+*   **Metadata Panel:** Displays key file information (path, format, size, total rows, column count, etc.). *Fields may vary slightly depending on the file format.*
+*   **Schema Explorer:**
+    *   Interactive list view of columns.
+    *   Clearly shows column names, data types, and nullability.
+*   **Data Table Viewer:**
+    *   Scrollable table preview of the file's data.
+    *   Attempts to preserve data types for better representation.
+*   **Column Statistics Viewer:**
+    *   Select a column in the Schema tab to view detailed statistics.
+    *   Shows counts (total, valid, null), percentages, and type-specific stats (min/max, mean, stddev, distinct counts, length stats, boolean value counts where applicable).
+*   **Row Group Inspector (Parquet Specific):**
+    *   *This panel only appears when viewing Parquet files.*
+    *   Lists row groups with stats (row count, compressed/uncompressed size).
+    *   (Planned) Select a row group for more details.
+
+## 🚀 Getting Started
+
+**1. Prerequisites:**
+*   **Python:** Version 3.10 or higher.
+*   **pip:** The Python package installer.
+
+**2. Install `parqv`:**
+*   Open your terminal and run:
+    ```bash
+    pip install parqv
+    ```
+    *(This will also install dependencies like `textual`, `pyarrow`, `pandas`, and `duckdb`)*
+*   **Updating `parqv`:**
+    ```bash
+    pip install --upgrade parqv
+    ```
+
+**3. Run `parqv`:**
+*   Point `parqv` to your data file:
+    ```bash
+    #parquet
+    parqv /path/to/your/data.parquet
+    
+    # json
+    parqv /path/to/your/data.json
+*   The interactive TUI will launch. Use your keyboard (and mouse, if supported by your terminal) to navigate:
+    *   **Arrow Keys / `j`,`k` (in lists):** Move selection up/down.
+    *   **`Tab` / `Shift+Tab`:** Cycle focus between the main tab content and potentially other areas. (Focus handling might evolve).
+    *   **`Enter` (in column list):** Select a column to view statistics.
+    *   **View Switching:** Use `Ctrl+N` (Next Tab) and `Ctrl+P` (Previous Tab) or click on the tabs (Metadata, Schema, Data Preview).
+    *   **Scrolling:** Use `PageUp` / `PageDown` / `Home` / `End` or arrow keys/mouse wheel within scrollable areas (like Schema stats or Data Preview).
+    *   **`q` / `Ctrl+C`:** Quit `parqv`.
+    *   *(Help Screen `?` is planned)*
+
+---
+
+## 📄 License
+
+Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for the full license text.

{parqv-0.1.0 → parqv-0.2.0}/pyproject.toml

@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "parqv"
-version = "0.1.0"
-description = "An interactive Python TUI for visualizing, exploring, and analyzing Parquet files directly in your terminal."
+version = "0.2.0"
+description = "An interactive Python TUI for visualizing, exploring, and analyzing files directly in your terminal."
 readme = "README.md"
 requires-python = ">=3.10"
 license = "Apache-2.0"
@@ -15,7 +15,8 @@ dependencies = [
     "textual>=1.0.0",
     "pyarrow>=16.0.0",
     "pandas>=2.0.0",
-    "numpy>=1.20.0"
+    "numpy>=1.20.0",
+    "duckdb>=1.2.0"
 ]
 
 [project.scripts]

parqv-0.2.0/src/parqv/app.py

@@ -0,0 +1,168 @@
+import logging
+import sys
+from logging.handlers import RotatingFileHandler
+from pathlib import Path
+from typing import Optional, Type
+
+from textual.app import App, ComposeResult, Binding
+from textual.containers import Container
+from textual.widgets import Header, Footer, Static, Label, TabbedContent, TabPane
+
+from .handlers import (
+    DataHandler,
+    DataHandlerError,
+    ParquetHandler,
+    JsonHandler,
+)
+from .views.data_view import DataView
+from .views.metadata_view import MetadataView
+from .views.schema_view import SchemaView
+
+LOG_FILENAME = "parqv.log"
+file_handler = RotatingFileHandler(
+    LOG_FILENAME, maxBytes=1024 * 1024 * 5, backupCount=3, encoding="utf-8"
+)
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s [%(levelname)-5.5s] %(name)s (%(filename)s:%(lineno)d) - %(message)s",
+    handlers=[file_handler, logging.StreamHandler(sys.stdout)],
+)
+log = logging.getLogger(__name__)
+
+AnyHandler = DataHandler
+AnyHandlerError = DataHandlerError
+
+
+class ParqV(App[None]):
+    """A Textual app to visualize Parquet or JSON files."""
+
+    CSS_PATH = "parqv.css"
+    BINDINGS = [
+        Binding("q", "quit", "Quit", priority=True),
+    ]
+
+    # App State
+    file_path: Optional[Path] = None
+    handler: Optional[AnyHandler] = None  # Use ABC type hint
+    handler_type: Optional[str] = None  # Keep for display ('parquet', 'json')
+    error_message: Optional[str] = None
+
+    def __init__(self, file_path_str: Optional[str] = None, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        if not file_path_str:
+            self.error_message = "No file path provided."
+            log.error(self.error_message)
+            return
+
+        self.file_path = Path(file_path_str)
+        log.debug(f"Input file path: {self.file_path}")
+
+        if not self.file_path.is_file():
+            self.error_message = f"File not found or is not a regular file: {self.file_path}"
+            log.error(self.error_message)
+            return
+
+        # Handler Detection
+        handler_class: Optional[Type[AnyHandler]] = None
+        handler_error_class: Type[AnyHandlerError] = DataHandlerError
+        detected_type = "unknown"
+        file_suffix = self.file_path.suffix.lower()
+
+        if file_suffix == ".parquet":
+            log.info("Detected '.parquet' extension, using ParquetHandler.")
+            handler_class = ParquetHandler
+            detected_type = "parquet"
+        elif file_suffix in [".json", ".ndjson"]:
+            log.info(f"Detected '{file_suffix}' extension, using JsonHandler.")
+            handler_class = JsonHandler
+            detected_type = "json"
+        else:
+            self.error_message = f"Unsupported file extension: '{file_suffix}'. Only .parquet, .json, .ndjson are supported."
+            log.error(self.error_message)
+            return
+
+        # Instantiate Handler
+        if handler_class:
+            log.info(f"Attempting to initialize {detected_type.capitalize()} handler for: {self.file_path}")
+            try:
+                self.handler = handler_class(self.file_path)
+                self.handler_type = detected_type
+                log.info(f"{detected_type.capitalize()} handler initialized successfully.")
+            except DataHandlerError as e:
+                self.error_message = f"Failed to initialize {detected_type} handler: {e}"
+                log.error(self.error_message, exc_info=True)
+            except Exception as e:
+                self.error_message = f"An unexpected error occurred during {detected_type} handler initialization: {e}"
+                log.exception(f"Unexpected error during {detected_type} handler initialization:")
+
+    def compose(self) -> ComposeResult:
+        yield Header()
+        if self.error_message:
+            log.error(f"Displaying error message: {self.error_message}")
+            yield Container(
+                Label("Error Loading File:", classes="error-title"),
+                Static(self.error_message, classes="error-content"),
+                id="error-container"
+            )
+        elif self.handler:
+            log.debug(f"Composing main layout with TabbedContent for {self.handler_type} handler.")
+            with TabbedContent(id="main-tabs"):
+                yield TabPane("Metadata", MetadataView(id="metadata-view"), id="tab-metadata")
+                yield TabPane("Schema", SchemaView(id="schema-view"), id="tab-schema")
+                yield TabPane("Data Preview", DataView(id="data-view"), id="tab-data")
+        else:
+            log.error("Compose called but no handler and no error message. Initialization likely failed silently.")
+            yield Container(Label("Initialization failed."), id="init-failed")
+        yield Footer()
+
+    def on_mount(self) -> None:
+        log.debug("App mounted.")
+        try:
+            header = self.query_one(Header)
+            display_name = "N/A"
+            format_name = "Unknown"
+            if self.handler and self.file_path:
+                display_name = self.file_path.name
+                format_name = self.handler_type.capitalize() if self.handler_type else "Unknown"
+                header.title = f"parqv - {display_name}"
+                header.sub_title = f"Format: {format_name}"
+            elif self.error_message:
+                header.title = "parqv - Error"
+            else:
+                header.title = "parqv"
+        except Exception as e:
+            log.error(f"Failed to set header title: {e}")
+
+    def action_quit(self) -> None:
+        log.info("Quit action triggered.")
+        if self.handler:
+            try:
+                self.handler.close()
+            except Exception as e:
+                log.error(f"Error during handler cleanup: {e}")
+        self.exit()
+
+
+# CLI Entry Point
+def run_app():
+    log.info("--- parqv (ABC Handler) started ---")
+    if len(sys.argv) < 2:
+        print("Usage: parqv <path_to_parquet_or_json_file>")
+        log.error("No file path provided.")
+        sys.exit(1)
+
+    file_path_str = sys.argv[1]
+    log.debug(f"File path from argument: {file_path_str}")
+
+    _path = Path(file_path_str)
+    if not _path.suffix.lower() in ['.parquet', '.json', '.ndjson']:
+        print(f"Error: Unsupported file type '{_path.suffix}'. Please provide a .parquet, .json, or .ndjson file.")
+        log.error(f"Unsupported file type provided via CLI: {_path.suffix}")
+        sys.exit(1)
+
+    app = ParqV(file_path_str=file_path_str)
+    app.run()
+
+
+if __name__ == "__main__":
+    run_app()

parqv-0.2.0/src/parqv/handlers/__init__.py

@@ -0,0 +1,13 @@
+# src/parqv/handlers/__init__.py
+from .base_handler import DataHandler, DataHandlerError
+from .parquet import ParquetHandler, ParquetHandlerError
+from .json import JsonHandler, JsonHandlerError
+
+__all__ = [
+    "DataHandler",
+    "DataHandlerError",
+    "ParquetHandler",
+    "ParquetHandlerError",
+    "JsonHandler",
+    "JsonHandlerError",
+]

parqv-0.2.0/src/parqv/handlers/base_handler.py

@@ -0,0 +1,114 @@
+import logging
+from abc import ABC, abstractmethod
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+
+import pandas as pd
+
+log = logging.getLogger(__name__)
+
+
+class DataHandlerError(Exception):
+    """Base exception for all data handler errors."""
+    pass
+
+
+class DataHandler(ABC):
+    """
+    Abstract Base Class for data handlers.
+    Defines the common interface required by the ParqV application
+    to interact with different data file formats.
+    """
+
+    def __init__(self, file_path: Path):
+        """
+        Initializes the handler with the file path.
+        Subclasses should open the file or set up necessary resources here.
+
+        Args:
+            file_path: Path to the data file.
+
+        Raises:
+            DataHandlerError: If initialization fails (e.g., file not found, format error).
+        """
+        self.file_path = file_path
+
+    @abstractmethod
+    def close(self) -> None:
+        """
+        Closes any open resources (files, connections, etc.).
+        Must be implemented by subclasses.
+        """
+        pass
+
+    @abstractmethod
+    def get_metadata_summary(self) -> Dict[str, Any]:
+        """
+        Returns a dictionary containing summary metadata about the data source.
+        Keys should be human-readable strings. Values can be of various types.
+        Should include an 'error' key if metadata retrieval fails.
+
+        Returns:
+            A dictionary with metadata summary or an error dictionary.
+        """
+        pass
+
+    @abstractmethod
+    def get_schema_data(self) -> Optional[List[Dict[str, str]]]:
+        """
+        Returns the schema as a list of dictionaries.
+        Each dictionary should represent a column and ideally contain keys:
+        'name' (str): Column name.
+        'type' (str): Formatted data type string.
+        'nullable' (Any): Indicator of nullability (e.g., bool, str "YES"/"NO").
+
+        Returns:
+            A list of schema dictionaries, an empty list if no columns,
+            or None if schema retrieval failed.
+        """
+        pass
+
+    @abstractmethod
+    def get_data_preview(self, num_rows: int = 50) -> Optional[pd.DataFrame]:
+        """
+        Fetches a preview of the data.
+
+        Args:
+            num_rows: The maximum number of rows to fetch.
+
+        Returns:
+            A pandas DataFrame with preview data, an empty DataFrame if no data,
+            a DataFrame with an 'error' column on failure, or None on critical failure.
+        """
+        pass
+
+    @abstractmethod
+    def get_column_stats(self, column_name: str) -> Dict[str, Any]:
+        """
+        Calculates and returns statistics for a specific column.
+        The returned dictionary should ideally contain keys like:
+        'column' (str): Column name.
+        'type' (str): Formatted data type string.
+        'nullable' (Any): Nullability indicator.
+        'calculated' (Dict[str, Any]): Dictionary of computed statistics.
+        'error' (Optional[str]): Error message if calculation failed.
+        'message' (Optional[str]): Informational message.
+
+        Args:
+            column_name: The name of the column.
+
+        Returns:
+            A dictionary containing column statistics or error information.
+        """
+        pass
+
+    def _format_size(self, num_bytes: int) -> str:
+        """Formats bytes into a human-readable string."""
+        if num_bytes < 1024:
+            return f"{num_bytes} bytes"
+        elif num_bytes < 1024 ** 2:
+            return f"{num_bytes / 1024:.1f} KB"
+        elif num_bytes < 1024 ** 3:
+            return f"{num_bytes / 1024 ** 2:.1f} MB"
+        else:
+            return f"{num_bytes / 1024 ** 3:.1f} GB"