tab-cli 0.1.5__tar.gz → 0.1.7__tar.gz

{tab_cli-0.1.5 → tab_cli-0.1.7}/AGENTS.md

@@ -143,6 +143,7 @@ Use it as the default operating guide when changing code in this repo.
 ## Documentation
 
 - Update `docs/` and CLI help text when changing user-facing behavior.
+- Update `CHANGELOG.md` with a clear description of user-facing changes and bug fixes.
 - Keep examples aligned with the actual command names and flags.
 - If you change build, test, or auth flows, reflect that in this file too.
 

{tab_cli-0.1.5 → tab_cli-0.1.7}/CHANGELOG.md

@@ -1,3 +1,8 @@
+  - 0.1.7:
+    - Optional dependency groups are now named `tab-cli[s3|gs|az]`, in accordance with the protocol name.
+  - 0.1.6:
+    - Fixed bug in pyarrow loading of Parquet files.
+    - Added global config file support: settings can be persisted in `~/.config/tab/config.json`. Config file values serve as defaults that CLI flags override.
   - 0.1.5:
     - Added stdin support: use `-` as the file path to read from stdin (e.g. `cat data.csv | tab view -i csv -`). Requires `-i`/`--input-format` since format cannot be inferred. Works with `view`, `schema`, `summary`, `convert`, and `cat`.
     - Added row-wise JMESPath queries via `--jmespath` / `--jp` on `view`, `convert`, and `cat`. Object results become columns; non-object results go into a `value` column. `--sql` and `--jp` are mutually exclusive.

tab_cli-0.1.7/PKG-INFO

@@ -0,0 +1,121 @@
+Metadata-Version: 2.4
+Name: tab-cli
+Version: 0.1.7
+Summary: A CLI tool for tabular data
+Author-email: Tongfei Chen <tongfei@pm.me>
+License-File: LICENSE
+Requires-Python: >=3.10
+Requires-Dist: blobfile>=3.0
+Requires-Dist: fsspec>=2025.1.0
+Requires-Dist: jmespath>=1.0
+Requires-Dist: loguru>=0.7.3
+Requires-Dist: polars-fastavro>=0.5.1
+Requires-Dist: polars>=1.0
+Requires-Dist: pyarrow>=15.0
+Requires-Dist: rich>=13.0
+Requires-Dist: typer>=0.21.1
+Provides-Extra: az
+Requires-Dist: adlfs>=2025.1.0; extra == 'az'
+Requires-Dist: azure-identity>=1.10.0; extra == 'az'
+Provides-Extra: gs
+Requires-Dist: gcsfs>=2025.1.0; extra == 'gs'
+Requires-Dist: google-auth>=2.27.0; extra == 'gs'
+Provides-Extra: s3
+Requires-Dist: boto3>=1.28.0; extra == 's3'
+Requires-Dist: s3fs>=2025.1.0; extra == 's3'
+Description-Content-Type: text/markdown
+
+# tab
+
+[![pypi](https://img.shields.io/pypi/v/tab-cli)](https://pypi.org/project/tab-cli/)
+
+A CLI tool for viewing, querying, and converting tabular data files.
+Reads CSV, TSV, JSON Lines, Parquet, and Avro -- locally or from S3, GCS, and Azure Blob Storage.
+
+```sh
+pip install tab-cli
+```
+
+**Documentation**: [tongfei.me/tab](https://tongfei.me/tab)
+
+---
+
+## Quick look
+
+### View any tabular file
+
+```bash
+tab view data.csv
+```
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/ctongfei/tab/refs/heads/gh-pages/assets/test.svg" alt="tab view" width="680">
+</p>
+
+### Query with SQL
+
+The table is always available as `t`:
+
+```bash
+tab view --sql 'SELECT * FROM t WHERE Metric_A_Value > 80' data.csv
+```
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/ctongfei/tab/refs/heads/gh-pages/assets/test-where.svg" alt="tab view --sql" width="680">
+</p>
+
+### Reshape rows with JMESPath
+
+```bash
+tab view --jp '{id: participant.id, city: profile.address.city}' data.parquet 
+```
+
+### Convert between formats
+
+```bash
+tab convert data.csv data.parquet
+tab convert data.parquet data.jsonl -o jsonl
+tab convert data.csv output_dir/ -o parquet -n 4   # partitioned
+```
+
+### Concatenate files
+
+```bash
+tab cat part1.csv part2.csv part3.csv -o jsonl > combined.jsonl
+```
+
+### Inspect schema and summary
+
+```bash
+tab schema data.parquet
+tab summary data.parquet
+```
+
+### Read from stdin
+
+```bash
+curl -s https://example.com/data.csv | tab view -i csv -
+```
+
+### Read from cloud storage
+
+```bash
+tab view s3://bucket/path/data.parquet
+tab view gs://bucket/path/data.csv
+tab view az://container/path/data.jsonl
+```
+
+Install cloud extras as needed:
+
+```sh
+pip install 'tab-cli[s3]'    # AWS S3
+pip install 'tab-cli[gs]'    # Google Cloud Storage
+pip install 'tab-cli[az]'    # Azure Blob Storage
+```
+
+## Supported formats
+ - csv
+ - tsv
+ - jsonl
+ - parquet
+ - avro

tab_cli-0.1.7/README.md

@@ -0,0 +1,94 @@
+# tab
+
+[![pypi](https://img.shields.io/pypi/v/tab-cli)](https://pypi.org/project/tab-cli/)
+
+A CLI tool for viewing, querying, and converting tabular data files.
+Reads CSV, TSV, JSON Lines, Parquet, and Avro -- locally or from S3, GCS, and Azure Blob Storage.
+
+```sh
+pip install tab-cli
+```
+
+**Documentation**: [tongfei.me/tab](https://tongfei.me/tab)
+
+---
+
+## Quick look
+
+### View any tabular file
+
+```bash
+tab view data.csv
+```
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/ctongfei/tab/refs/heads/gh-pages/assets/test.svg" alt="tab view" width="680">
+</p>
+
+### Query with SQL
+
+The table is always available as `t`:
+
+```bash
+tab view --sql 'SELECT * FROM t WHERE Metric_A_Value > 80' data.csv
+```
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/ctongfei/tab/refs/heads/gh-pages/assets/test-where.svg" alt="tab view --sql" width="680">
+</p>
+
+### Reshape rows with JMESPath
+
+```bash
+tab view --jp '{id: participant.id, city: profile.address.city}' data.parquet 
+```
+
+### Convert between formats
+
+```bash
+tab convert data.csv data.parquet
+tab convert data.parquet data.jsonl -o jsonl
+tab convert data.csv output_dir/ -o parquet -n 4   # partitioned
+```
+
+### Concatenate files
+
+```bash
+tab cat part1.csv part2.csv part3.csv -o jsonl > combined.jsonl
+```
+
+### Inspect schema and summary
+
+```bash
+tab schema data.parquet
+tab summary data.parquet
+```
+
+### Read from stdin
+
+```bash
+curl -s https://example.com/data.csv | tab view -i csv -
+```
+
+### Read from cloud storage
+
+```bash
+tab view s3://bucket/path/data.parquet
+tab view gs://bucket/path/data.csv
+tab view az://container/path/data.jsonl
+```
+
+Install cloud extras as needed:
+
+```sh
+pip install 'tab-cli[s3]'    # AWS S3
+pip install 'tab-cli[gs]'    # Google Cloud Storage
+pip install 'tab-cli[az]'    # Azure Blob Storage
+```
+
+## Supported formats
+ - csv
+ - tsv
+ - jsonl
+ - parquet
+ - avro

tab_cli-0.1.7/docs/configuration.md

@@ -0,0 +1,34 @@
+# Configuration
+
+`tab` supports a global configuration file at `~/.config/tab/config.json`. Settings in this file serve as defaults and are overridden by CLI flags.
+
+## Setup
+
+Create the config file:
+
+```bash
+mkdir -p ~/.config/tab
+cat > ~/.config/tab/config.json << 'EOF'
+{
+  "az_url_authority_is_account": false,
+  "sampling_size_for_schema_inference": 32
+}
+EOF
+```
+
+## Available settings
+
+| Key | Type | Default | Description |
+|-----|------|---------|-------------|
+| `az_url_authority_is_account` | `bool` | `false` | Interpret `az://` URL authority as storage account name instead of container name. |
+| `sampling_size_for_schema_inference` | `int` | `32` | Number of rows sampled for schema inference (e.g. when using `--jp`). |
+
+## Precedence
+
+Settings are applied in this order (last wins):
+
+1. Built-in defaults
+2. Config file (`~/.config/tab/config.json`)
+3. CLI flags (e.g. `--az-url-authority-is-account`)
+
+If the config file does not exist, built-in defaults are used. Unknown keys in the file are ignored with a warning.

{tab_cli-0.1.5 → tab_cli-0.1.7}/mkdocs.yml

@@ -31,6 +31,7 @@ repo_url: 'https://github.com/ctongfei/tab'
 
 nav:
   - 'Introduction': 'index.md'
+  - 'Configuration': 'configuration.md'
   - 'Cloud support': 'cloud.md'
   - 'CLI reference': 'cli-ref.md'
 

{tab_cli-0.1.5 → tab_cli-0.1.7}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "tab-cli"
-version = "0.1.5"
+version = "0.1.7"
 description = "A CLI tool for tabular data"
 authors = [{name = "Tongfei Chen", email = "tongfei@pm.me"}]
 readme = "README.md"
@@ -20,8 +20,8 @@ dependencies = [
 
 [project.optional-dependencies]
 s3 = ["s3fs>=2025.1.0", "boto3>=1.28.0"]
-gcs = ["gcsfs>=2025.1.0", "google-auth>=2.27.0"]
-azure = ["adlfs>=2025.1.0", "azure-identity>=1.10.0"]
+gs = ["gcsfs>=2025.1.0", "google-auth>=2.27.0"]
+az = ["adlfs>=2025.1.0", "azure-identity>=1.10.0"]
 
 [dependency-groups]
 dev = [

{tab_cli-0.1.5 → tab_cli-0.1.7}/src/tab_cli/cli.py

@@ -12,27 +12,67 @@ from rich.console import Console
 from rich.logging import RichHandler
 
 from tab_cli import config
-from tab_cli.config import Config
-from tab_cli.handlers import TableWriter, infer_reader, infer_writer, is_stdin, read_stdin
+from tab_cli.config import Config, load_config_file
+from tab_cli.handlers import (
+    TableWriter,
+    infer_reader,
+    infer_writer,
+    is_stdin,
+    read_stdin,
+)
 from tab_cli.handlers.base import TableSchema, TableSummary
 
 # Reusable type aliases for common CLI options
-PathArg: TypeAlias = Annotated[str, typer.Argument(help="Path to the data file or directory")]
-PathsArg: TypeAlias = Annotated[list[str], typer.Argument(help="Paths to the data files or directories")]
-SrcArg: TypeAlias = Annotated[str, typer.Argument(help="Path to the source file or directory")]
-DstArg: TypeAlias = Annotated[str, typer.Argument(help="Path to the destination file or directory")]
-InputOpt: TypeAlias = Annotated[Optional[str], typer.Option("-i", "--input-format", help="Input format, auto-detected from extension if omitted")]
-OutputOpt: TypeAlias = Annotated[Optional[str], typer.Option("-o", "--output-format", help="Output format")]
-SqlOpt: TypeAlias = Annotated[Optional[str], typer.Option("--sql", help="SQL query to apply (table is available as 't')")]
+PathArg: TypeAlias = Annotated[
+    str, typer.Argument(help="Path to the data file or directory")
+]
+PathsArg: TypeAlias = Annotated[
+    list[str], typer.Argument(help="Paths to the data files or directories")
+]
+SrcArg: TypeAlias = Annotated[
+    str, typer.Argument(help="Path to the source file or directory")
+]
+DstArg: TypeAlias = Annotated[
+    str, typer.Argument(help="Path to the destination file or directory")
+]
+InputOpt: TypeAlias = Annotated[
+    Optional[str],
+    typer.Option(
+        "-i",
+        "--input-format",
+        help="Input format, auto-detected from extension if omitted",
+    ),
+]
+OutputOpt: TypeAlias = Annotated[
+    Optional[str], typer.Option("-o", "--output-format", help="Output format")
+]
+SqlOpt: TypeAlias = Annotated[
+    Optional[str],
+    typer.Option("--sql", help="SQL query to apply (table is available as 't')"),
+]
 JmespathOpt: TypeAlias = Annotated[
     Optional[str],
-    typer.Option("--jmespath", "--jp", help="JMESPath expression to apply to each row as JSON"),
+    typer.Option(
+        "--jmespath", "--jp", help="JMESPath expression to apply to each row as JSON"
+    ),
+]
+LimitOpt: TypeAlias = Annotated[
+    Optional[int], typer.Option("--limit", help="Maximum number of rows to display")
+]
+SkipOpt: TypeAlias = Annotated[
+    int, typer.Option("--skip", help="Number of rows to skip")
+]
+MaxCellLenOpt: TypeAlias = Annotated[
+    Optional[int],
+    typer.Option("--max-cell-len", help="Truncate cell contents longer than this"),
+]
+TableSvgOpt: TypeAlias = Annotated[
+    bool, typer.Option("--table-svg", help="Output table as SVG")
+]
+NumPartitionsOpt: TypeAlias = Annotated[
+    Optional[int],
+    typer.Option("-n", "--num-partitions", help="Number of output partitions"),
 ]
-LimitOpt: TypeAlias = Annotated[Optional[int], typer.Option("--limit", help="Maximum number of rows to display")]
-SkipOpt: TypeAlias = Annotated[int, typer.Option("--skip", help="Number of rows to skip")]
-MaxCellLenOpt: TypeAlias = Annotated[Optional[int], typer.Option("--max-cell-len", help="Truncate cell contents longer than this")]
-TableSvgOpt: TypeAlias = Annotated[bool, typer.Option("--table-svg", help="Output table as SVG")]
-NumPartitionsOpt: TypeAlias = Annotated[Optional[int], typer.Option("-n", "--num-partitions", help="Number of output partitions")]
 
 app = typer.Typer(
     help="A CLI tool for viewing and manipulating tabular data.",
@@ -51,11 +91,12 @@ def main_callback(
     ] = False,
     log_level: Annotated[
         str,
-        typer.Option("--log-level", help="Log level from {DEBUG, INFO, WARNING, ERROR, CRITICAL}"),
+        typer.Option(
+            "--log-level", help="Log level from {DEBUG, INFO, WARNING, ERROR, CRITICAL}"
+        ),
     ] = "INFO",
 ) -> None:
     """Global options for tab_cli CLI."""
-    config.config.az_url_authority_is_account = az_url_authority_is_account
     logger.remove()
     logger.add(
         RichHandler(
@@ -66,6 +107,10 @@ def main_callback(
         format="{message}",
         level=log_level.upper(),
     )
+    load_config_file()
+    # CLI flags override config file values
+    if az_url_authority_is_account:
+        config.config.az_url_authority_is_account = az_url_authority_is_account
 
 
 def _apply_sql(lf: pl.LazyFrame, sql: str | None) -> pl.LazyFrame:
@@ -94,21 +139,29 @@ def _transform_jmespath_batch(
             if mode is None:
                 mode = "object"
             elif mode != "object":
-                raise ValueError("JMESPath query must return a consistent shape across rows")
+                raise ValueError(
+                    "JMESPath query must return a consistent shape across rows"
+                )
 
             if expected_columns is not None:
                 extra_columns = set(result) - set(expected_columns)
                 if extra_columns:
                     extras = ", ".join(sorted(extra_columns))
-                    raise ValueError(f"JMESPath query produced unexpected columns: {extras}")
-                normalized_row = {column: result.get(column) for column in expected_columns}
+                    raise ValueError(
+                        f"JMESPath query produced unexpected columns: {extras}"
+                    )
+                normalized_row = {
+                    column: result.get(column) for column in expected_columns
+                }
             else:
                 normalized_row = result
         else:
             if mode is None:
                 mode = "value"
             elif mode != "value":
-                raise ValueError("JMESPath query must return a consistent shape across rows")
+                raise ValueError(
+                    "JMESPath query must return a consistent shape across rows"
+                )
             normalized_row = {"value": result}
 
         rows.append(normalized_row)
@@ -143,10 +196,14 @@ def _apply_jmespath(lf: pl.LazyFrame, expression: str) -> pl.LazyFrame:
     )
 
 
-def _apply_query(lf: pl.LazyFrame, sql: str | None, jmespath_expr: str | None) -> pl.LazyFrame:
+def _apply_query(
+    lf: pl.LazyFrame, sql: str | None, jmespath_expr: str | None
+) -> pl.LazyFrame:
     """Apply exactly zero or one supported query transform to a LazyFrame."""
     if sql is not None and jmespath_expr is not None:
-        raise ValueError("At most one query may be provided: use either --sql or --jmespath/--jp")
+        raise ValueError(
+            "At most one query may be provided: use either --sql or --jmespath/--jp"
+        )
     if sql is not None:
         return _apply_sql(lf, sql)
     if jmespath_expr is not None:
@@ -196,11 +253,18 @@ def view(
         reader = infer_reader(path, format=input)
         lf = reader.read(path)
     lf = _apply_query(lf, sql=sql, jmespath_expr=jmespath_expr)
-    lf, truncated = _apply_limit(lf, limit=limit, skip=skip, default_limit=20 if limit is None else None)
-    writer = infer_writer("table-svg" if table_svg else None, truncated=truncated, max_cell_len=max_cell_len)
+    lf, truncated = _apply_limit(
+        lf, limit=limit, skip=skip, default_limit=20 if limit is None else None
+    )
+    writer = infer_writer(
+        "table-svg" if table_svg else None,
+        truncated=truncated,
+        max_cell_len=max_cell_len,
+    )
     for chunk in writer.write(lf):
         sys.stdout.buffer.write(chunk)
 
+
 @app.command()
 def schema(
     path: PathArg,
@@ -258,7 +322,9 @@ def convert(
         elif input is not None:
             writer = infer_writer(format=input)
         else:
-            raise ValueError("Output format (-o/--output-format) is required when reading from stdin (-)")
+            raise ValueError(
+                "Output format (-o/--output-format) is required when reading from stdin (-)"
+            )
         assert isinstance(writer, TableWriter)
         writer.write_to_path(lf, dst, partitions=num_partitions)
     else:
@@ -305,7 +371,9 @@ def cat(
         writer = infer_writer(format=input)
         assert isinstance(writer, TableWriter)
     else:
-        raise ValueError("Output format (-o/--output-format) or input format (-i/--input-format) is required when reading from stdin (-)")
+        raise ValueError(
+            "Output format (-o/--output-format) or input format (-i/--input-format) is required when reading from stdin (-)"
+        )
     for chunk in writer.write(lf):
         sys.stdout.buffer.write(chunk)
 

tab_cli-0.1.7/src/tab_cli/config.py

@@ -0,0 +1,47 @@
+"""Global configuration for tab_cli-cli."""
+
+import json
+from dataclasses import dataclass, fields
+from pathlib import Path
+
+from loguru import logger
+
+CONFIG_DIR = Path.home() / ".config" / "tab"
+CONFIG_FILE = CONFIG_DIR / "config.json"
+
+
+@dataclass
+class Config:
+    """Global configuration settings."""
+
+    az_url_authority_is_account: bool = False
+    sampling_size_for_schema_inference: int = 32
+
+
+# Global config instance
+config: Config = Config()
+
+
+def load_config_file(path: Path = CONFIG_FILE) -> None:
+    """Load settings from a JSON config file into the global config.
+
+    Unknown keys are logged and ignored. Type mismatches raise ValueError.
+    If the file does not exist, this is a no-op.
+    """
+    if not path.is_file():
+        return
+
+    text = path.read_text(encoding="utf-8")
+    data = json.loads(text)
+    if not isinstance(data, dict):
+        raise ValueError(
+            f"Config file must contain a JSON object, got {type(data).__name__}"
+        )
+
+    known = {f.name: f.type for f in fields(Config)}
+    for key, value in data.items():
+        if key not in known:
+            logger.warning("Unknown config key '{}' in {}", key, path)
+            continue
+        setattr(config, key, value)
+    logger.debug("Loaded config from {}", path)

{tab_cli-0.1.5 → tab_cli-0.1.7}/src/tab_cli/formats/parquet.py

@@ -30,7 +30,7 @@ def _scan_parquet_with_pyarrow_fallback(
             "Polars native Parquet reader failed ({}), retrying with PyArrow backend",
             e,
         )
-        return pl.scan_parquet(url, storage_options=storage_options, use_pyarrow=True)
+        return pl.read_parquet(url, storage_options=storage_options, use_pyarrow=True).lazy()
 
 
 class ParquetFormat(FormatHandler):