cloudcat 0.3.8__tar.gz → 0.5.1__tar.gz

{cloudcat-0.3.8 → cloudcat-0.5.1}/PKG-INFO

@@ -1,10 +1,9 @@
 Metadata-Version: 2.4
 Name: cloudcat
-Version: 0.3.8
+Version: 0.5.1
 Summary: Preview and analyze data files in Google Cloud Storage, AWS S3, and Azure Data Lake Storage Gen2 from your terminal
-Home-page: https://github.com/jonathansudhakar1/cloudcat
 Author: Jonathan Sudhakar
-Author-email: jonathan@example.com
+License: MIT
 Project-URL: Homepage, https://github.com/jonathansudhakar1/cloudcat
 Project-URL: Documentation, https://cloudcatcli.com
 Project-URL: Bug Reports, https://github.com/jonathansudhakar1/cloudcat/issues
@@ -22,14 +21,12 @@ Classifier: Environment :: Console
 Classifier: License :: OSI Approved :: MIT License
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.7
-Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
-Requires-Python: >=3.7
+Requires-Python: >=3.9
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: click>=8.0.0
@@ -39,7 +36,9 @@ Requires-Dist: colorama>=0.4.4
 Requires-Dist: google-cloud-storage>=2.0.0
 Requires-Dist: boto3>=1.18.0
 Requires-Dist: azure-storage-file-datalake>=12.0.0
+Requires-Dist: azure-storage-blob>=12.0.0
 Requires-Dist: azure-identity>=1.0.0
+Requires-Dist: tomli>=1.1.0; python_version < "3.11"
 Provides-Extra: parquet
 Requires-Dist: pyarrow>=5.0.0; extra == "parquet"
 Provides-Extra: avro
@@ -62,19 +61,16 @@ Requires-Dist: fastavro>=1.4.0; extra == "all"
 Requires-Dist: zstandard>=0.15.0; extra == "all"
 Requires-Dist: lz4>=3.0.0; extra == "all"
 Requires-Dist: python-snappy>=0.6.0; extra == "all"
-Dynamic: author
-Dynamic: author-email
-Dynamic: classifier
-Dynamic: description
-Dynamic: description-content-type
-Dynamic: home-page
-Dynamic: keywords
+Provides-Extra: test
+Requires-Dist: pytest>=6.0.0; extra == "test"
+Requires-Dist: pytest-mock>=3.6.0; extra == "test"
+Provides-Extra: dev
+Requires-Dist: pytest>=6.0.0; extra == "dev"
+Requires-Dist: pytest-mock>=3.6.0; extra == "dev"
+Requires-Dist: moto>=4.0.0; extra == "dev"
+Requires-Dist: build; extra == "dev"
+Requires-Dist: twine; extra == "dev"
 Dynamic: license-file
-Dynamic: project-url
-Dynamic: provides-extra
-Dynamic: requires-dist
-Dynamic: requires-python
-Dynamic: summary
 
 <p align="center">
   <img src="https://raw.githubusercontent.com/jonathansudhakar1/cloudcat/main/assets/logo.png" alt="CloudCat Logo" width="200">
@@ -174,14 +170,17 @@ pip install --upgrade 'cloudcat[all]'
 ### Requirements
 
 - **Homebrew**: macOS (Apple Silicon only). Intel Mac users should use pip.
-- **pip**: Python 3.7+ (all platforms)
+- **pip**: Python 3.9+ (all platforms)
 - Cloud provider credentials configured (see [Authentication](#authentication))
 
 ## Quick Start
 
 ```bash
-# Preview a CSV file from GCS
-cloudcat -p gcs://my-bucket/data.csv
+# Preview a CSV file from GCS (PATH is positional; -p/--path also works)
+cloudcat gcs://my-bucket/data.csv
+
+# Preview a local file — no cloud credentials needed
+cloudcat ./data.parquet
 
 # Preview a Parquet file from S3
 cloudcat -p s3://my-bucket/analytics/events.parquet
@@ -204,8 +203,8 @@ cloudcat -p s3://my-bucket/spark-output/ -i parquet
 # Read compressed files (auto-detected)
 cloudcat -p gcs://my-bucket/data.csv.gz
 
-# Filter rows with WHERE clause
-cloudcat -p s3://bucket/users.parquet --where "status=active"
+# Filter rows with WHERE (streams; stops at -n matches; AND/OR supported)
+cloudcat s3://bucket/users.parquet --where "status=active AND age>30"
 
 # Skip first 100 rows (pagination)
 cloudcat -p gcs://bucket/data.csv --offset 100 -n 10
@@ -220,6 +219,7 @@ cloudcat -p gcs://bucket/data.csv --offset 100 -n 10
 | Google Cloud Storage | `gcs://` or `gs://` | ✅ Supported |
 | Amazon S3 | `s3://` | ✅ Supported |
 | Azure Data Lake Gen2 | `abfss://` | ✅ Supported |
+| Local files | `file://` or a plain path | ✅ Supported (no credentials needed) |
 
 ### File Format Support
 
@@ -242,8 +242,8 @@ CloudCat uses intelligent streaming to minimize data transfer and egress costs:
 |--------|-------------|---------|-------------------|----------------|
 | Parquet | None/Internal | ✅ | ✅ Range requests | ✅ |
 | Parquet | External (.gz) | ❌ | ❌ | ❌ |
-| ORC | None/Internal | ❌ | ❌ | ❌ |
-| ORC | External (.gz) | ❌ | ❌ | ❌ |
+| ORC | None/Internal | ✅ Stripe-by-stripe | ✅ | ✅ |
+| ORC | External (.gz) | ❌ | ✅ | ✅ |
 | CSV | None | ✅ | ❌ | ✅ |
 | CSV | gzip/zstd/lz4/bz2 | ✅ | ❌ | ✅ |
 | CSV | snappy | ❌ | ❌ | ❌ |
@@ -272,7 +272,7 @@ CloudCat automatically detects and decompresses files based on extension (e.g.,
 
 | Format | Flag | Description |
 |--------|------|-------------|
-| Table | `-o table` | Beautiful ASCII table with colored headers (default) |
+| Table | `-o table` | Rounded table with bold headers and type-aware coloring (default) |
 | JSON | `-o json` | Standard JSON Lines output |
 | Pretty JSON | `-o jsonp` | Syntax-highlighted, indented JSON |
 | CSV | `-o csv` | Comma-separated values |
@@ -329,8 +329,8 @@ cloudcat -p gcs://bucket/events.jsonl
 ### Filtering and Pagination
 
 ```bash
-# Filter rows with WHERE clause
-cloudcat -p s3://bucket/users.parquet --where "status=active"
+# Filter rows with WHERE (streams; stops at -n matches; AND/OR supported)
+cloudcat s3://bucket/users.parquet --where "status=active AND age>30"
 cloudcat -p gcs://bucket/events.json --where "age>30"
 cloudcat -p s3://bucket/logs.csv --where "level=ERROR"
 
@@ -479,13 +479,17 @@ cloudcat -p s3://api-dumps/response.json -o csv > data.csv
 Usage: cloudcat [OPTIONS]
 
 Options:
-  -p, --path TEXT              Cloud storage path (required)
+  PATH                         Positional path: gs://, gcs://, s3://, abfss://,
+                               file:// URL, or a plain local path
+                               (-p/--path remains as a compatible alias)
                                Format: gcs://bucket/path, s3://bucket/path,
                                or abfss://container@account.dfs.core.windows.net/path
 
   -o, --output-format TEXT     Output format: table, json, jsonp, csv
                                [default: table]
 
+  -O, --output-file PATH       Write rendered data to a file instead of stdout
+
   -i, --input-format TEXT      Input format: csv, json, parquet, avro, orc, text
                                [default: auto-detect from extension]
 
@@ -498,9 +502,12 @@ Options:
   --offset INTEGER             Skip first N rows
                                [default: 0]
 
-  -w, --where TEXT             Filter rows with SQL-like conditions
-                               Examples: "status=active", "age>30",
-                               "name contains john", "email endswith @gmail.com"
+  -w, --where TEXT             Filter rows with SQL-like conditions; combine
+                               with AND/OR. Streams the file and stops at
+                               --num-rows matches; Parquet skips row groups
+                               via column statistics.
+                               Examples: "status=active", "age>30 AND age<65",
+                               "level=ERROR or level=FATAL"
 
   -s, --schema TEXT            Schema display: show, dont_show, schema_only
                                [default: show]
@@ -516,13 +523,25 @@ Options:
   -d, --delimiter TEXT         CSV delimiter (use \t for tab)
                                [default: comma]
 
+  --no-color                   Disable colored output (also honors NO_COLOR).
+                               Color is auto-disabled when output is piped.
+
   --profile TEXT               AWS profile name (for S3 access)
 
   --project TEXT               GCP project ID (for GCS access)
 
   --credentials TEXT           Path to GCP service account JSON file
 
-  --account TEXT               Azure storage account name
+  --stats                      Show per-column statistics (nulls, distinct,
+                               min/max) over the retrieved rows
+
+  --config-profile TEXT        Named profile from ~/.config/cloudcat/config.toml
+
+  --completion [bash|zsh|fish] Print the shell completion script and exit
+
+  --az-access-key TEXT         Azure storage account access key
+
+  -y, --yes                    Skip confirmation prompts (for scripting)
 
   --help                       Show this message and exit
 ```
@@ -696,6 +715,50 @@ pip install -e ".[all]"
 pytest
 ```
 
+## Use with AI Agents
+
+CloudCat is optimized for AI coding agents: **stdout carries only data**
+(diagnostics go to stderr, color auto-disables when piped), `-o json` emits
+parseable NDJSON, `-y` makes every command non-interactive, filtering streams
+with Parquet row-group pushdown, and local files need no credentials.
+
+The repo ships an [Agent Skill](https://agentskills.io) at
+[`skills/cloudcat/SKILL.md`](skills/cloudcat/SKILL.md) — a compact reference
+that teaches agents the optimal scanning recipes (schema discovery, filtered
+sampling, metadata-fast counts, column profiling) with zero trial-and-error.
+
+Install the skill (pick one):
+
+```bash
+# 1. Built-in installer — the skill ships inside the package (works offline)
+cloudcat --install-skill claude          # Claude Code, all projects (~/.claude/skills)
+cloudcat --install-skill claude-project  # Claude Code, this project (./.claude/skills)
+cloudcat --install-skill codex           # OpenAI Codex CLI (~/.codex/skills)
+cloudcat --install-skill print           # stdout — pipe into any other agent's instructions
+
+# 2. Claude Code plugin marketplace
+/plugin marketplace add jonathansudhakar1/cloudcat
+/plugin install cloudcat@cloudcat
+
+# 3. Or fetch it directly (no cloudcat install needed)
+curl -fsSL --create-dirs -o ~/.claude/skills/cloudcat/SKILL.md \
+  https://raw.githubusercontent.com/jonathansudhakar1/cloudcat/main/skills/cloudcat/SKILL.md
+```
+
+The recipes agents learn:
+
+```bash
+cloudcat s3://bucket/events/ -s schema_only -y                 # structure, no data read
+cloudcat s3://bucket/events/ --count -s schema_only -y         # exact rows (Parquet/ORC: metadata-only)
+cloudcat s3://bucket/events/ -w "type=purchase AND amount>250" \
+         -n 5 -o json -s dont_show -y                          # filtered NDJSON sample
+cloudcat s3://bucket/events/ --stats -n 0 -s dont_show -y      # nulls/distinct/min/max per column
+```
+
+Works with any agent framework that can run shell commands and load the
+SKILL.md (or plain markdown) as instructions. See the
+[AI Agents docs](https://cloudcatcli.com/#agents) for details.
+
 ## Roadmap
 
 - [x] Azure Blob Storage support
@@ -705,9 +768,14 @@ pytest
 - [x] SQL-like filtering (`--where` clause)
 - [x] Compression support (gzip, zstd, lz4, snappy, bz2)
 - [x] Row offset/pagination (`--offset`)
+- [x] Output to file with `--output-file`
+- [x] Compound WHERE conditions (`AND`/`OR`)
+- [x] Streaming WHERE with Parquet row-group pushdown
+- [x] Local file support (`file://` or plain paths)
+- [x] Configuration file with named profiles
+- [x] Column statistics (`--stats`)
+- [x] Shell completion (bash/zsh/fish)
 - [ ] Interactive mode with pagination
-- [ ] Output to file with `--output-file`
-- [ ] Configuration file support
 
 ## Related Projects
 

{cloudcat-0.3.8 → cloudcat-0.5.1}/README.md

@@ -96,14 +96,17 @@ pip install --upgrade 'cloudcat[all]'
 ### Requirements
 
 - **Homebrew**: macOS (Apple Silicon only). Intel Mac users should use pip.
-- **pip**: Python 3.7+ (all platforms)
+- **pip**: Python 3.9+ (all platforms)
 - Cloud provider credentials configured (see [Authentication](#authentication))
 
 ## Quick Start
 
 ```bash
-# Preview a CSV file from GCS
-cloudcat -p gcs://my-bucket/data.csv
+# Preview a CSV file from GCS (PATH is positional; -p/--path also works)
+cloudcat gcs://my-bucket/data.csv
+
+# Preview a local file — no cloud credentials needed
+cloudcat ./data.parquet
 
 # Preview a Parquet file from S3
 cloudcat -p s3://my-bucket/analytics/events.parquet
@@ -126,8 +129,8 @@ cloudcat -p s3://my-bucket/spark-output/ -i parquet
 # Read compressed files (auto-detected)
 cloudcat -p gcs://my-bucket/data.csv.gz
 
-# Filter rows with WHERE clause
-cloudcat -p s3://bucket/users.parquet --where "status=active"
+# Filter rows with WHERE (streams; stops at -n matches; AND/OR supported)
+cloudcat s3://bucket/users.parquet --where "status=active AND age>30"
 
 # Skip first 100 rows (pagination)
 cloudcat -p gcs://bucket/data.csv --offset 100 -n 10
@@ -142,6 +145,7 @@ cloudcat -p gcs://bucket/data.csv --offset 100 -n 10
 | Google Cloud Storage | `gcs://` or `gs://` | ✅ Supported |
 | Amazon S3 | `s3://` | ✅ Supported |
 | Azure Data Lake Gen2 | `abfss://` | ✅ Supported |
+| Local files | `file://` or a plain path | ✅ Supported (no credentials needed) |
 
 ### File Format Support
 
@@ -164,8 +168,8 @@ CloudCat uses intelligent streaming to minimize data transfer and egress costs:
 |--------|-------------|---------|-------------------|----------------|
 | Parquet | None/Internal | ✅ | ✅ Range requests | ✅ |
 | Parquet | External (.gz) | ❌ | ❌ | ❌ |
-| ORC | None/Internal | ❌ | ❌ | ❌ |
-| ORC | External (.gz) | ❌ | ❌ | ❌ |
+| ORC | None/Internal | ✅ Stripe-by-stripe | ✅ | ✅ |
+| ORC | External (.gz) | ❌ | ✅ | ✅ |
 | CSV | None | ✅ | ❌ | ✅ |
 | CSV | gzip/zstd/lz4/bz2 | ✅ | ❌ | ✅ |
 | CSV | snappy | ❌ | ❌ | ❌ |
@@ -194,7 +198,7 @@ CloudCat automatically detects and decompresses files based on extension (e.g.,
 
 | Format | Flag | Description |
 |--------|------|-------------|
-| Table | `-o table` | Beautiful ASCII table with colored headers (default) |
+| Table | `-o table` | Rounded table with bold headers and type-aware coloring (default) |
 | JSON | `-o json` | Standard JSON Lines output |
 | Pretty JSON | `-o jsonp` | Syntax-highlighted, indented JSON |
 | CSV | `-o csv` | Comma-separated values |
@@ -251,8 +255,8 @@ cloudcat -p gcs://bucket/events.jsonl
 ### Filtering and Pagination
 
 ```bash
-# Filter rows with WHERE clause
-cloudcat -p s3://bucket/users.parquet --where "status=active"
+# Filter rows with WHERE (streams; stops at -n matches; AND/OR supported)
+cloudcat s3://bucket/users.parquet --where "status=active AND age>30"
 cloudcat -p gcs://bucket/events.json --where "age>30"
 cloudcat -p s3://bucket/logs.csv --where "level=ERROR"
 
@@ -401,13 +405,17 @@ cloudcat -p s3://api-dumps/response.json -o csv > data.csv
 Usage: cloudcat [OPTIONS]
 
 Options:
-  -p, --path TEXT              Cloud storage path (required)
+  PATH                         Positional path: gs://, gcs://, s3://, abfss://,
+                               file:// URL, or a plain local path
+                               (-p/--path remains as a compatible alias)
                                Format: gcs://bucket/path, s3://bucket/path,
                                or abfss://container@account.dfs.core.windows.net/path
 
   -o, --output-format TEXT     Output format: table, json, jsonp, csv
                                [default: table]
 
+  -O, --output-file PATH       Write rendered data to a file instead of stdout
+
   -i, --input-format TEXT      Input format: csv, json, parquet, avro, orc, text
                                [default: auto-detect from extension]
 
@@ -420,9 +428,12 @@ Options:
   --offset INTEGER             Skip first N rows
                                [default: 0]
 
-  -w, --where TEXT             Filter rows with SQL-like conditions
-                               Examples: "status=active", "age>30",
-                               "name contains john", "email endswith @gmail.com"
+  -w, --where TEXT             Filter rows with SQL-like conditions; combine
+                               with AND/OR. Streams the file and stops at
+                               --num-rows matches; Parquet skips row groups
+                               via column statistics.
+                               Examples: "status=active", "age>30 AND age<65",
+                               "level=ERROR or level=FATAL"
 
   -s, --schema TEXT            Schema display: show, dont_show, schema_only
                                [default: show]
@@ -438,13 +449,25 @@ Options:
   -d, --delimiter TEXT         CSV delimiter (use \t for tab)
                                [default: comma]
 
+  --no-color                   Disable colored output (also honors NO_COLOR).
+                               Color is auto-disabled when output is piped.
+
   --profile TEXT               AWS profile name (for S3 access)
 
   --project TEXT               GCP project ID (for GCS access)
 
   --credentials TEXT           Path to GCP service account JSON file
 
-  --account TEXT               Azure storage account name
+  --stats                      Show per-column statistics (nulls, distinct,
+                               min/max) over the retrieved rows
+
+  --config-profile TEXT        Named profile from ~/.config/cloudcat/config.toml
+
+  --completion [bash|zsh|fish] Print the shell completion script and exit
+
+  --az-access-key TEXT         Azure storage account access key
+
+  -y, --yes                    Skip confirmation prompts (for scripting)
 
   --help                       Show this message and exit
 ```
@@ -618,6 +641,50 @@ pip install -e ".[all]"
 pytest
 ```
 
+## Use with AI Agents
+
+CloudCat is optimized for AI coding agents: **stdout carries only data**
+(diagnostics go to stderr, color auto-disables when piped), `-o json` emits
+parseable NDJSON, `-y` makes every command non-interactive, filtering streams
+with Parquet row-group pushdown, and local files need no credentials.
+
+The repo ships an [Agent Skill](https://agentskills.io) at
+[`skills/cloudcat/SKILL.md`](skills/cloudcat/SKILL.md) — a compact reference
+that teaches agents the optimal scanning recipes (schema discovery, filtered
+sampling, metadata-fast counts, column profiling) with zero trial-and-error.
+
+Install the skill (pick one):
+
+```bash
+# 1. Built-in installer — the skill ships inside the package (works offline)
+cloudcat --install-skill claude          # Claude Code, all projects (~/.claude/skills)
+cloudcat --install-skill claude-project  # Claude Code, this project (./.claude/skills)
+cloudcat --install-skill codex           # OpenAI Codex CLI (~/.codex/skills)
+cloudcat --install-skill print           # stdout — pipe into any other agent's instructions
+
+# 2. Claude Code plugin marketplace
+/plugin marketplace add jonathansudhakar1/cloudcat
+/plugin install cloudcat@cloudcat
+
+# 3. Or fetch it directly (no cloudcat install needed)
+curl -fsSL --create-dirs -o ~/.claude/skills/cloudcat/SKILL.md \
+  https://raw.githubusercontent.com/jonathansudhakar1/cloudcat/main/skills/cloudcat/SKILL.md
+```
+
+The recipes agents learn:
+
+```bash
+cloudcat s3://bucket/events/ -s schema_only -y                 # structure, no data read
+cloudcat s3://bucket/events/ --count -s schema_only -y         # exact rows (Parquet/ORC: metadata-only)
+cloudcat s3://bucket/events/ -w "type=purchase AND amount>250" \
+         -n 5 -o json -s dont_show -y                          # filtered NDJSON sample
+cloudcat s3://bucket/events/ --stats -n 0 -s dont_show -y      # nulls/distinct/min/max per column
+```
+
+Works with any agent framework that can run shell commands and load the
+SKILL.md (or plain markdown) as instructions. See the
+[AI Agents docs](https://cloudcatcli.com/#agents) for details.
+
 ## Roadmap
 
 - [x] Azure Blob Storage support
@@ -627,9 +694,14 @@ pytest
 - [x] SQL-like filtering (`--where` clause)
 - [x] Compression support (gzip, zstd, lz4, snappy, bz2)
 - [x] Row offset/pagination (`--offset`)
+- [x] Output to file with `--output-file`
+- [x] Compound WHERE conditions (`AND`/`OR`)
+- [x] Streaming WHERE with Parquet row-group pushdown
+- [x] Local file support (`file://` or plain paths)
+- [x] Configuration file with named profiles
+- [x] Column statistics (`--stats`)
+- [x] Shell completion (bash/zsh/fish)
 - [ ] Interactive mode with pagination
-- [ ] Output to file with `--output-file`
-- [ ] Configuration file support
 
 ## Related Projects
 

{cloudcat-0.3.8 → cloudcat-0.5.1}/cloudcat/__init__.py

@@ -2,4 +2,4 @@
 #!/usr/bin/env python
 """cloudcat - A CLI utility to read and display files from cloud storage."""
 
-__version__ = "0.3.8"
+__version__ = "0.5.1"