herdux-cli 0.5.0 → 0.8.2

package/README.md

@@ -8,11 +8,11 @@
   <img src=".github/assets/logo.svg" alt="Herdux banner" style="max-width: 100%; width: 600px;" />
 </p>
 
-## ⏭️ Herdux — Database Workflow CLI
+## Herdux — Database Workflow CLI
 
 A fast, interactive CLI that removes friction from daily local database workflows, especially when juggling multiple instances and large datasets.
 
-![Version](https://img.shields.io/badge/version-0.5.0-blue.svg)
+![Version](https://img.shields.io/badge/version-0.8.2-blue.svg)
 ![License](https://img.shields.io/badge/license-MIT-green.svg)
 ![Node](https://img.shields.io/badge/node-18%2B-43853d.svg)
 ![PostgreSQL](https://img.shields.io/badge/PostgreSQL-316192?style=flat&logo=postgresql&logoColor=white)
@@ -23,27 +23,21 @@ A fast, interactive CLI that removes friction from daily local database workflow
 
 > Optimized for local and development environments. Production use is supported with explicit configuration.
 
-<!-- <p align="center">
-  <img src=".github/herdux.gif" alt="herdux terminal gif" width="1220" />
-</p> -->
-
 ---
 
-## ⚡ Quick Start
+## Quick Start
 
 ```bash
 npm install -g herdux-cli
 
-# You can use either 'herdux' or the shorter 'hdx' alias
+# Use either 'herdux' or the shorter 'hdx' alias
 hdx doctor
 herdux list
 ```
 
-That's it. You're managing databases.
-
 ---
 
-## 🔌 Supported Engines
+## Supported Engines
 
 | Engine     | Status | Client Tools Required           |
 | ---------- | ------ | ------------------------------- |
@@ -51,57 +45,22 @@ That's it. You're managing databases.
 | MySQL      | ✅     | `mysql`, `mysqldump`            |
 | SQLite     | ✅     | `sqlite3`                       |
 
-Herdux resolves the engine explicitly using a strict priority order (CLI flags → profiles → saved defaults → fallback).
-
-Internally, no command ever runs without a fully resolved engine.
+Use `--engine <name>` or configure the engine in a saved profile. PostgreSQL is the default.
 
 ```bash
-# PostgreSQL (default)
-herdux list
-herdux create mydb
-
-# MySQL
-herdux --engine mysql list
-herdux --engine mysql create mydb
-
-# SQLite (file-based, no server required)
-herdux --engine sqlite list
-herdux --engine sqlite create mydb
-
-# Or save it in a profile and forget about it
-herdux config add mysql-local --port 3306 --user root --password secret --engine mysql
-herdux list -s mysql-local
+herdux list                        # PostgreSQL (default)
+herdux --engine mysql list         # MySQL
+herdux --engine sqlite list        # SQLite (file-based, no server required)
+herdux list -s my-profile          # Using a saved server profile
 ```
 
 ---
 
-## 🧠 How Herdux Thinks
-
-Herdux is designed around **strict separation of concerns**:
-
-- **Commands decide _what_ to do**
-- **Engines decide _how_ to do it**
-- **Binaries are never called directly by commands**
-- **All external behavior is isolated behind engine contracts**
-
-This architecture guarantees:
-
-- predictable behavior
-- engine-agnostic commands
-- safer destructive operations
-- easier extension to new databases
-
-If something feels “magical”, it’s probably wrong.
-
-Any change that breaks these boundaries is considered a bug.
-
----
-
 ## Why Herdux?
 
-Managing local databases through raw bash scripts or binaries is repetitive, error-prone, and painful at scale.
+Managing local databases through raw binaries is repetitive, error-prone, and different for every engine.
 
-### ❌ Without Herdux
+**Before:**
 
 ```bash
 # PostgreSQL backup
@@ -110,15 +69,14 @@ pg_dump -U postgres -h localhost -p 5416 -Fc -f ./backups/mydb.dump mydb
 # MySQL backup
 mysqldump -u root -h localhost -P 3306 -p mydb > ./backups/mydb.sql
 
-# Manually drop, restore, check tools...
 # Different flags, different tools, different muscle memory for each engine.
 ```
 
-### ✅ With Herdux
+**After:**
 
 ```bash
 herdux backup mydb --drop --yes        # Backup + drop in one shot
-herdux restore ./backups/mydb.dump --db mydb   # Auto-creates DB if missing & detects format
+herdux restore ./backups/mydb.dump --db mydb   # Detects format, creates DB if missing
 herdux clean                            # Multi-select and batch-drop databases
 herdux doctor                           # Full system health check
 ```
@@ -127,80 +85,23 @@ Same commands. Any engine. Fewer flags. Fewer mistakes. Zero terminal fatigue.
 
 ---
 
-## 🎯 Who is Herdux for?
-
-**Herdux** was built _for developers, by developers_.
-
-It was born from the daily frustration of constantly having to restore backups to test specific states, drop corrupted databases during development, and juggle raw database binaries.
-
-It is specifically designed for developers who:
-
-- Manage local infrastructures and need to check disk sizes before seeding new databases.
-- Want to quickly clone, seed, and reset databases without reading `man` pages.
-- Need safe backup & restore workflows that don't rely on fragile bash scripts.
-- Prefer terminal-first tooling.
-- Want predictable connection resolution without hidden magic.
-- Work with **multiple database engines** (PostgreSQL, MySQL, SQLite) and want a unified interface.
-
-If you manage databases locally, Herdux was created to solve your pain.
-
----
-
-## 🚀 Key Features
-
-- **🔌 Multi-Engine Support** — First-class support for PostgreSQL, MySQL, and SQLite. Same commands, same workflow, any engine.
-- **📋 Smart Listing** — Optimized listing strategy for massive clusters. Optional `--size` flag for disk usage analysis, sorted largest-first.
-- **💾 Intelligent Backup & Restore** — Supports Custom (`.dump`) and Plain (`.sql`) formats. Auto-detects the right tool for restores.
-- **🧹 Bulk Cleanup** — Multi-select databases, optionally backup, and batch-drop them. Reclaim disk space instantly.
-- **🩺 System Diagnostics** — One-command health check verifying binaries, authentication, and connectivity.
-- **⚙️ Persistent Profiles** — Save named server configurations with engine type. Switch between environments with `-s pg16`.
-- **🎯 Smart Connection & Engine Resolution** — Explicit CLI flags → profiles → saved defaults → auto-discovery. Always predictable.
-
----
-
-## 💡 Philosophy
-
-**Herdux** combines _herd_ and _UX_ — delivering a better developer experience when managing your local database clusters. The name reflects our focus on improving the developer experience of managing database herds.
-
-**Herdux** follows three principles:
-
-- **Safety first** — Never drops data without explicit confirmation or a verified backup.
-- **Explicit over implicit** — Connection and engine resolution follows a strict, documented priority. No magic.
-- **Developer workflow optimization** — Every command is designed to save you from repetitive terminal work.
-
----
-
-## 🔒 Safety
-
-**Herdux** handles destructive operations with care:
-
-- **Never drops a database** unless explicit confirmation is given
-- **Aborts the entire operation** if a safety backup fails during `herdux clean`
-- **Validates backup tool exit codes** before considering a backup successful
-- **Requires `--drop` flag** intentionally — dropping is never the default
-- **`--yes` must be combined with `--drop`** — cannot skip confirmation alone
-
-> If you request a backup before dropping and that backup fails, **Herdux** stops immediately. No data is lost.
-
----
-
-## 🧩 Requirements
+## Requirements
 
 - **Node.js** 18 or higher
-- **For PostgreSQL:** `psql`, `pg_dump`, `pg_restore` installed and available in your `PATH`
-- **For MySQL:** `mysql`, `mysqldump` installed and available in your `PATH`
-- **For SQLite:** `sqlite3` installed and available in your `PATH`
+- **For PostgreSQL:** `psql`, `pg_dump`, `pg_restore` installed and in your `PATH`
+- **For MySQL:** `mysql`, `mysqldump` installed and in your `PATH`
+- **For SQLite:** `sqlite3` installed and in your `PATH`
 
 > [!TIP]
-> Run `herdux doctor` after installation to verify everything is set up correctly. The doctor command checks the tools for the active engine.
+> Run `herdux doctor` after installation to verify everything is correctly set up.
 
 ---
 
-## 📦 Installation
+## Installation
 
 **npm (recommended):**
 
-> **⚠️ IMPORTANT:** You must use the `-g` (global) flag for the CLI to be accessible anywhere in your terminal.
+> **Important:** Use the `-g` flag so the CLI is accessible anywhere in your terminal.
 
 ```bash
 npm install -g herdux-cli
@@ -218,9 +119,7 @@ npm link
 
 ---
 
-## 🛠️ Commands
-
-All commands work with PostgreSQL, MySQL, and SQLite. Use `--engine mysql` or `--engine sqlite`, or configure the engine in your server profile.
+## Commands
 
 ### `herdux version`
 
@@ -233,11 +132,7 @@ herdux --engine mysql version
 
 ### `herdux doctor`
 
-Runs a full system health check:
-
-- Verifies the required client tools are installed and reachable (engine-specific)
-- Attempts a live connection using the resolved configuration
-- Tests authentication against the target server
+Runs a full system health check: verifies client tools, tests connectivity, and validates authentication.
 
 ```bash
 herdux doctor
@@ -246,17 +141,17 @@ herdux --engine mysql doctor
 
 ---
 
-### 📋 `herdux list`
+### `herdux list`
 
 Lists all databases on the connected server.
 
 ```bash
-herdux list              # Fast listing (name, owner, encoding)
-herdux ls --size         # Includes disk size, sorted largest → smallest
+herdux list              # Name, owner, encoding
+herdux ls --size         # Includes disk size, sorted largest to smallest
 ```
 
 > [!NOTE]
-> The `--size` flag calculates physical disk usage. On servers with dozens of multi-GB databases, this may take a few minutes depending on disk speed.
+> The `--size` flag calculates physical disk usage. On servers with dozens of multi-GB databases, this may take a few minutes.
 
 ---
 
@@ -279,210 +174,233 @@ herdux drop my_old_db
 
 ---
 
-### 🧹 `herdux clean` — Bulk Cleanup
-
-Working with seed-heavy development databases? Need to reclaim disk space fast?
-
-`herdux clean` allows you to:
+### `herdux clean`
 
-- **Multi-select** databases from an interactive checkbox UI
-- **Optionally generate safety backups** before any destructive action
-- **Batch-drop** all selected databases safely
-- **Abort immediately** if any backup fails, preventing data loss
+Interactive bulk cleanup: multi-select databases, optionally back them up, and batch-drop them.
 
 ```bash
 herdux clean
 ```
 
-This is designed for the real-world dev workflow: clone databases, experiment, then clean up everything in one shot.
+Aborts immediately if any safety backup fails. No data is dropped without a confirmed backup.
 
 ---
 
-### 📦 `herdux backup <database>`
+### `herdux backup <database>`
 
-Generates a timestamped backup in `./backups/`.
+Creates a timestamped backup in `~/.herdux/backups/` by default.
 
 ```bash
-herdux backup mydb                       # Custom format (.dump for PG, .sql for MySQL)
-herdux backup mydb --format plain        # Plain SQL (.sql)
-herdux backup mydb --drop                # Backup, then ask to drop
-herdux backup mydb --drop --yes          # Backup + drop, no questions
-herdux backup mydb -o ./my-backups       # Custom output directory
+herdux backup mydb                             # Engine-native format (.dump for PG, .db for SQLite, .sql for MySQL)
+herdux backup mydb --format plain              # Plain SQL (.sql)
+herdux backup mydb --drop                      # Backup, then prompt to drop
+herdux backup mydb --drop --yes                # Backup + drop, no confirmation
+herdux backup mydb -o ./my-backups             # Custom output directory
+herdux backup mydb --upload backups/           # Backup and upload to S3 prefix backups/
+herdux backup mydb --upload backups/ --no-keep # Backup, upload, then delete local file
 ```
 
-| Option                | Description                                         |
-| --------------------- | --------------------------------------------------- |
-| `-F, --format <type>` | `custom` (default, compressed) or `plain` (raw SQL) |
-| `-d, --drop`          | Prompt to drop database after successful backup     |
-| `-y, --yes`           | Skip drop confirmation (requires `--drop`)          |
-| `-o, --output <dir>`  | Output directory (default: `./backups`)             |
+| Option                | Description                                                       |
+| --------------------- | ----------------------------------------------------------------- |
+| `-F, --format <type>` | `custom` (default, engine-native) or `plain` (SQL)                |
+| `-d, --drop`          | Prompt to drop the database after a successful backup             |
+| `-y, --yes`           | Skip drop confirmation (requires `--drop`)                        |
+| `-o, --output <dir>`  | Output directory (default: `~/.herdux/backups`)                   |
+| `--upload [prefix]`   | Upload backup to S3 after creation (requires cloud config)        |
+| `--no-keep`           | Delete local backup after successful upload (requires `--upload`) |
 
 ---
 
-### 📥 `herdux restore <file>`
+### `herdux restore <file>`
 
-Restores a database from a backup file. Automatically detects the format:
-
-- `.sql` → uses the appropriate SQL import tool
-- `.dump` or any other extension → uses the appropriate restore tool
+Restores a database from a backup file. Auto-detects the format based on file extension.
 
 ```bash
 herdux restore ./backups/mydb_2026-02-23.dump --db mydb
 herdux restore ./exports/data.sql --db mydb
+herdux restore archive.bkp --db mydb --format custom   # Override auto-detection
 ```
 
-Need to override auto-detection? Use `--format`:
-
-```bash
-herdux restore archive.bkp --db mydb --format custom
-herdux restore script.txt --db mydb --format plain
-```
+The target database is automatically created if it does not exist.
 
 > [!NOTE]
-> When restoring dumps from managed environments (e.g., AWS RDS), Herdux automatically configures the underlying engine to ignore ownership and role assignments. This prevents errors caused by production roles that do not exist locally. If the restore engine completes with non-fatal warnings (such as missing roles), Herdux will inform you and proceed normally rather than failing.
+> When restoring dumps from managed environments (e.g. AWS RDS), Herdux configures the restore tool to ignore ownership and role assignments, preventing errors from missing production roles.
 
 ---
 
-## ⚙️ Configuration & Server Profiles
+### `herdux inspect <file>`
 
-`herdux` stores configuration locally at `~/.herdux/config.json`.
+Inspects the contents of a backup file without connecting to a database. Completely offline.
 
-### Set Global Defaults
+| Extension         | Output                                                                 |
+| ----------------- | ---------------------------------------------------------------------- |
+| `.dump`           | PostgreSQL custom format: full Table of Contents (`pg_restore --list`) |
+| `.tar`            | PostgreSQL tar format: full Table of Contents (`pg_restore --list`)    |
+| `.sql`            | Plain SQL (any engine): CREATE TABLE, VIEW, INDEX, SEQUENCE statements |
+| `.db` / `.sqlite` | SQLite database file: schema (`sqlite3 .schema`)                       |
 
 ```bash
-herdux config set engine postgres        # Default engine
-herdux config set user postgres
-herdux config set password my_secret
-herdux config set port 5432
+hdx inspect backup.dump           # Table of Contents of a PostgreSQL custom dump
+hdx inspect backup.tar            # Table of Contents of a PostgreSQL tar dump
+hdx inspect export.sql            # CREATE statements extracted from plain SQL
+hdx inspect mydb.db               # SQLite schema
 ```
 
-### Named Server Profiles
+---
+
+### `herdux docker`
 
-Manage multiple database instances effortlessly:
+Manages database containers running via Docker. Does not require a live database connection.
 
 ```bash
-# PostgreSQL profiles
-herdux config add pg16 --port 5416
-herdux config add pg17 --port 5417 --user admin
+hdx docker list             # List running postgres/mysql containers
+hdx docker list --all       # Include stopped containers
+hdx docker start pg-dev     # Start a stopped container
+hdx docker stop pg-dev      # Stop a running container
+hdx docker stop pg-dev --remove   # Stop and remove the container
+```
 
-# MySQL profiles (engine is saved in the profile)
-herdux config add mysql-dev --port 3306 --user root --password secret --engine mysql
+---
 
-# Remote servers
-herdux config add staging --host 192.168.0.10 --port 5432
-```
+### `herdux cloud`
 
-Then connect using the `-s` flag:
+Manages backup files in S3-compatible cloud storage (AWS S3, Cloudflare R2, MinIO, DigitalOcean Spaces, and others).
 
 ```bash
-herdux list -s pg16
-herdux backup mydb -s mysql-dev
+# Configure
+hdx cloud config bucket my-bucket
+hdx cloud config region us-east-1
+hdx cloud config access-key AKIAIO...
+hdx cloud config secret-key wJalrX...
+hdx cloud config endpoint https://account.r2.cloudflarestorage.com  # optional, for non-AWS providers
+
+# Browse and manage backups in the bucket
+hdx cloud list                                    # Directory mode: list immediate children at root
+hdx cloud list backups/mydb/                      # List immediate children at path (positional)
+hdx cloud list --prefix backups/mydb/             # Same as above (flag form)
+hdx cloud list --recursive                        # List all objects in the bucket
+hdx cloud list backups/ --recursive               # List all objects under a prefix
+hdx cloud download backups/mydb_2026-03-03.dump              # Save to ~/.herdux/backups/
+hdx cloud download backups/mydb_2026-03-03.dump -o /tmp/     # Save to custom directory
+hdx cloud upload ./mydb_2026-03-03.dump                      # Upload file to bucket root
+hdx cloud upload ./mydb_2026-03-03.dump --prefix backups/    # Upload under a prefix
+hdx cloud delete backups/mydb_2026-03-03.dump     # Verifies existence, then asks for confirmation
+hdx cloud delete backups/mydb_2026-03-03.dump --yes
+
+# Backup directly to S3
+hdx backup mydb --upload backups/              # Backup and upload to prefix backups/
+hdx backup mydb --upload                       # Backup and upload to bucket root
+hdx backup mydb --upload backups/ --no-keep    # Backup, upload, then delete local file
+
+# Restore directly from S3
+hdx restore s3://my-bucket/backups/mydb_2026-03-03.dump --db mydb
 ```
 
-Or just run a command without flags — if you have saved profiles, Herdux will show an interactive selection menu displaying the engine for each profile.
-
-### View & Manage Config
+Credentials can also be provided via env vars (recommended for CI/production):
 
 ```bash
-herdux config list           # Show all saved settings and profiles
-herdux config get port       # Get a specific value
-herdux config rm pg16        # Remove a server profile
-herdux config reset          # Clear all configuration
+export AWS_ACCESS_KEY_ID=AKIAIO...
+export AWS_SECRET_ACCESS_KEY=wJalrX...
+export AWS_DEFAULT_REGION=us-east-1
 ```
 
 ---
 
-## 🔌 Connection & Engine Resolution
-
-When resolving how to connect and which engine to use, **Herdux** follows a strict, predictable priority order:
+## Configuration & Server Profiles
 
-### Engine Priority
+Configuration is stored at `~/.herdux/config.json`.
 
-| Priority | Source             | Example                          |
-| -------- | ------------------ | -------------------------------- |
-| 1️⃣       | **CLI flag**       | `herdux --engine mysql list`     |
-| 2️⃣       | **Server profile** | Profile's `engine` field         |
-| 3️⃣       | **Saved default**  | `herdux config set engine mysql` |
-| 4️⃣       | **Fallback**       | `postgres`                       |
+### Global defaults
 
-### Connection Priority
+```bash
+herdux config set engine postgres
+herdux config set user postgres
+herdux config set password my_secret
+herdux config set port 5432
+```
 
-| Priority | Source             | Example                                       |
-| -------- | ------------------ | --------------------------------------------- |
-| 1️⃣       | **CLI flags**      | `herdux list --port 5417`                     |
-| 2️⃣       | **Server profile** | `herdux list -s pg16`                         |
-| 3️⃣       | **Saved defaults** | `herdux config set port 5432`                 |
-| 4️⃣       | **Auto-discovery** | Scans common ports; prompts if multiple found |
+### Named server profiles
 
-This means explicit input always wins. No surprises.
+```bash
+herdux config add pg16 --port 5416
+herdux config add pg17 --port 5417 --user admin
+herdux config add mysql-dev --port 3306 --user root --password secret --engine mysql
+herdux config add staging --host 192.168.0.10 --port 5432
+```
 
----
+Use profiles with the `-s` flag:
 
-## 🤔 Why not pgAdmin / phpMyAdmin?
+```bash
+herdux list -s pg16
+herdux backup mydb -s mysql-dev
+```
 
-**Herdux** is not a GUI replacement.
-It's a workflow accelerator for developers who live in the terminal.
+### Manage config
 
-No GUI. No overhead. Just speed.
+```bash
+herdux config list           # Show all settings and profiles
+herdux config get port       # Get a specific value
+herdux config rm pg16        # Remove a profile
+herdux config reset          # Clear all configuration
+```
 
 ---
 
-## 🧠 Design Principles
+## Connection & Engine Resolution
 
-- No hidden defaults.
-- No destructive magic.
-- Deterministic connection and engine resolution.
-- Explicit and composable commands.
-- Engine-agnostic: same interface, any database.
+Herdux follows a strict, predictable priority when resolving how to connect.
 
----
-
-## 🐳 Docker Support
+**Engine priority:**
 
-> Docker MUST NOT be required for normal CLI usage.
-> Docker is currently used internally for end-to-end testing to validate real database workflows.
-> Runtime Docker integration (detecting and managing live containers) is planned.
+| Priority | Source         | Example                          |
+| -------- | -------------- | -------------------------------- |
+| 1        | CLI flag       | `herdux --engine mysql list`     |
+| 2        | Server profile | Profile's `engine` field         |
+| 3        | Saved default  | `herdux config set engine mysql` |
+| 4        | Fallback       | `postgres`                       |
 
----
+**Connection priority:**
 
-## 🗺 Roadmap
+| Priority | Source         | Example                                       |
+| -------- | -------------- | --------------------------------------------- |
+| 1        | CLI flags      | `herdux list --port 5417`                     |
+| 2        | Server profile | `herdux list -s pg16`                         |
+| 3        | Saved defaults | `herdux config set port 5432`                 |
+| 4        | Auto-discovery | Scans common ports; prompts if multiple found |
 
-See [ROADMAP.md](./ROADMAP.md) for our detailed future plans, including Docker integration and encrypted backups.
+Explicit input always wins. No surprises.
 
 ---
 
-## 🤝 Contributing
-
-PRs are welcome! Please open an issue first to discuss major changes.
+## Contributing
 
 ```bash
 git clone https://github.com/herdux/herdux-cli.git
 cd herdux-cli
 npm install
-npm run dev
 
-# Run the unit test suite
-npm run test:unit
-# Run the E2E test suites (requires Docker)
-npm run test:e2e:pgsql
-npm run test:e2e:mysql
+npm run test:unit           # Unit tests (238 tests, all engines)
+npm run test:integration    # Integration tests
+npm run test:e2e:pgsql      # E2E tests for PostgreSQL (requires Docker)
+npm run test:e2e:mysql      # E2E tests for MySQL (requires Docker)
+npm run test:e2e:sqlite     # E2E tests for SQLite (requires sqlite3)
 ```
 
-> Herdux follows strict architectural boundaries.
-> Commands are engine-agnostic, engines encapsulate all database-specific behavior, and all binaries are isolated behind adapters.
-> Please keep these boundaries intact when contributing.
+Herdux follows strict architectural boundaries: commands are engine-agnostic, engines encapsulate all database-specific behavior, and all binaries are isolated behind adapters. Keep these boundaries intact when contributing.
+
+PRs are welcome. Open an issue first for major changes.
 
 ---
 
-## ☕ Support the Project
+## Support
 
-If **Herdux** has saved you hours of debugging and database wrangling, consider supporting the project! It helps keep it active and open-source.
+If Herdux has saved you hours of debugging and database wrangling, consider supporting the project:
 
 <a href="https://github.com/sponsors/eduardozaniboni" target="_blank"><img src="https://img.shields.io/badge/Sponsor-%E2%9D%A4-pink?style=for-the-badge&logo=github" alt="GitHub Sponsors"></a>
 <a href="https://www.buymeacoffee.com/eduardozaniboni" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 28px !important;width: 100px !important;" ></a>
 
 ---
 
-## 📄 License
+## License
 
 MIT