@appkit/llamacpp-cli 1.8.0 → 1.10.0

package/CHANGELOG.md

@@ -2,6 +2,64 @@
 
 All notable changes to this project will be documented in this file. See [commit-and-tag-version](https://github.com/absolute-version/commit-and-tag-version) for commit guidelines.
 
+## [1.10.0](https://github.com/appkitstudio/llamacpp-cli/compare/v1.9.0...v1.10.0) (2026-02-02)
+
+
+### Features
+
+* add Config screen TUI for editing server configuration ([0555eb8](https://github.com/appkitstudio/llamacpp-cli/commit/0555eb87b25fd9c0496ddf04bf59433ae0e9dc48))
+* add Models management TUI for displaying and deleting models ([45b312f](https://github.com/appkitstudio/llamacpp-cli/commit/45b312f160e476a8effd0ad92a7f88e2b533f9c5))
+* add splash screen with ASCII logo on TUI startup ([9a2bbba](https://github.com/appkitstudio/llamacpp-cli/commit/9a2bbbac80c8736548a71d38230a69b3745198ea))
+* add Start/Stop controls and UI improvements to TUI ([2409c8e](https://github.com/appkitstudio/llamacpp-cli/commit/2409c8e283f37050f2b4faccbaef82bf6db2ac39))
+* enhance TUI with interactive dashboard and deprecate old commands ([464c224](https://github.com/appkitstudio/llamacpp-cli/commit/464c224aefc2592442f80e72a076ec792a9d36dc))
+
+
+### Code Refactoring
+
+* simplify TUI code for clarity and maintainability ([2b50d52](https://github.com/appkitstudio/llamacpp-cli/commit/2b50d5288b2ecd5bed4619518f50d74bd43c6b76))
+
+## [1.9.0](https://github.com/appkitstudio/llamacpp-cli/compare/v1.8.0...v1.9.0) (2026-01-27)
+
+
+### Features
+
+* add unified router endpoint for automatic model routing ([6db0892](https://github.com/appkitstudio/llamacpp-cli/commit/6db0892adca753c021c7d9be6b69231c983170a1))
+
+## [1.9.0] (2026-01-26)
+
+### Features
+
+* **router**: add unified router endpoint for automatic model routing ([#TBD])
+
+  The router provides a single OpenAI-compatible endpoint that automatically routes requests to the correct backend server based on model name. Perfect for LLM clients that don't support multiple endpoints.
+
+  **Key features:**
+  - Single endpoint (default: http://localhost:9100) for all models
+  - Automatic routing based on `model` field in requests
+  - Zero-config model discovery from running servers
+  - Aggregated `/v1/models` endpoint
+  - Native Node.js HTTP proxy (no external dependencies)
+  - Streaming support for chat completions
+  - Comprehensive error handling (404, 503, 502, 504)
+
+  **Commands:**
+  - `llamacpp router start` - Start router service
+  - `llamacpp router stop` - Stop router service
+  - `llamacpp router status` - Show status and available models
+  - `llamacpp router restart` - Restart router
+  - `llamacpp router config` - Update configuration (port, host, timeout, health-interval)
+
+  **Usage:**
+  ```python
+  from openai import OpenAI
+
+  client = OpenAI(base_url="http://localhost:9100/v1", api_key="not-needed")
+  response = client.chat.completions.create(
+      model="llama-3.2-3b-instruct-q4_k_m.gguf",
+      messages=[{"role": "user", "content": "Hello!"}]
+  )
+  ```
+
 ## [1.7.0](https://github.com/appkitstudio/llamacpp-cli/compare/v1.6.0...v1.7.0) (2026-01-23)
 
 

package/README.md

@@ -12,7 +12,9 @@ CLI tool to manage local llama.cpp servers on macOS. Provides an Ollama-like exp
 ## Features
 
 - 🚀 **Easy server management** - Start, stop, and monitor llama.cpp servers
+- 🔀 **Unified router** - Single OpenAI-compatible endpoint for all models with automatic routing and request logging
 - 🤖 **Model downloads** - Pull GGUF models from Hugging Face
+- 📦 **Models Management TUI** - Browse, search, and delete models without leaving the TUI. Search HuggingFace, download with progress tracking, manage local models
 - ⚙️ **Smart defaults** - Auto-configure threads, context size, and GPU layers based on model size
 - 🔌 **Auto port assignment** - Automatically find available ports (9000-9999)
 - 📊 **Real-time monitoring TUI** - Multi-server dashboard with drill-down details, live GPU/CPU/memory metrics, token generation speeds, and animated loading states
@@ -74,18 +76,16 @@ llamacpp ls
 # Create and start a server (auto-assigns port, uses smart defaults)
 llamacpp server create llama-3.2-3b-instruct-q4_k_m.gguf
 
-# View running servers
+# Open interactive TUI dashboard (multi-server monitoring)
+llamacpp
+# Press 'M' to access Models Management TUI
+
+# List all servers (static table)
 llamacpp ps
 
 # View log sizes for all servers
 llamacpp logs
 
-# Monitor all servers (multi-server dashboard)
-llamacpp server monitor
-
-# Or monitor a specific server
-llamacpp server monitor llama-3.2-3b
-
 # Chat with your model interactively
 llamacpp server run llama-3.2-3b
 
@@ -140,6 +140,166 @@ curl http://localhost:9000/health
 
 The server is fully compatible with OpenAI's API format, so you can use it with any OpenAI-compatible client library.
 
+## Router (Unified Endpoint)
+
+The router provides a single OpenAI-compatible endpoint that automatically routes requests to the correct backend server based on the model name. This is perfect for LLM clients that don't support multiple endpoints.
+
+### Quick Start
+
+```bash
+# Start the router (default port: 9100)
+llamacpp router start
+
+# Configure your LLM client to use http://localhost:9100
+# The router automatically routes requests to the correct server based on model name
+```
+
+### Commands
+
+```bash
+llamacpp router start       # Start the router service
+llamacpp router stop        # Stop the router service
+llamacpp router status      # Show router status and available models
+llamacpp router restart     # Restart the router
+llamacpp router config      # Update router settings (--port, --host, --timeout, --health-interval, --verbose)
+llamacpp router logs        # View router logs (with --follow, --verbose, --clear options)
+```
+
+### Usage Example
+
+The router acts as a single endpoint for all your models:
+
+```python
+from openai import OpenAI
+
+client = OpenAI(
+    base_url="http://localhost:9100/v1",
+    api_key="not-needed"  # API key not required for local servers
+)
+
+# Router automatically routes to the correct server based on model name
+response = client.chat.completions.create(
+    model="llama-3.2-3b-instruct-q4_k_m.gguf",
+    messages=[{"role": "user", "content": "Hello!"}]
+)
+```
+
+### Supported Endpoints
+
+- `POST /v1/chat/completions` - Chat completions (routes to correct backend)
+- `POST /v1/embeddings` - Text embeddings (routes to correct backend)
+- `GET /v1/models` - List all available models from running servers
+- `GET /health` - Router health check
+
+### Configuration
+
+The router can be configured with:
+
+```bash
+# Change port
+llamacpp router config --port 9200 --restart
+
+# Update request timeout (ms)
+llamacpp router config --timeout 60000 --restart
+
+# Update health check interval (ms)
+llamacpp router config --health-interval 3000 --restart
+
+# Change bind address (for remote access)
+llamacpp router config --host 0.0.0.0 --restart
+
+# Enable verbose logging (saves detailed JSON logs)
+llamacpp router config --verbose true --restart
+
+# Disable verbose logging
+llamacpp router config --verbose false --restart
+```
+
+**Note:** Changes require a restart to take effect. Use `--restart` flag to apply immediately.
+
+### Logging
+
+The router uses separate log streams for different purposes (nginx-style):
+
+| Log File | Purpose | Content |
+|----------|---------|---------|
+| `router.stdout` | Request activity | Model routing, status codes, timing, prompts |
+| `router.stderr` | System messages | Startup, shutdown, errors, proxy failures |
+| `router.log` | Structured JSON | Detailed entries for programmatic parsing (verbose mode) |
+
+**View recent logs:**
+```bash
+# Show activity logs (default - stdout)
+llamacpp router logs
+
+# Show system logs (errors, startup messages)
+llamacpp router logs --stderr
+
+# Follow activity in real-time
+llamacpp router logs --follow
+
+# Show last 10 lines
+llamacpp router logs --lines 10
+```
+
+**Log formats:**
+
+Activity logs (stdout):
+```
+200 POST /v1/chat/completions → llama-3.2-3b-instruct-q4_k_m.gguf (127.0.0.1:9001) 1234ms | "What is..."
+404 POST /v1/chat/completions → unknown-model 3ms | "test" | Error: No server found
+```
+
+System logs (stderr):
+```
+[Router] Listening on http://127.0.0.1:9100
+[Router] PID: 12345
+[Router] Proxy request failed: ECONNREFUSED
+```
+
+Verbose JSON logs (router.log) - enable with `--verbose true`:
+```bash
+llamacpp router logs --verbose
+```
+
+**Log management:**
+```bash
+# Clear activity log
+llamacpp router logs --clear
+
+# Clear all router logs (stdout, stderr, verbose)
+llamacpp router logs --clear-all
+
+# Rotate log files with timestamp
+llamacpp router logs --rotate
+
+# View system logs instead of activity
+llamacpp router logs --stderr
+```
+
+**What's logged (activity):**
+- ✅ Model name used
+- ✅ HTTP status code (color-coded)
+- ✅ Request duration (ms)
+- ✅ Backend server (host:port)
+- ✅ First 50 chars of prompt
+- ✅ Error messages (if failed)
+
+**Verbose mode benefits:**
+- Detailed JSON logs for LLM/script parsing
+- Stored in `~/.llamacpp/logs/router.log`
+- Automatic rotation when exceeding 100MB
+- Machine-readable format with timestamps
+
+### How It Works
+
+1. Router receives request with `model` field
+2. Finds running server configured for that model
+3. Proxies request to backend server
+4. Streams response back to client
+
+If the requested model's server is not running, the router returns a 503 error with a helpful message.
+
 ### Example Output
 
 Creating a server:
@@ -189,6 +349,15 @@ Assistant: The capital of France is Paris...
 
 ## Commands
 
+### `llamacpp`
+Launch the interactive TUI dashboard for monitoring and managing servers.
+
+```bash
+llamacpp
+```
+
+See [Interactive TUI Dashboard](#interactive-tui-dashboard) for full details.
+
 ### `llamacpp ls`
 List all GGUF models in ~/models directory.
 
@@ -307,6 +476,47 @@ llamacpp logs --rotate
 
 **Use case:** Quickly see which servers are accumulating large logs, or clean up all logs at once.
 
+## Models Management TUI
+
+The Models Management TUI is accessible by pressing `M` from the `llamacpp` list view. It provides a full-featured interface for managing local models and searching/downloading new ones.
+
+**Features:**
+- **Browse local models** - View all GGUF files with size, modification date, and server usage
+- **Delete models** - Remove models with automatic cleanup of associated servers
+- **Search HuggingFace** - Find and browse models from Hugging Face repository
+- **Download with progress** - Real-time progress tracking for model downloads
+- **Seamless navigation** - Switch between monitoring and models management
+
+**Quick Access:**
+```bash
+# Launch TUI and press 'M' to open Models Management
+llamacpp
+```
+
+**Models View:**
+- View all installed models in scrollable table
+- See which servers are using each model
+- Color-coded status (green = safe to delete, yellow/gray = servers using)
+- Delete models with Enter or D key
+- Cascade deletion: automatically removes associated servers
+
+**Search View (press 'S' from Models view):**
+- Search HuggingFace models by name
+- Browse search results with download counts and likes
+- Expand models to show available GGUF files
+- Download files with real-time progress tracking
+- Cancel downloads with ESC (cleans up partial files)
+
+**Keyboard Controls:**
+- **M** - Switch to Models view (from TUI list view)
+- **↑/↓** or **k/j** - Navigate lists
+- **Enter** - Select/download/delete
+- **S** - Open search view (from models view)
+- **/** or **I** - Focus search input (in search view)
+- **R** - Refresh view
+- **ESC** - Back/cancel
+- **Q** - Quit
+
 ## Server Management
 
 ### `llamacpp server create <model> [options]`
@@ -523,30 +733,23 @@ The compact format shows one line per HTTP request and includes:
 
 Use `--http` to see full request/response JSON, or `--verbose` option to see all internal server logs.
 
-### `llamacpp server monitor [identifier]`
-Real-time monitoring TUI showing server metrics, GPU/CPU usage, and active inference slots.
+## Interactive TUI Dashboard
 
-![Server Monitoring TUI](https://raw.githubusercontent.com/dweaver/llamacpp-cli/main/docs/images/monitor-detail.png)
-
-**Two Modes:**
+The main way to monitor and manage servers is through the interactive TUI dashboard, launched by running `llamacpp` with no arguments.
 
-**1. Multi-Server Dashboard (no identifier):**
 ```bash
-llamacpp server monitor
+llamacpp
 ```
-Shows overview of all servers with system resources. Use arrow keys (↑/↓) or vim keys (k/j) to navigate, then press Enter to view server details.
 
-**2. Single-Server Monitor (with identifier):**
-```bash
-# Monitor by partial name
-llamacpp server monitor llama-3.2-3b
-
-# Monitor by port
-llamacpp server monitor 9000
+![Server Monitoring TUI](https://raw.githubusercontent.com/dweaver/llamacpp-cli/main/docs/images/monitor-detail.png)
 
-# Monitor by server ID
-llamacpp server monitor llama-3-2-3b
-```
+**Features:**
+- Multi-server dashboard with real-time metrics
+- Drill-down to single-server detail view
+- Create, start, stop, and remove servers without leaving the TUI
+- Edit server configuration inline
+- Access Models Management (press `M`)
+- Historical metrics with time-series charts
 
 **Multi-Server Dashboard:**
 ```
@@ -571,19 +774,21 @@ llamacpp server monitor llama-3-2-3b
 - **System Resources** - GPU/CPU/ANE utilization, memory usage, temperature
 
 **Keyboard Shortcuts:**
-- **Multi-Server Mode:**
+- **List View (Multi-Server):**
   - `↑/↓` or `k/j` - Navigate server list
   - `Enter` - View details for selected server
-  - `ESC` - Back to list (from detail view)
-  - `H` - View historical metrics
-  - `R` - Force refresh now
-  - `+/-` - Adjust update speed
-  - `Q` - Quit
-- **Single-Server Mode:**
-  - `H` - View historical metrics
-  - `R` - Force refresh now
-  - `+/-` - Adjust update speed
-  - `Q` - Quit
+  - `N` - Create new server
+  - `M` - Switch to Models Management
+  - `H` - View historical metrics (all servers)
+  - `ESC` - Exit TUI
+  - `Q` - Quit immediately
+- **Detail View (Single-Server):**
+  - `S` - Start/Stop server (toggles based on status)
+  - `C` - Open configuration screen
+  - `R` - Remove server (with confirmation)
+  - `H` - View historical metrics (this server)
+  - `ESC` - Back to list view
+  - `Q` - Quit immediately
 - **Historical View:**
   - `H` - Toggle Hour View (Recent ↔ Hour)
   - `ESC` - Back to live monitoring
@@ -621,7 +826,7 @@ Press `H` from any live monitoring view to see historical time-series charts. Th
 
 **Data Collection:**
 
-Historical data is automatically collected whenever you run the monitor command. Data is retained for 24 hours in `~/.llamacpp/history/<server-id>.json` files, then automatically pruned.
+Historical data is automatically collected whenever you run the TUI (`llamacpp`). Data is retained for 24 hours in `~/.llamacpp/history/<server-id>.json` files, then automatically pruned.
 
 **Multi-Server Historical View:**
 
@@ -643,13 +848,15 @@ For GPU and CPU utilization metrics, install macmon:
 brew install vladkens/tap/macmon
 ```
 
-Without macmon, the monitor still shows:
+Without macmon, the TUI still shows:
 - ✅ Server status and uptime
 - ✅ Active slots and token generation speeds
 - ✅ Memory usage (via built-in vm_stat)
 - ❌ GPU/CPU/ANE utilization (requires macmon)
 
-**Identifiers:** Port number, server ID, or partial model name
+### Deprecated: `llamacpp server monitor`
+
+The `llamacpp server monitor` command is deprecated. Use `llamacpp` instead to launch the TUI dashboard.
 
 ## Configuration
 
@@ -735,7 +942,9 @@ llamacpp server logs <identifier> --errors
 npm install
 
 # Run in development mode
-npm run dev -- ps
+npm run dev              # Launch TUI
+npm run dev -- ps        # List servers (static table)
+npm run dev -- ls        # List models
 
 # Build for production
 npm run build

package/dist/cli.js

@@ -1,5 +1,38 @@
 #!/usr/bin/env node
 "use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
+    Object.defineProperty(o, "default", { enumerable: true, value: v });
+}) : function(o, v) {
+    o["default"] = v;
+});
+var __importStar = (this && this.__importStar) || (function () {
+    var ownKeys = function(o) {
+        ownKeys = Object.getOwnPropertyNames || function (o) {
+            var ar = [];
+            for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
+            return ar;
+        };
+        return ownKeys(o);
+    };
+    return function (mod) {
+        if (mod && mod.__esModule) return mod;
+        var result = {};
+        if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
+        __setModuleDefault(result, mod);
+        return result;
+    };
+})();
 var __importDefault = (this && this.__importDefault) || function (mod) {
     return (mod && mod.__esModule) ? mod : { "default": mod };
 };
@@ -23,12 +56,29 @@ const server_show_1 = require("./commands/server-show");
 const config_1 = require("./commands/config");
 const config_global_1 = require("./commands/config-global");
 const monitor_1 = require("./commands/monitor");
+const start_2 = require("./commands/router/start");
+const stop_2 = require("./commands/router/stop");
+const status_1 = require("./commands/router/status");
+const restart_1 = require("./commands/router/restart");
+const config_2 = require("./commands/router/config");
+const logs_2 = require("./commands/router/logs");
 const package_json_1 = __importDefault(require("../package.json"));
 const program = new commander_1.Command();
 program
     .name('llamacpp')
     .description('CLI tool to manage local llama.cpp servers on macOS')
-    .version(package_json_1.default.version, '-v, --version', 'Output the version number');
+    .version(package_json_1.default.version, '-v, --version', 'Output the version number')
+    .action(async () => {
+    // Default action: launch TUI when no command provided
+    try {
+        const { tuiCommand } = await Promise.resolve().then(() => __importStar(require('./commands/tui')));
+        await tuiCommand();
+    }
+    catch (error) {
+        console.error(chalk_1.default.red('❌ Error:'), error.message);
+        process.exit(1);
+    }
+});
 // List models
 program
     .command('ls')
@@ -42,14 +92,13 @@ program
         process.exit(1);
     }
 });
-// List running servers
+// List servers (static table)
 program
-    .command('ps [identifier]')
-    .description('Interactive server monitoring dashboard')
-    .option('--table', 'Show static table instead of TUI (for scripting)')
-    .action(async (identifier, options) => {
+    .command('ps')
+    .description('List all servers with status (static table)')
+    .action(async () => {
     try {
-        await (0, ps_1.psCommand)(identifier, options);
+        await (0, ps_1.psCommand)();
     }
     catch (error) {
         console.error(chalk_1.default.red('❌ Error:'), error.message);
@@ -293,14 +342,14 @@ server
         process.exit(1);
     }
 });
-// Monitor server (deprecated - redirects to ps)
+// Monitor server (deprecated - redirects to TUI)
 server
     .command('monitor [identifier]')
-    .description('Monitor server with real-time metrics TUI (deprecated: use "llamacpp ps" instead)')
+    .description('Monitor server with real-time metrics TUI (deprecated: use "llamacpp" instead)')
     .action(async (identifier) => {
     try {
         console.log(chalk_1.default.yellow('⚠️  The "monitor" command is deprecated and will be removed in a future version.'));
-        console.log(chalk_1.default.dim('   Please use "llamacpp ps" instead for the same functionality.\n'));
+        console.log(chalk_1.default.dim('   Please use "llamacpp" instead for the same functionality.\n'));
         await (0, monitor_1.monitorCommand)(identifier);
     }
     catch (error) {
@@ -308,6 +357,101 @@ server
         process.exit(1);
     }
 });
+// Router management commands
+const router = program
+    .command('router')
+    .description('Manage the unified router endpoint');
+// Start router
+router
+    .command('start')
+    .description('Start the router service')
+    .action(async () => {
+    try {
+        await (0, start_2.routerStartCommand)();
+    }
+    catch (error) {
+        console.error(chalk_1.default.red('❌ Error:'), error.message);
+        process.exit(1);
+    }
+});
+// Stop router
+router
+    .command('stop')
+    .description('Stop the router service')
+    .action(async () => {
+    try {
+        await (0, stop_2.routerStopCommand)();
+    }
+    catch (error) {
+        console.error(chalk_1.default.red('❌ Error:'), error.message);
+        process.exit(1);
+    }
+});
+// Show router status
+router
+    .command('status')
+    .description('Show router status and configuration')
+    .action(async () => {
+    try {
+        await (0, status_1.routerStatusCommand)();
+    }
+    catch (error) {
+        console.error(chalk_1.default.red('❌ Error:'), error.message);
+        process.exit(1);
+    }
+});
+// Restart router
+router
+    .command('restart')
+    .description('Restart the router service')
+    .action(async () => {
+    try {
+        await (0, restart_1.routerRestartCommand)();
+    }
+    catch (error) {
+        console.error(chalk_1.default.red('❌ Error:'), error.message);
+        process.exit(1);
+    }
+});
+// Configure router
+router
+    .command('config')
+    .description('Update router configuration')
+    .option('-p, --port <number>', 'Update port number', parseInt)
+    .option('-h, --host <address>', 'Update bind address')
+    .option('--timeout <ms>', 'Update request timeout (milliseconds)', parseInt)
+    .option('--health-interval <ms>', 'Update health check interval (milliseconds)', parseInt)
+    .option('-v, --verbose [boolean]', 'Enable/disable verbose logging to file (true/false)', (val) => val === 'true' || val === '1')
+    .option('-r, --restart', 'Automatically restart router if running')
+    .action(async (options) => {
+    try {
+        await (0, config_2.routerConfigCommand)(options);
+    }
+    catch (error) {
+        console.error(chalk_1.default.red('❌ Error:'), error.message);
+        process.exit(1);
+    }
+});
+// Router logs
+router
+    .command('logs')
+    .description('View router logs')
+    .option('-f, --follow', 'Follow logs in real-time (like tail -f)')
+    .option('-n, --lines <number>', 'Number of lines to show (default: 50)', parseInt)
+    .option('--stderr', 'Show system logs (stderr) instead of activity logs (stdout)')
+    .option('-v, --verbose', 'Show verbose JSON log file (if enabled)')
+    .option('--clear', 'Clear the log file')
+    .option('--rotate', 'Rotate the log file with timestamp')
+    .option('--clear-all', 'Clear all router logs (activity, system, verbose)')
+    .action(async (options) => {
+    try {
+        await (0, logs_2.routerLogsCommand)(options);
+    }
+    catch (error) {
+        console.error(chalk_1.default.red('❌ Error:'), error.message);
+        process.exit(1);
+    }
+});
 // Parse arguments
 program.parse();
 //# sourceMappingURL=cli.js.map