@ceschiatti/redistail 0.0.2 → 0.0.4

package/README.md

@@ -0,0 +1,680 @@
+A command-line utility for monitoring Redis messages in real-time, similar to how `tail -f` works for log files. Redistail supports both Redis Pub/Sub channels and Redis Streams, making it an essential tool for debugging and monitoring Redis-based applications.
+
+Originally part of the `effect-redis` library, it is now available as a standalone package for easier installation and use.
+
+## Table of Contents
+
+- [Installation](#installation)
+- [Quick Start](#quick-start)
+- [Usage](#usage)
+- [Configuration](#configuration)
+- [Examples](#examples)
+- [Environment Variables](#environment-variables)
+- [Output Format](#output-format)
+- [Troubleshooting](#troubleshooting)
+- [Advanced Usage](#advanced-usage)
+
+## Installation
+
+### Global Installation
+
+Install `redistail` globally to make the `redistail` command available everywhere on your system:
+
+```bash
+# Using npm
+npm install -g @ceschiatti/redistail
+
+# Using pnpm
+pnpm add -g @ceschiatti/redistail
+
+# Using bun
+bun add -g @ceschiatti/redistail
+```
+
+### Local Installation
+
+If you're working within a project:
+
+```bash
+# Using npm
+npm install @ceschiatti/redistail
+
+# Using pnpm
+pnpm add @ceschiatti/redistail
+
+# Using bun
+bun add @ceschiatti/redistail
+
+# Run locally using npx
+npx redistail --help
+```
+
+### Verify Installation
+
+```bash
+redistail --version
+redistail --help
+```
+
+## Quick Start
+
+### Monitor a Pub/Sub Channel
+
+```bash
+redistail pubsub my-channel
+```
+
+### Monitor a Redis Stream
+
+```bash
+redistail stream my-stream
+```
+
+### With Custom Redis Connection
+
+```bash
+REDIS_HOST=redis.example.com redistail pubsub notifications
+```
+
+## Usage
+
+### Basic Syntax
+
+```bash
+redistail <CONNECTION_TYPE> <TOPIC_NAME> [OPTIONS]
+```
+
+### Arguments
+
+- **CONNECTION_TYPE**: Either `pubsub` or `stream`
+- **TOPIC_NAME**: The channel name (for Pub/Sub) or stream key (for Streams)
+
+### Options
+
+- `-h, --help`: Show help message
+- `-v, --version`: Show version information
+
+### Connection Types
+
+#### Pub/Sub Monitoring
+
+Monitor Redis Pub/Sub channels for real-time message broadcasting:
+
+```bash
+redistail pubsub channel-name
+```
+
+- Subscribes to the specified channel
+- Displays messages as they are published
+- Shows timestamp, channel name, and message content
+- Continues running until interrupted (Ctrl+C)
+
+#### Stream Monitoring
+
+Monitor Redis Streams for persistent message queues:
+
+```bash
+redistail stream stream-name
+```
+
+- Reads from the stream starting from the latest entry
+- Uses blocking XREAD to wait for new entries
+- Shows timestamp, entry ID, and all message fields
+- Maintains continuity across reconnections
+
+## Configuration
+
+Redistail can be configured through environment variables to work with different Redis instances and customize display options.
+
+### Redis Connection
+
+Configure Redis connection settings:
+
+```bash
+# Basic connection
+export REDIS_HOST=localhost
+export REDIS_PORT=6379
+
+# Or use a complete connection URL
+export REDIS_URL=redis://username:password@redis.example.com:6380
+
+# Connection timeouts and retries
+export REDIS_TIMEOUT=5000
+export REDIS_RETRY_ATTEMPTS=3
+export REDIS_RETRY_DELAY=1000
+```
+
+### Display Options
+
+Customize output formatting:
+
+```bash
+# Enable/disable colored output
+export REDISTAIL_COLORS=true
+
+# Enable/disable timestamps
+export REDISTAIL_TIMESTAMPS=true
+
+# Enable/disable JSON pretty-printing
+export REDISTAIL_PRETTY_JSON=true
+```
+
+### Monitoring Options
+
+Configure monitoring behavior:
+
+```bash
+# Blocking timeout for stream reads (milliseconds)
+export REDISTAIL_BLOCK_TIMEOUT=5000
+
+# Maximum reconnection attempts
+export REDISTAIL_MAX_RECONNECT_ATTEMPTS=5
+```
+
+## Examples
+
+### Basic Pub/Sub Monitoring
+
+Monitor a simple notification channel:
+
+```bash
+redistail pubsub notifications
+```
+
+Output:
+```
+2024-01-07 15:30:45.123 [notifications] Welcome to the system!
+2024-01-07 15:30:47.456 [notifications] New user registered: john@example.com
+2024-01-07 15:30:50.789 [notifications] System maintenance scheduled for tonight
+```
+
+### Stream Monitoring with JSON Messages
+
+Monitor a stream containing structured data:
+
+```bash
+redistail stream user-events
+```
+
+Output:
+```
+2024-01-07 15:31:12.345 [user-events:1704636672345-0] event=login user=alice timestamp=1704636672345
+2024-01-07 15:31:15.678 [user-events:1704636675678-0] event=purchase user=bob amount=29.99 product=premium-plan
+2024-01-07 15:31:18.901 [user-events:1704636678901-0] event=logout user=alice session_duration=300
+```
+
+### Remote Redis Instance
+
+Connect to a remote Redis server:
+
+```bash
+REDIS_HOST=redis.production.com REDIS_PORT=6380 redistail pubsub alerts
+```
+
+### Authenticated Redis Connection
+
+Use Redis with authentication:
+
+```bash
+REDIS_URL=redis://myuser:mypassword@redis.example.com:6379 redistail stream orders
+```
+
+### Custom Display Settings
+
+Monitor with custom formatting:
+
+```bash
+# Disable colors and timestamps for clean output
+REDISTAIL_COLORS=false REDISTAIL_TIMESTAMPS=false redistail pubsub logs
+```
+
+### JSON Message Pretty-Printing
+
+When messages contain JSON, redistail automatically formats them:
+
+```bash
+redistail pubsub api-events
+```
+
+Input message: `{"user":"alice","action":"login","timestamp":1704636672345}`
+
+Output:
+```
+2024-01-07 15:31:12.345 [api-events] {
+  "user": "alice",
+  "action": "login",
+  "timestamp": 1704636672345
+}
+```
+
+### Multiple Instances
+
+Run multiple redistail instances to monitor different channels:
+
+```bash
+# Terminal 1: Monitor user events
+redistail stream user-events
+
+# Terminal 2: Monitor system notifications
+redistail pubsub system-notifications
+
+# Terminal 3: Monitor error logs
+redistail pubsub error-logs
+```
+
+### Development Workflow
+
+Common development patterns:
+
+```bash
+# Monitor application logs during development
+redistail pubsub app-logs &
+
+# Monitor user activity in another terminal
+redistail stream user-activity &
+
+# Start your application
+npm run dev
+
+# Stop monitoring (kill background processes)
+jobs
+kill %1 %2
+```
+
+## Environment Variables
+
+### Redis Connection Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `REDIS_HOST` | `127.0.0.1` | Redis server hostname |
+| `REDIS_PORT` | `6379` | Redis server port |
+| `REDIS_URL` | - | Complete Redis connection URL (overrides host/port) |
+| `REDIS_TIMEOUT` | `5000` | Connection timeout in milliseconds |
+| `REDIS_RETRY_ATTEMPTS` | `3` | Number of connection retry attempts |
+| `REDIS_RETRY_DELAY` | `1000` | Delay between retry attempts in milliseconds |
+
+### Display Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `REDISTAIL_COLORS` | `true` | Enable colored output |
+| `REDISTAIL_TIMESTAMPS` | `true` | Show timestamps in output |
+| `REDISTAIL_PRETTY_JSON` | `true` | Pretty-print JSON content |
+
+### Monitoring Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `REDISTAIL_BLOCK_TIMEOUT` | `5000` | Stream blocking timeout in milliseconds |
+| `REDISTAIL_MAX_RECONNECT_ATTEMPTS` | `5` | Maximum reconnection attempts |
+
+### Configuration Examples
+
+#### Production Environment
+
+```bash
+# .env.production
+REDIS_URL=redis://prod-user:secure-password@redis-cluster.prod.com:6380
+REDISTAIL_COLORS=false
+REDISTAIL_TIMESTAMPS=true
+REDISTAIL_MAX_RECONNECT_ATTEMPTS=10
+```
+
+#### Development Environment
+
+```bash
+# .env.development
+REDIS_HOST=localhost
+REDIS_PORT=6379
+REDISTAIL_COLORS=true
+REDISTAIL_PRETTY_JSON=true
+REDISTAIL_BLOCK_TIMEOUT=1000
+```
+
+#### CI/CD Environment
+
+```bash
+# Minimal output for automated systems
+REDISTAIL_COLORS=false
+REDISTAIL_TIMESTAMPS=false
+REDISTAIL_PRETTY_JSON=false
+```
+
+## Output Format
+
+### Pub/Sub Messages
+
+```
+[TIMESTAMP] [CHANNEL] MESSAGE_CONTENT
+```
+
+Example:
+```
+2024-01-07 15:30:45.123 [notifications] User alice logged in
+```
+
+### Stream Messages
+
+```
+[TIMESTAMP] [STREAM_KEY:ENTRY_ID] FIELD1=VALUE1 FIELD2=VALUE2 ...
+```
+
+Example:
+```
+2024-01-07 15:31:12.345 [events:1704636672345-0] user=alice action=login ip=192.168.1.100
+```
+
+### Color Coding
+
+When colors are enabled (`REDISTAIL_COLORS=true`):
+
+- **Gray**: Timestamps
+- **Cyan**: Pub/Sub channel names
+- **Magenta**: Stream names and entry IDs
+- **Yellow**: Field names in streams
+- **White**: Message content and field values
+- **Red**: Error messages
+
+### Error Messages
+
+Errors are displayed on stderr with clear descriptions:
+
+```
+❌ Error: Connection failed to redis://localhost:6379
+❌ Error: Invalid topic name: cannot be empty
+❌ Error: Maximum retry attempts (5) exceeded
+```
+
+## Troubleshooting
+
+### Common Issues
+
+#### Connection Refused
+
+**Problem**: `Error: Connection failed to redis://localhost:6379`
+
+**Solutions**:
+1. Verify Redis is running: `redis-cli ping`
+2. Check Redis configuration: `redis-cli info server`
+3. Verify host and port: `REDIS_HOST=localhost REDIS_PORT=6379 redistail --help`
+4. Test with redis-cli: `redis-cli -h localhost -p 6379 ping`
+
+#### Authentication Failed
+
+**Problem**: `Error: Authentication failed`
+
+**Solutions**:
+1. Include credentials in URL: `REDIS_URL=redis://user:pass@host:port`
+2. Check Redis AUTH configuration: `redis-cli config get requirepass`
+3. Verify username/password are correct
+
+#### No Messages Appearing
+
+**Problem**: Redistail connects but shows no messages
+
+**Solutions**:
+1. Verify the channel/stream exists:
+   ```bash
+   # For Pub/Sub
+   redis-cli publish test-channel "test message"
+   
+   # For Streams
+   redis-cli xadd test-stream * field value
+   ```
+2. Check if you're monitoring the correct topic name
+3. Ensure messages are being published to the topic
+4. For streams, verify entries exist: `redis-cli xlen stream-name`
+
+#### Permission Denied
+
+**Problem**: `Error: Permission denied for command`
+
+**Solutions**:
+1. Check Redis ACL permissions: `redis-cli acl whoami`
+2. Verify user has subscribe permissions: `redis-cli acl getuser username`
+3. Use a user with appropriate permissions
+
+#### High Memory Usage
+
+**Problem**: Redistail consuming too much memory
+
+**Solutions**:
+1. Reduce block timeout: `REDISTAIL_BLOCK_TIMEOUT=1000`
+2. Monitor smaller streams or channels
+3. Disable JSON pretty-printing: `REDISTAIL_PRETTY_JSON=false`
+4. Use multiple instances for different topics instead of one large stream
+
+### Network Issues
+
+#### Intermittent Disconnections
+
+**Problem**: Frequent connection drops
+
+**Solutions**:
+1. Increase retry attempts: `REDISTAIL_MAX_RECONNECT_ATTEMPTS=10`
+2. Adjust retry delay: `REDIS_RETRY_DELAY=2000`
+3. Check network stability between client and Redis server
+4. Monitor Redis server logs for connection issues
+
+#### Slow Performance
+
+**Problem**: Messages appear with delay
+
+**Solutions**:
+1. Reduce block timeout: `REDISTAIL_BLOCK_TIMEOUT=1000`
+2. Check Redis server performance: `redis-cli --latency`
+3. Monitor network latency: `ping redis-host`
+4. Verify Redis server isn't overloaded: `redis-cli info stats`
+
+### Configuration Issues
+
+#### Environment Variables Not Working
+
+**Problem**: Environment variables seem to be ignored
+
+**Solutions**:
+1. Verify variable names are correct (case-sensitive)
+2. Export variables in current shell: `export REDIS_HOST=myhost`
+3. Check variable values: `echo $REDIS_HOST`
+4. Use inline variables: `REDIS_HOST=myhost redistail pubsub test`
+
+#### Colors Not Displaying
+
+**Problem**: Output appears without colors
+
+**Solutions**:
+1. Enable colors explicitly: `REDISTAIL_COLORS=true`
+2. Check terminal color support: `echo $TERM`
+3. Test with a color-supporting terminal
+4. Verify output isn't being piped: `redistail pubsub test | cat` (disables colors)
+
+### Debugging Steps
+
+#### Enable Verbose Logging
+
+While redistail doesn't have a verbose mode, you can debug connection issues:
+
+```bash
+# Test Redis connection manually
+redis-cli -h $REDIS_HOST -p $REDIS_PORT ping
+
+# Monitor Redis server logs
+redis-cli monitor
+
+# Check Redis configuration
+redis-cli config get "*"
+```
+
+#### Test with Simple Commands
+
+```bash
+# Test basic connectivity
+redis-cli ping
+
+# Test pub/sub manually
+redis-cli subscribe test-channel
+# In another terminal:
+redis-cli publish test-channel "hello"
+
+# Test streams manually
+redis-cli xadd test-stream * message "hello"
+redis-cli xread STREAMS test-stream 0
+```
+
+#### Check System Resources
+
+```bash
+# Check available memory
+free -h
+
+# Check network connectivity
+netstat -an | grep 6379
+
+# Check process limits
+ulimit -a
+```
+
+### Getting Help
+
+If you encounter issues not covered here:
+
+1. Check the [GitHub Issues](https://github.com/6qat/monorepo/issues)
+2. Review Redis server logs
+3. Test with the standard `redis-cli` tool first
+4. Create a minimal reproduction case
+5. Include environment details (OS, Redis version, Node.js version)
+
+### Performance Tips
+
+#### Optimize for High-Volume Streams
+
+```bash
+# Reduce output overhead
+REDISTAIL_COLORS=false REDISTAIL_TIMESTAMPS=false redistail stream high-volume
+
+# Use shorter block timeouts
+REDISTAIL_BLOCK_TIMEOUT=100 redistail stream fast-stream
+
+# Monitor specific time ranges (use redis-cli for historical data)
+redis-cli xrange stream-name - + COUNT 100
+```
+
+#### Monitor Multiple Topics Efficiently
+
+```bash
+# Use separate processes for different priorities
+redistail pubsub critical-alerts &
+redistail pubsub info-logs > /dev/null &  # Discard low-priority output
+```
+
+## Advanced Usage
+
+### Integration with Other Tools
+
+#### Log Aggregation
+
+```bash
+# Send output to syslog
+redistail pubsub app-logs | logger -t redistail
+
+# Append to log files with timestamps
+redistail stream events >> /var/log/redis-events.log
+
+# Filter and process messages
+redistail pubsub notifications | grep "ERROR" | mail -s "Redis Errors" admin@example.com
+```
+
+#### Monitoring and Alerting
+
+```bash
+# Count messages per minute
+redistail pubsub events | pv -l -i 60 > /dev/null
+
+# Alert on specific patterns
+redistail stream errors | grep "CRITICAL" | while read line; do
+  echo "Critical error detected: $line" | mail -s "ALERT" ops@example.com
+done
+
+# Integration with monitoring systems
+redistail pubsub metrics | while read metric; do
+  curl -X POST http://monitoring-system/api/metrics -d "$metric"
+done
+```
+
+#### Development Workflows
+
+```bash
+# Auto-restart on configuration changes
+while inotifywait -e modify redis.conf; do
+  pkill redistail
+  redistail pubsub app-logs &
+done
+
+# Conditional monitoring based on environment
+if [ "$NODE_ENV" = "development" ]; then
+  REDISTAIL_COLORS=true redistail stream debug-events
+else
+  REDISTAIL_COLORS=false redistail stream production-events | logger
+fi
+```
+
+### Scripting Examples
+
+#### Bash Script for Multiple Channels
+
+```bash
+#!/bin/bash
+# monitor-all.sh
+
+CHANNELS=("user-events" "system-logs" "error-alerts")
+PIDS=()
+
+# Start monitoring each channel
+for channel in "${CHANNELS[@]}"; do
+  redistail pubsub "$channel" > "/var/log/redis-$channel.log" &
+  PIDS+=($!)
+  echo "Started monitoring $channel (PID: $!)"
+done
+
+# Wait for interrupt
+trap 'kill "${PIDS[@]}"; exit' INT TERM
+
+# Keep script running
+wait
+```
+
+#### Python Integration
+
+```python
+#!/usr/bin/env python3
+import subprocess
+import json
+import sys
+
+def monitor_json_stream(stream_name):
+    """Monitor a Redis stream and parse JSON messages."""
+    cmd = ['redistail', 'stream', stream_name]
+    env = {'REDISTAIL_COLORS': 'false', 'REDISTAIL_TIMESTAMPS': 'false'}
+    
+    with subprocess.Popen(cmd, stdout=subprocess.PIPE, 
+                         stderr=subprocess.PIPE, text=True, env=env) as proc:
+        try:
+            for line in proc.stdout:
+                try:
+                    # Parse redistail output and extract JSON
+                    parts = line.strip().split(' ', 2)
+                    if len(parts) >= 3:
+                        message_data = parts[2]
+                        # Process the message
+                        print(f"Processed: {message_data}")
+                except Exception as e:
+                    print(f"Error processing line: {e}", file=sys.stderr)
+        except KeyboardInterrupt:
+            proc.terminate()
+
+if __name__ == "__main__":
+    monitor_json_stream("api-events")
+```
+
+This comprehensive documentation covers all aspects of using the redistail CLI tool, from basic usage to advanced integration scenarios, troubleshooting common issues, and providing practical examples for different use cases.