@synsci/cli-darwin-x64 1.1.49

package/bin/skills/autogpt/references/troubleshooting.md

@@ -0,0 +1,420 @@
+# AutoGPT Troubleshooting Guide
+
+## Installation Issues
+
+### Docker compose fails
+
+**Error**: `Cannot connect to the Docker daemon`
+
+**Fix**:
+```bash
+# Start Docker daemon
+sudo systemctl start docker
+
+# Or on macOS
+open -a Docker
+
+# Verify Docker is running
+docker ps
+```
+
+**Error**: `Port already in use`
+
+**Fix**:
+```bash
+# Find process using port
+lsof -i :8006
+
+# Kill process
+kill -9 <PID>
+
+# Or change port in docker-compose.yml
+```
+
+### Database migration fails
+
+**Error**: `Migration failed: relation already exists`
+
+**Fix**:
+```bash
+# Reset database
+docker compose down -v
+docker compose up -d db
+
+# Re-run migrations
+cd backend
+poetry run prisma migrate reset --force
+poetry run prisma migrate deploy
+```
+
+**Error**: `Connection refused to database`
+
+**Fix**:
+```bash
+# Check database is running
+docker compose ps db
+
+# Check database logs
+docker compose logs db
+
+# Verify DATABASE_URL in .env
+echo $DATABASE_URL
+```
+
+### Frontend build fails
+
+**Error**: `Module not found: Can't resolve '@/components/...'`
+
+**Fix**:
+```bash
+# Clear node modules and reinstall
+rm -rf node_modules
+rm -rf .next
+npm install
+
+# Or with pnpm
+pnpm install --force
+```
+
+**Error**: `Supabase client not initialized`
+
+**Fix**:
+```bash
+# Verify environment variables
+cat .env | grep SUPABASE
+
+# Required variables:
+# NEXT_PUBLIC_SUPABASE_URL=http://localhost:8000
+# NEXT_PUBLIC_SUPABASE_ANON_KEY=your-key
+```
+
+## Service Issues
+
+### Backend services not starting
+
+**Error**: `rest_server exited with code 1`
+
+**Diagnose**:
+```bash
+# Check logs
+docker compose logs rest_server
+
+# Common issues:
+# - Missing environment variables
+# - Database connection failed
+# - Redis connection failed
+```
+
+**Fix**:
+```bash
+# Verify all dependencies are running
+docker compose ps
+
+# Restart services in order
+docker compose restart db redis rabbitmq
+sleep 10
+docker compose restart rest_server executor
+```
+
+### Executor not processing tasks
+
+**Error**: Tasks stuck in QUEUED status
+
+**Diagnose**:
+```bash
+# Check executor logs
+docker compose logs executor
+
+# Check RabbitMQ queue
+# Visit http://localhost:15672 (guest/guest)
+# Look at queue depths
+```
+
+**Fix**:
+```bash
+# Restart executor
+docker compose restart executor
+
+# If queue is backlogged, scale executors
+docker compose up -d --scale executor=3
+```
+
+### WebSocket connection fails
+
+**Error**: `WebSocket connection to 'ws://localhost:8001/ws' failed`
+
+**Fix**:
+```bash
+# Check WebSocket server is running
+docker compose logs websocket_server
+
+# Verify port is accessible
+nc -zv localhost 8001
+
+# Check firewall rules
+sudo ufw allow 8001
+```
+
+## Agent Execution Issues
+
+### Agent stuck in running state
+
+**Diagnose**:
+```bash
+# Check execution status via API
+curl http://localhost:8006/api/v1/executions/{execution_id}
+
+# Check node execution logs
+docker compose logs executor | grep {execution_id}
+```
+
+**Fix**:
+```python
+# Cancel stuck execution via API
+import requests
+
+response = requests.post(
+    f"http://localhost:8006/api/v1/executions/{execution_id}/cancel",
+    headers={"Authorization": f"Bearer {token}"}
+)
+```
+
+### LLM block timeout
+
+**Error**: `TimeoutError: LLM call exceeded timeout`
+
+**Fix**:
+```python
+# Increase timeout in block configuration
+{
+    "block_id": "llm-block",
+    "config": {
+        "timeout_seconds": 120,  # Increase from default 60
+        "max_retries": 3
+    }
+}
+```
+
+### Credential errors
+
+**Error**: `CredentialsNotFoundError: No credentials for provider openai`
+
+**Fix**:
+1. Navigate to Profile > Integrations
+2. Add OpenAI API key
+3. Ensure graph has credential mapping
+
+```json
+{
+    "credential_mapping": {
+        "openai": "user_credential_id"
+    }
+}
+```
+
+### Memory issues during execution
+
+**Error**: `MemoryError` or container killed (OOMKilled)
+
+**Fix**:
+```yaml
+# Increase memory limits in docker-compose.yml
+executor:
+    deploy:
+        resources:
+            limits:
+                memory: 4G
+            reservations:
+                memory: 2G
+```
+
+## Graph/Block Issues
+
+### Block not appearing in UI
+
+**Diagnose**:
+```python
+# Check block registration
+from backend.data.block import get_all_blocks
+
+blocks = get_all_blocks()
+print([b.name for b in blocks])
+```
+
+**Fix**:
+```python
+# Ensure block is imported in __init__.py
+# backend/blocks/__init__.py
+from backend.blocks.my_block import MyBlock
+
+BLOCKS = [
+    MyBlock,
+    # ...
+]
+```
+
+### Graph save fails
+
+**Error**: `GraphValidationError: Invalid link configuration`
+
+**Diagnose**:
+```python
+# Validate graph structure
+from backend.data.graph import validate_graph
+
+errors = validate_graph(graph_data)
+print(errors)
+```
+
+**Fix**:
+- Ensure all links connect valid nodes
+- Check input/output name matches
+- Verify required inputs are connected
+
+### Circular dependency detected
+
+**Error**: `GraphValidationError: Circular dependency in graph`
+
+**Fix**:
+```python
+# Find cycle
+import networkx as nx
+
+G = nx.DiGraph()
+for link in graph.links:
+    G.add_edge(link.source_id, link.sink_id)
+
+cycles = list(nx.simple_cycles(G))
+print(f"Cycles found: {cycles}")
+```
+
+## Performance Issues
+
+### Slow graph execution
+
+**Diagnose**:
+```python
+# Profile execution
+import cProfile
+
+profiler = cProfile.Profile()
+profiler.enable()
+await executor.execute_graph(graph_id, inputs)
+profiler.disable()
+profiler.print_stats(sort='cumulative')
+```
+
+**Fix**:
+- Parallelize independent nodes
+- Reduce unnecessary API calls
+- Cache repeated computations
+
+### High database query latency
+
+**Diagnose**:
+```bash
+# Enable query logging in PostgreSQL
+docker exec -it autogpt-db psql -U postgres
+\x
+SHOW log_min_duration_statement;
+SET log_min_duration_statement = 100;  -- Log queries > 100ms
+```
+
+**Fix**:
+```sql
+-- Add missing indexes
+CREATE INDEX CONCURRENTLY idx_executions_user_created
+ON "AgentGraphExecution" ("userId", "createdAt" DESC);
+
+ANALYZE "AgentGraphExecution";
+```
+
+### Redis memory growing
+
+**Diagnose**:
+```bash
+# Check Redis memory usage
+docker exec -it autogpt-redis redis-cli INFO memory
+
+# Check key count
+docker exec -it autogpt-redis redis-cli DBSIZE
+```
+
+**Fix**:
+```bash
+# Clear expired keys
+docker exec -it autogpt-redis redis-cli --scan --pattern "exec:*" | head -1000 | xargs docker exec -i autogpt-redis redis-cli DEL
+
+# Set memory policy
+docker exec -it autogpt-redis redis-cli CONFIG SET maxmemory-policy volatile-lru
+```
+
+## Debugging Tips
+
+### Enable debug logging
+
+```bash
+# Set in .env
+LOG_LEVEL=DEBUG
+
+# Or for specific module
+LOG_LEVEL_EXECUTOR=DEBUG
+LOG_LEVEL_BLOCKS=DEBUG
+```
+
+### Trace execution flow
+
+```python
+import logging
+
+logging.basicConfig(level=logging.DEBUG)
+logger = logging.getLogger("backend.executor")
+
+# Add to executor
+logger.debug(f"Executing node {node_id} with inputs: {inputs}")
+```
+
+### Test block in isolation
+
+```python
+import asyncio
+from backend.blocks.my_block import MyBlock
+
+async def test_block():
+    block = MyBlock()
+    inputs = {"query": "test"}
+
+    async for output_name, value in block.execute(inputs):
+        print(f"{output_name}: {value}")
+
+asyncio.run(test_block())
+```
+
+### Inspect message queues
+
+```bash
+# RabbitMQ management UI
+# http://localhost:15672 (guest/guest)
+
+# List queues via CLI
+docker exec autogpt-rabbitmq rabbitmqctl list_queues name messages consumers
+
+# Purge a queue
+docker exec autogpt-rabbitmq rabbitmqctl purge_queue graph-execution
+```
+
+## Getting Help
+
+1. **Documentation**: https://docs.agpt.co
+2. **GitHub Issues**: https://github.com/Significant-Gravitas/AutoGPT/issues
+3. **Discord**: https://discord.gg/autogpt
+
+### Reporting Issues
+
+Include:
+- AutoGPT version: `git describe --tags`
+- Docker version: `docker --version`
+- Error logs: `docker compose logs > logs.txt`
+- Steps to reproduce
+- Graph configuration (sanitized)
+- Environment: OS, hardware specs

package/bin/skills/awq/SKILL.md

@@ -0,0 +1,310 @@
+---
+name: awq-quantization
+description: Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.
+version: 1.0.0
+author: Synthetic Sciences
+license: MIT
+tags: [Optimization, AWQ, Quantization, 4-Bit, Activation-Aware, Memory Optimization, Fast Inference, vLLM Integration, Marlin Kernels]
+dependencies: [autoawq, transformers>=4.45.0, torch>=2.0.0]
+---
+
+# AWQ (Activation-aware Weight Quantization)
+
+4-bit quantization that preserves salient weights based on activation patterns, achieving 3x speedup with minimal accuracy loss.
+
+## When to use AWQ
+
+**Use AWQ when:**
+- Need 4-bit quantization with <5% accuracy loss
+- Deploying instruction-tuned or chat models (AWQ generalizes better)
+- Want ~2.5-3x inference speedup over FP16
+- Using vLLM for production serving
+- Have Ampere+ GPUs (A100, H100, RTX 40xx) for Marlin kernel support
+
+**Use GPTQ instead when:**
+- Need maximum ecosystem compatibility (more tools support GPTQ)
+- Working with ExLlamaV2 backend specifically
+- Have older GPUs without Marlin support
+
+**Use bitsandbytes instead when:**
+- Need zero calibration overhead (quantize on-the-fly)
+- Want to fine-tune with QLoRA
+- Prefer simpler integration
+
+## Quick start
+
+### Installation
+
+```bash
+# Default (Triton kernels)
+pip install autoawq
+
+# With optimized CUDA kernels + Flash Attention
+pip install autoawq[kernels]
+
+# Intel CPU/XPU optimization
+pip install autoawq[cpu]
+```
+
+**Requirements**: Python 3.8+, CUDA 11.8+, Compute Capability 7.5+
+
+### Load pre-quantized model
+
+```python
+from awq import AutoAWQForCausalLM
+from transformers import AutoTokenizer
+
+model_name = "TheBloke/Mistral-7B-Instruct-v0.2-AWQ"
+
+model = AutoAWQForCausalLM.from_quantized(
+    model_name,
+    fuse_layers=True  # Enable fused attention for speed
+)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+# Generate
+inputs = tokenizer("Explain quantum computing", return_tensors="pt").to("cuda")
+outputs = model.generate(**inputs, max_new_tokens=200)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+### Quantize your own model
+
+```python
+from awq import AutoAWQForCausalLM
+from transformers import AutoTokenizer
+
+model_path = "mistralai/Mistral-7B-Instruct-v0.2"
+
+# Load model and tokenizer
+model = AutoAWQForCausalLM.from_pretrained(model_path)
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+
+# Quantization config
+quant_config = {
+    "zero_point": True,      # Use zero-point quantization
+    "q_group_size": 128,     # Group size (128 recommended)
+    "w_bit": 4,              # 4-bit weights
+    "version": "GEMM"        # GEMM for batch, GEMV for single-token
+}
+
+# Quantize (uses pileval dataset by default)
+model.quantize(tokenizer, quant_config=quant_config)
+
+# Save
+model.save_quantized("mistral-7b-awq")
+tokenizer.save_pretrained("mistral-7b-awq")
+```
+
+**Timing**: ~10-15 min for 7B, ~1 hour for 70B models.
+
+## AWQ vs GPTQ vs bitsandbytes
+
+| Feature | AWQ | GPTQ | bitsandbytes |
+|---------|-----|------|--------------|
+| **Speedup (4-bit)** | ~2.5-3x | ~2x | ~1.5x |
+| **Accuracy loss** | <5% | ~5-10% | ~5-15% |
+| **Calibration** | Minimal (128-1K tokens) | More extensive | None |
+| **Overfitting risk** | Low | Higher | N/A |
+| **Best for** | Production inference | GPU inference | Easy integration |
+| **vLLM support** | Native | Yes | Limited |
+
+**Key insight**: AWQ assumes not all weights are equally important. It protects ~1% of salient weights identified by activation patterns, reducing quantization error without mixed-precision overhead.
+
+## Kernel backends
+
+### GEMM (default, batch inference)
+
+```python
+quant_config = {
+    "zero_point": True,
+    "q_group_size": 128,
+    "w_bit": 4,
+    "version": "GEMM"  # Best for batch sizes > 1
+}
+```
+
+### GEMV (single-token generation)
+
+```python
+quant_config = {
+    "version": "GEMV"  # 20% faster for batch_size=1
+}
+```
+
+**Limitation**: Only batch size 1, not good for large context.
+
+### Marlin (Ampere+ GPUs)
+
+```python
+from transformers import AwqConfig, AutoModelForCausalLM
+
+config = AwqConfig(
+    bits=4,
+    version="marlin"  # 2x faster on A100/H100
+)
+
+model = AutoModelForCausalLM.from_pretrained(
+    "TheBloke/Mistral-7B-AWQ",
+    quantization_config=config
+)
+```
+
+**Requirements**: Compute Capability 8.0+ (A100, H100, RTX 40xx)
+
+### ExLlamaV2 (AMD compatible)
+
+```python
+config = AwqConfig(
+    bits=4,
+    version="exllama"  # Faster prefill, AMD GPU support
+)
+```
+
+## HuggingFace Transformers integration
+
+### Direct loading
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained(
+    "TheBloke/zephyr-7B-alpha-AWQ",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained("TheBloke/zephyr-7B-alpha-AWQ")
+```
+
+### Fused modules (recommended)
+
+```python
+from transformers import AwqConfig, AutoModelForCausalLM
+
+config = AwqConfig(
+    bits=4,
+    fuse_max_seq_len=512,  # Max sequence length for fusing
+    do_fuse=True           # Enable fused attention/MLP
+)
+
+model = AutoModelForCausalLM.from_pretrained(
+    "TheBloke/Mistral-7B-OpenOrca-AWQ",
+    quantization_config=config
+)
+```
+
+**Note**: Fused modules cannot combine with FlashAttention2.
+
+## vLLM integration
+
+```python
+from vllm import LLM, SamplingParams
+
+# vLLM auto-detects AWQ models
+llm = LLM(
+    model="TheBloke/Llama-2-7B-AWQ",
+    quantization="awq",
+    dtype="half"
+)
+
+sampling = SamplingParams(temperature=0.7, max_tokens=200)
+outputs = llm.generate(["Explain AI"], sampling)
+```
+
+## Performance benchmarks
+
+### Memory reduction
+
+| Model | FP16 | AWQ 4-bit | Reduction |
+|-------|------|-----------|-----------|
+| Mistral 7B | 14 GB | 5.5 GB | 2.5x |
+| Llama 2-13B | 26 GB | 10 GB | 2.6x |
+| Llama 2-70B | 140 GB | 35 GB | 4x |
+
+### Inference speed (RTX 4090)
+
+| Model | Prefill (tok/s) | Decode (tok/s) | Memory |
+|-------|-----------------|----------------|--------|
+| Mistral 7B GEMM | 3,897 | 114 | 5.55 GB |
+| TinyLlama 1B GEMV | 5,179 | 431 | 2.10 GB |
+| Llama 2-13B GEMM | 2,279 | 74 | 10.28 GB |
+
+### Accuracy (perplexity)
+
+| Model | FP16 | AWQ 4-bit | Degradation |
+|-------|------|-----------|-------------|
+| Llama 3 8B | 8.20 | 8.48 | +3.4% |
+| Mistral 7B | 5.25 | 5.42 | +3.2% |
+| Qwen2 72B | 4.85 | 4.95 | +2.1% |
+
+## Custom calibration data
+
+```python
+# Use custom dataset for domain-specific models
+model.quantize(
+    tokenizer,
+    quant_config=quant_config,
+    calib_data="wikitext",       # Or custom list of strings
+    max_calib_samples=256,       # More samples = better accuracy
+    max_calib_seq_len=512        # Sequence length
+)
+
+# Or provide your own samples
+calib_samples = [
+    "Your domain-specific text here...",
+    "More examples from your use case...",
+]
+model.quantize(tokenizer, quant_config=quant_config, calib_data=calib_samples)
+```
+
+## Multi-GPU deployment
+
+```python
+model = AutoAWQForCausalLM.from_quantized(
+    "TheBloke/Llama-2-70B-AWQ",
+    device_map="auto",  # Auto-split across GPUs
+    max_memory={0: "40GB", 1: "40GB"}
+)
+```
+
+## Supported models
+
+35+ architectures including:
+- **Llama family**: Llama 2/3, Code Llama, Mistral, Mixtral
+- **Qwen**: Qwen, Qwen2, Qwen2.5-VL
+- **Others**: Falcon, MPT, Phi, Yi, DeepSeek, Gemma
+- **Multimodal**: LLaVA, LLaVA-Next, Qwen2-VL
+
+## Common issues
+
+**CUDA OOM during quantization**:
+```python
+# Reduce batch size
+model.quantize(tokenizer, quant_config=quant_config, max_calib_samples=64)
+```
+
+**Slow inference**:
+```python
+# Enable fused layers
+model = AutoAWQForCausalLM.from_quantized(model_name, fuse_layers=True)
+```
+
+**AMD GPU support**:
+```python
+# Use ExLlama backend
+config = AwqConfig(bits=4, version="exllama")
+```
+
+## Deprecation notice
+
+AutoAWQ is officially deprecated. For new projects, consider:
+- **vLLM llm-compressor**: https://github.com/vllm-project/llm-compressor
+- **MLX-LM**: For Mac devices with Apple Silicon
+
+Existing quantized models remain usable.
+
+## References
+
+- **Paper**: AWQ: Activation-aware Weight Quantization (arXiv:2306.00978) - MLSys 2024 Best Paper
+- **GitHub**: https://github.com/casper-hansen/AutoAWQ
+- **MIT Han Lab**: https://github.com/mit-han-lab/llm-awq
+- **Models**: https://huggingface.co/models?library=awq