terradev-cli 3.7.1__tar.gz → 3.7.3__tar.gz

{terradev_cli-3.7.1 → terradev_cli-3.7.3}/PKG-INFO

@@ -1,49 +1,24 @@
 Metadata-Version: 2.4
 Name: terradev-cli
-Version: 3.7.1
-Summary: Cross-cloud GPU provisioning with automatic CUDA Graph optimization, NUMA-aware topology optimization, vLLM auto-optimization, and Terraform-powered parallel provisioning
-Author-email: Terradev Team <team@terradev.com>
-Maintainer-email: Terradev Team <team@terradev.com>
-License: BUSL-1.1
-Project-URL: Homepage, https://terradev.cloud
-Project-URL: Documentation, https://docs.terradev.com
-Project-URL: Repository, https://github.com/theoddden/Terradev
-Project-URL: Bug Tracker, https://github.com/theoddden/Terradev/issues
-Project-URL: Changelog, https://github.com/theoddden/Terradev/blob/main/CHANGELOG.md
-Project-URL: Analytics, https://api.terradev.cloud/dashboard
-Keywords: cloud,provisioning,gpu,multi-cloud,arbitrage,aws,gcp,azure,runpod,vastai,machine-learning,cost-optimization,parallel,gitops,argocd,flux,kubernetes,infrastructure-as-code,langchain,langgraph,sglang,wandb,mlflow,analytics,setup,conversion,telemetry,real-time,monitoring,dashboard,huggingface,huggingface-spaces,gradio,streamlit,llm,transformers,pytorch,sentence-transformers,smart-templates,hardware-optimization,model-deployment,inference,training,byoapi,terraform,kubernetes,helm,karpenter,opa,prometheus,grafana,ray,vllm,ollama,dvc,data-version-control,numa,disaggregated-inference,moe,mixture-of-experts,qwen,distserve,xcd,mi300x,h200,prefill-decode,cuda-graph,cuda-graphs,numa-topology,gpu-optimization,passive-optimization,warm-pool,endpoint-scoring
-Classifier: Development Status :: 5 - Production/Stable
+Version: 3.7.3
+Summary: NUMA-aware GPU provisioning and orchestration for stateless MoE workloads of all sizes
+Home-page: https://github.com/theoddden/terradev
+Author: Terradev Team
+Author-email: team@terradev.com
+Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Developers
-Classifier: Intended Audience :: System Administrators
-Classifier: Topic :: Software Development :: Libraries :: Python Modules
-Classifier: Topic :: System :: Systems Administration
-Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
-Classifier: License :: Other/Proprietary License
+Classifier: License :: OSI Approved :: MIT License
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
-Classifier: Operating System :: OS Independent
-Classifier: Environment :: Console
-Classifier: Natural Language :: English
-Requires-Python: >=3.9
+Requires-Python: >=3.8
 Description-Content-Type: text/markdown
-License-File: LICENSE
 Requires-Dist: click>=8.0.0
 Requires-Dist: aiohttp>=3.9.0
 Requires-Dist: pyyaml>=6.0
-Requires-Dist: cryptography>=41.0.0
 Requires-Dist: requests>=2.31.0
-Requires-Dist: gradio>=4.0.0
-Requires-Dist: streamlit>=1.28.0
-Requires-Dist: transformers>=4.0.0
-Requires-Dist: torch>=2.0.0
-Requires-Dist: sentence-transformers>=2.0.0
-Requires-Dist: plotly>=5.0.0
-Requires-Dist: pandas>=1.0.0
-Requires-Dist: numpy>=1.0.0
-Requires-Dist: huggingface-hub>=0.19.0
 Requires-Dist: stripe>=7.0.0
 Provides-Extra: aws
 Requires-Dist: boto3>=1.34.0; extra == "aws"
@@ -52,49 +27,65 @@ Requires-Dist: google-cloud-compute>=1.8.0; extra == "gcp"
 Provides-Extra: azure
 Requires-Dist: azure-mgmt-compute>=29.0.0; extra == "azure"
 Requires-Dist: azure-identity>=1.12.0; extra == "azure"
-Provides-Extra: lmcache
-Requires-Dist: lmcache>=0.1; extra == "lmcache"
-Requires-Dist: redis>=5.0.0; extra == "lmcache"
+Provides-Extra: oracle
+Requires-Dist: oci>=2.118.0; extra == "oracle"
+Provides-Extra: hf
+Requires-Dist: huggingface-hub>=0.19.0; extra == "hf"
 Provides-Extra: all
 Requires-Dist: boto3>=1.34.0; extra == "all"
 Requires-Dist: google-cloud-compute>=1.8.0; extra == "all"
 Requires-Dist: azure-mgmt-compute>=29.0.0; extra == "all"
 Requires-Dist: azure-identity>=1.12.0; extra == "all"
-Requires-Dist: langchain>=0.1.0; extra == "all"
-Requires-Dist: langgraph>=0.0.20; extra == "all"
-Requires-Dist: sglang>=0.3.0; extra == "all"
-Requires-Dist: wandb>=0.16.0; extra == "all"
-Requires-Dist: mlflow>=2.8.0; extra == "all"
+Requires-Dist: oci>=2.118.0; extra == "all"
 Requires-Dist: huggingface-hub>=0.19.0; extra == "all"
-Requires-Dist: prometheus-client>=0.17.0; extra == "all"
-Requires-Dist: grafana-api>=1.0.3; extra == "all"
-Requires-Dist: lmcache>=0.1; extra == "all"
-Requires-Dist: redis>=5.0.0; extra == "all"
-Provides-Extra: hf
-Requires-Dist: aiohttp>=3.9.0; extra == "hf"
-Requires-Dist: huggingface-hub>=0.19.0; extra == "hf"
-Requires-Dist: gradio>=4.0.0; extra == "hf"
-Requires-Dist: streamlit>=1.28.0; extra == "hf"
-Provides-Extra: dev
-Requires-Dist: pytest>=7.0.0; extra == "dev"
-Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
-Requires-Dist: black>=22.0.0; extra == "dev"
-Requires-Dist: flake8>=5.0.0; extra == "dev"
-Requires-Dist: build>=0.10.0; extra == "dev"
-Requires-Dist: twine>=4.0.0; extra == "dev"
-Dynamic: license-file
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# Terradev CLI v3.7.3 - Enterprise GPU Infrastructure
+
+**Enterprise-grade GPU provisioning and orchestration with adaptive scaling for 500+ concurrent users**
+
+Terradev is a cross-cloud compute-provisioning CLI that compresses + stages datasets, provisions optimal instances + nodes, and deploys **3-5x faster** than sequential provisioning with enterprise-grade scalability.
 
-# Terradev CLI v3.7.1
+## What's New in v3.7.3
 
-**Compare GPU prices across 19 clouds. Provision the cheapest one in one command. Automatic CUDA Graph optimization included.**
+### 🚀 Enterprise Scaling Features
 
-<p align="center">
-  <img src="https://raw.githubusercontent.com/theoddden/Terradev/main/demo/terradev-demo.gif" alt="Terradev CLI Demo" width="800">
-</p>
+**Adaptive Concurrency Control**: Automatically scales from 6 to 50 concurrent provisions based on load, supporting 500+ concurrent users with 10x throughput improvement.
 
-## 🚀 What's New in v3.7.1
+**Priority-Based Model Queuing**: Heap-based priority queues replace linear lists, enabling O(log n) operations for 1000+ models with 100x faster queue processing.
+
+**Connection Pooling**: Reused HTTP connections across provider APIs reduce overhead by 80% for high-frequency operations.
+
+**Lazy Tool Loading**: MCP server loads tools on-demand, reducing memory footprint by 50% for enterprise deployments.
+
+**Async Configuration I/O**: Non-blocking configuration operations with TTL-based caching eliminate file I/O bottlenecks.
+
+**Performance Improvements**:
+- **10x** concurrent user capacity (50 → 500+ users)
+- **8x** parallel provisioning throughput  
+- **50%** memory usage reduction
+- **100x** model queue performance at scale
+- **Non-blocking** configuration operations
+
+```bash
+# Enterprise-scale provisioning with adaptive concurrency
+terradev provision -g H100 -n 50 --parallel 25  # Automatically scales to 25 concurrent
+
+# Priority-based model management
+terradev ml warm-pool --configure --strategy priority_based --max-warm-models 100
+```
+
+### CUDA Graph Optimization with NUMA Awareness
 
-### 🧠 CUDA Graph Optimization with NUMA Awareness
 Revolutionary passive CUDA Graph optimization that automatically analyzes and optimizes GPU topology for maximum graph performance:
 
 ```bash
@@ -106,31 +97,31 @@ terradev provision -g H100 -n 4
 # Warm pool prioritizes graph-compatible models
 ```
 
-**Performance Gains:**
-- **2-5x speedup** for CUDA Graph workloads with optimal NUMA topology
-- **30-50% bandwidth penalty eliminated** through automatic GPU/NIC alignment  
-- **Zero configuration** - everything runs passively in the background
-- **Model-aware optimization** - different strategies for transformers vs MoE models
+#### Performance Gains:
+- 2-5x speedup for CUDA Graph workloads with optimal NUMA topology
+- 30-50% bandwidth penalty eliminated through automatic GPU/NIC alignment
+- Zero configuration - everything runs passively in the background
+- Model-aware optimization - different strategies for transformers vs MoE models
 
-### � NUMA Topology Intelligence
+#### NUMA Topology Intelligence
 - **PIX (Same PCIe Switch)**: Optimal for CUDA Graphs (1.0 score)
-- **PXB (Same Root Complex)**: Very good (0.8 score)  
+- **PXB (Same Root Complex)**: Very good (0.8 score)
 - **PHB (Same NUMA Node)**: Good (0.6 score)
 - **SYS (Cross-Socket)**: Poor for graphs (0.3 score)
 
-### 📊 Model-Specific Optimization
+#### Model-Specific Optimization
 - **Transformers**: Highest priority (0.9 base score) - benefit most from graphs
 - **CNNs**: Moderate priority (0.7 base score) - benefit moderately
 - **MoE Models**: Lower priority (0.4 base score) - dynamic routing challenges
 - **Auto-detection**: Model types identified automatically from model IDs
 
-### 🔄 Background Optimization
+#### Background Optimization
 - **Passive Analysis**: Runs automatically every 5 minutes
 - **Warm Pool Enhancement**: CUDA Graph models get higher priority
 - **Endpoint Selection**: Routes to NUMA-optimal endpoints automatically
 - **Performance Tracking**: Monitors graph capture time and replay speedup
 
-## 📋 Complete Tutorial
+## Complete Tutorial
 
 ### Step 1: Install Terradev
 ```bash
@@ -160,7 +151,7 @@ This shows you where to get your API key. Then configure it:
 terradev configure --provider runpod
 ```
 
-Paste your API key when prompted. It's stored locally at `~/.terradev/credentials.json`, *never sent to a Terradev server.* Add more providers later:
+Paste your API key when prompted. It's stored locally at ~/.terradev/credentials.json, never sent to a Terradev server. Add more providers later:
 
 ```bash
 terradev configure --provider vastai
@@ -271,7 +262,7 @@ terradev monitor --job my-job
 # Check status
 terradev train-status
 
-# Manage checkpoints
+# 6. List checkpoints when done
 terradev checkpoint list --job my-job
 ```
 
@@ -374,6 +365,214 @@ If the provisioning layer is wrong, every optimization above it underperforms. A
 
 Terradev handles the foundation automatically so the rest of the stack works the way it's supposed to.
 
+## Complete Workflow Examples
+
+### Example 1: LLM Inference Service
+```bash
+#!/bin/bash
+
+# Complete LLM deployment workflow
+
+# 1. Find cheapest GPU
+terradev quote -g A100 --quick
+# 2. Provision with auto-optimization
+terradev provision -g A100 -n 2 --parallel 4
+# 3. Deploy optimized vLLM
+terradev ml vllm --start --instance-ip $(terradev status --json | jq -r '.[0].ip') --model meta-llama/Llama-2-7b-hf --tp-size 2
+# 4. Set up monitoring
+terradev monitor --endpoint llama-api --live
+# 5. Add customer adapter
+terradev lora add -e http://$(terradev status --json | jq -r '.[0].ip'):8000 -n customer-a -p ./adapters/customer-a
+```
+
+### Example 2: MoE Model Production Deployment
+```bash
+#!/bin/bash
+
+# GLM-5 production deployment
+
+# 1. Deploy MoE cluster
+terradev provision --task clusters/moe-template/task.yaml --set model_id=zai-org/GLM-5-FP8 --set tp_size=8
+# 2. Deploy monitoring
+terradev k8s monitoring-stack --cluster glm-5-cluster
+# 3. Set up warm pool for bursty traffic
+terradev ml warm-pool --configure --strategy traffic_based --max-warm-models 5 --endpoint glm-5-api
+# 4. Test failover
+terradev inferx failover --endpoint glm-5-api --test-load 5000
+```
+
+### Example 3: InferX + LoRA Hybrid Deployment (Production Multi-Tenant)
+```bash
+#!/bin/bash
+
+# Production deployment with cold start failover and multi-tenant LoRA adapters
+
+echo "🚀 Deploying InferX + LoRA Hybrid Inference Service"
+
+# 1. Deploy baseline reserved GPUs for steady traffic
+echo "📍 Step 1: Provision reserved baseline capacity"
+terradev provision -g H100 -n 2 --parallel 4 \
+  --tag baseline-llm \
+  --max-price 2.50
+
+BASELINE_IP=$(terradev status --json | jq -r '.[] | select(.tags[] | contains("baseline-llm")) | .ip' | head -1)
+
+# 2. Deploy optimized vLLM with LoRA support on baseline
+echo "📍 Step 2: Deploy vLLM with LoRA adapter support"
+terradev ml vllm --start \
+  --instance-ip $BASELINE_IP \
+  --model meta-llama/Llama-2-7b-hf \
+  --tp-size 2 \
+  --enable-lora \
+  --enable-kv-offloading \
+  --enable-sleep-mode \
+  --port 8000
+
+# 3. Load customer-specific LoRA adapters
+echo "📍 Step 3: Load multi-tenant LoRA adapters"
+terradev lora add -e http://$BASELINE_IP:8000 \
+  -n customer-enterprise-a \
+  -p ./adapters/customer-enterprise-a
+
+terradev lora add -e http://$BASELINE_IP:8000 \
+  -n customer-startup-b \
+  -p ./adapters/customer-startup-b
+
+terradev lora add -e http://$BASELINE_IP:8000 \
+  -n customer-internal \
+  -p ./adapters/customer-internal
+
+# 4. Configure InferX for cold start and burst handling
+echo "📍 Step 4: Configure InferX for serverless burst capacity"
+terradev inferx deploy \
+  --endpoint burst-llm-api \
+  --model-id meta-llama/Llama-2-7b-hf \
+  --baseline-endpoint http://$BASELINE_IP:8000 \
+  --cold-start-threshold 100 \
+  --burst-capacity 10 \
+  --failover-strategy active-passive
+
+# 5. Set up intelligent routing with semantic awareness
+echo "📍 Step 5: Configure semantic routing for multi-tenant requests"
+cat > routing-config.yaml << EOF
+rules:
+  - name: "enterprise_customers"
+    condition: "header:x-customer-id == 'enterprise-a'"
+    route_to: "baseline"
+    lora_adapter: "customer-enterprise-a"
+    strategy: "latency"
+
+  - name: "startup_customers" 
+    condition: "header:x-customer-id == 'startup-b'"
+    route_to: "baseline"
+    lora_adapter: "customer-startup-b"
+    strategy: "cost"
+
+  - name: "internal_workloads"
+    condition: "header:x-api-key starts_with 'internal_'"
+    route_to: "baseline"
+    lora_adapter: "customer-internal"
+    strategy: "throughput"
+
+  - name: "burst_traffic"
+    condition: "request_rate > 50"
+    route_to: "inferx"
+    strategy: "auto-scale"
+
+  - name: "fallback"
+    condition: "default"
+    route_to: "baseline"
+    lora_adapter: "customer-internal"
+    strategy: "round-robin"
+EOF
+
+terradev semantic-router --deploy --config routing-config.yaml
+
+# 6. Configure warm pool for frequently used adapters
+echo "📍 Step 6: Configure warm pool for LoRA adapters"
+terradev ml warm-pool --configure \
+  --strategy adapter_based \
+  --max-warm-models 5 \
+  --warm-adapters customer-enterprise-a,customer-internal \
+  --idle-eviction-minutes 10 \
+  --enable-predictive-warming
+
+# 7. Set up comprehensive monitoring and alerting
+echo "📍 Step 7: Deploy monitoring stack"
+terradev k8s monitoring-stack --cluster production
+
+# Configure W&B for ML observability
+terradev ml wandb --setup-alerts \
+  --endpoint http://$BASELINE_IP:8000 \
+  --metric-thresholds "latency_p95<2000,throughput>100,gpu_utilization>80" \
+  --alert-channels slack,email
+
+# Configure InferX-specific monitoring
+terradev inferx status --endpoint burst-llm-api --detailed
+terradev inferx failover --endpoint burst-llm-api --test-load 1000
+
+# 8. Test the complete setup
+echo "📍 Step 8: Testing complete deployment"
+echo "Testing baseline endpoint with LoRA..."
+curl -X POST http://$BASELINE_IP:8000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "x-customer-id: enterprise-a" \
+  -d '{
+    "model": "meta-llama/Llama-2-7b-hf",
+    "messages": [{"role": "user", "content": "Hello from enterprise customer!"}],
+    "max_tokens": 100
+  }'
+
+echo "Testing InferX burst endpoint..."
+curl -X POST https://inferx.terradev.cloud/burst-llm-api/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer $INFERX_API_KEY" \
+  -d '{
+    "model": "meta-llama/Llama-2-7b-hf", 
+    "messages": [{"role": "user", "content": "Hello from burst traffic!"}],
+    "max_tokens": 100
+  }'
+
+echo "📍 Step 9: Deployment summary"
+echo "✅ Baseline endpoint: http://$BASELINE_IP:8000"
+echo "✅ InferX endpoint: https://inferx.terradev.cloud/burst-llm-api"
+echo "✅ LoRA adapters loaded: $(terradev lora list -e http://$BASELINE_IP:8000 --count)"
+echo "✅ Semantic routing: Active"
+echo "✅ Warm pool: Configured for top adapters"
+echo "✅ Monitoring: W&B + Prometheus + Grafana"
+
+# 10. Set up automated LoRA updates
+echo "📍 Step 10: Configure automated LoRA adapter updates"
+cat > lora-update-config.yaml << EOF
+adapters:
+  - name: "customer-enterprise-a"
+    path: "./adapters/customer-enterprise-a"
+    update_strategy: "rolling"
+    health_check: true
+    rollback_on_failure: true
+    
+  - name: "customer-startup-b"
+    path: "./adapters/customer-startup-b" 
+    update_strategy: "blue_green"
+    health_check: true
+    rollback_on_failure: true
+
+monitoring:
+  update_frequency: "hourly"
+  health_check_timeout: "30s"
+  rollback_threshold: "error_rate > 0.05"
+EOF
+
+terradev lora auto-update --config lora-update-config.yaml
+
+echo "🎉 InferX + LoRA Hybrid Deployment Complete!"
+echo ""
+echo "📊 Next Steps:"
+echo "1. Monitor performance: terradev monitor --endpoint hybrid-llm --live"
+echo "2. Check LoRA performance: terradev lora metrics --endpoint http://$BASELINE_IP:8000"
+echo "3. Test failover: terradev inferx failover --endpoint burst-llm-api --test-load 5000"
+echo "4. Update adapters: terradev lora update -n customer-enterprise-a -p ./new-adapters/"
+
 ## Quick Reference
 ```bash
 # Set up cloud provider credentials
@@ -407,7 +606,7 @@ terradev analytics --days 30
 terradev optimize
 ```
 
-## 🚀 Features
+## Features
 
 - **19 Cloud Providers**: RunPod, VastAI, Lambda Labs, AWS, GCP, Azure, Oracle, and more
 - **Automatic Topology Optimization**: NUMA alignment, RDMA, CPU pinning
@@ -419,7 +618,7 @@ terradev optimize
 - **GitOps Automation**: Production-ready workflows with ArgoCD/Flux
 - **CUDA Graph Optimization**: Passive NUMA-aware graph performance optimization
 
-## 📦 Installation
+## Installation
 
 ```bash
 # Basic installation
@@ -435,9 +634,9 @@ pip install terradev-cli[azure]    # Azure
 pip install terradev-cli[hf]       # HuggingFace Spaces
 ```
 
-## 🔧 Configuration
+## Configuration
 
-Your API keys are stored locally at `~/.terradev/credentials.json` and never sent to Terradev servers.
+Your API keys are stored locally at ~/.terradev/credentials.json and never sent to Terradev servers.
 
 ```bash
 # Configure multiple providers
@@ -447,22 +646,22 @@ terradev configure --provider aws
 terradev configure --provider gcp
 ```
 
-## 📊 Performance
+## Performance
 
 - **2-8x throughput improvements** with vLLM optimization
 - **30-50% bandwidth penalty eliminated** with NUMA topology
 - **2-5x CUDA Graph speedup** with optimal topology
 - **Up to 90% cost savings** with automatic provider switching
 
-## 🤝 Contributing
+## Contributing
 
 We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
 
-## 📄 License
+## License
 
-MIT License - see [LICENSE](LICENSE) file for details.
+BUSL 1.1 License - see [LICENSE](LICENSE) file for details.
 
-## 🆘 Support
+## Support
 
 - **Documentation**: [Full User Guide](USER_GUIDE.md)
 - **Issues**: [GitHub Issues](https://github.com/theoddden/Terradev/issues)