terradev-cli 3.5.2__tar.gz → 3.7.0__tar.gz

{terradev_cli-3.5.2 → terradev_cli-3.7.0}/PKG-INFO

@@ -1,49 +1,24 @@
 Metadata-Version: 2.4
 Name: terradev-cli
-Version: 3.5.2
-Summary: Cross-cloud GPU provisioning with GitOps automation and HuggingFace Spaces deployment
-Author-email: Terradev Team <team@terradev.com>
-Maintainer-email: Terradev Team <team@terradev.com>
-License: BUSL-1.1
-Project-URL: Homepage, https://terradev.cloud
-Project-URL: Documentation, https://docs.terradev.com
-Project-URL: Repository, https://github.com/theoddden/Terradev
-Project-URL: Bug Tracker, https://github.com/theoddden/Terradev/issues
-Project-URL: Changelog, https://github.com/theoddden/Terradev/blob/main/CHANGELOG.md
-Project-URL: Analytics, https://api.terradev.cloud/dashboard
-Keywords: cloud,provisioning,gpu,multi-cloud,arbitrage,aws,gcp,azure,runpod,vastai,machine-learning,cost-optimization,parallel,gitops,argocd,flux,kubernetes,infrastructure-as-code,langchain,langgraph,sglang,wandb,mlflow,analytics,setup,conversion,telemetry,real-time,monitoring,dashboard,huggingface,huggingface-spaces,gradio,streamlit,llm,transformers,pytorch,sentence-transformers,smart-templates,hardware-optimization,model-deployment,inference,training,byoapi,terraform,kubernetes,helm,karpenter,opa,prometheus,grafana,ray,vllm,ollama,dvc,data-version-control,numa,disaggregated-inference,moe,mixture-of-experts,qwen,distserve,xcd,mi300x,h200,prefill-decode
-Classifier: Development Status :: 5 - Production/Stable
+Version: 3.7.0
+Summary: Cross-cloud GPU provisioning with Arize Phoenix observability, NeMo Guardrails safety, Qdrant vector DB, vLLM/SGLang inference, and Terraform-powered parallel provisioning
+Home-page: https://github.com/theoddden/terradev
+Author: Terradev Team
+Author-email: team@terradev.com
+Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Developers
-Classifier: Intended Audience :: System Administrators
-Classifier: Topic :: Software Development :: Libraries :: Python Modules
-Classifier: Topic :: System :: Systems Administration
-Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
-Classifier: License :: Other/Proprietary License
+Classifier: License :: OSI Approved :: MIT License
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
-Classifier: Operating System :: OS Independent
-Classifier: Environment :: Console
-Classifier: Natural Language :: English
-Requires-Python: >=3.9
+Requires-Python: >=3.8
 Description-Content-Type: text/markdown
-License-File: LICENSE
 Requires-Dist: click>=8.0.0
 Requires-Dist: aiohttp>=3.9.0
 Requires-Dist: pyyaml>=6.0
-Requires-Dist: cryptography>=41.0.0
 Requires-Dist: requests>=2.31.0
-Requires-Dist: gradio>=4.0.0
-Requires-Dist: streamlit>=1.28.0
-Requires-Dist: transformers>=4.0.0
-Requires-Dist: torch>=2.0.0
-Requires-Dist: sentence-transformers>=2.0.0
-Requires-Dist: plotly>=5.0.0
-Requires-Dist: pandas>=1.0.0
-Requires-Dist: numpy>=1.0.0
-Requires-Dist: huggingface-hub>=0.19.0
 Requires-Dist: stripe>=7.0.0
 Provides-Extra: aws
 Requires-Dist: boto3>=1.34.0; extra == "aws"
@@ -52,51 +27,72 @@ Requires-Dist: google-cloud-compute>=1.8.0; extra == "gcp"
 Provides-Extra: azure
 Requires-Dist: azure-mgmt-compute>=29.0.0; extra == "azure"
 Requires-Dist: azure-identity>=1.12.0; extra == "azure"
-Provides-Extra: lmcache
-Requires-Dist: lmcache>=0.1; extra == "lmcache"
-Requires-Dist: redis>=5.0.0; extra == "lmcache"
+Provides-Extra: oracle
+Requires-Dist: oci>=2.118.0; extra == "oracle"
+Provides-Extra: hf
+Requires-Dist: huggingface-hub>=0.19.0; extra == "hf"
 Provides-Extra: all
 Requires-Dist: boto3>=1.34.0; extra == "all"
 Requires-Dist: google-cloud-compute>=1.8.0; extra == "all"
 Requires-Dist: azure-mgmt-compute>=29.0.0; extra == "all"
 Requires-Dist: azure-identity>=1.12.0; extra == "all"
-Requires-Dist: langchain>=0.1.0; extra == "all"
-Requires-Dist: langgraph>=0.0.20; extra == "all"
-Requires-Dist: sglang>=0.3.0; extra == "all"
-Requires-Dist: wandb>=0.16.0; extra == "all"
-Requires-Dist: mlflow>=2.8.0; extra == "all"
+Requires-Dist: oci>=2.118.0; extra == "all"
 Requires-Dist: huggingface-hub>=0.19.0; extra == "all"
-Requires-Dist: prometheus-client>=0.17.0; extra == "all"
-Requires-Dist: grafana-api>=1.0.3; extra == "all"
-Requires-Dist: lmcache>=0.1; extra == "all"
-Requires-Dist: redis>=5.0.0; extra == "all"
-Provides-Extra: hf
-Requires-Dist: aiohttp>=3.9.0; extra == "hf"
-Requires-Dist: huggingface-hub>=0.19.0; extra == "hf"
-Requires-Dist: gradio>=4.0.0; extra == "hf"
-Requires-Dist: streamlit>=1.28.0; extra == "hf"
-Provides-Extra: dev
-Requires-Dist: pytest>=7.0.0; extra == "dev"
-Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
-Requires-Dist: black>=22.0.0; extra == "dev"
-Requires-Dist: flake8>=5.0.0; extra == "dev"
-Requires-Dist: build>=0.10.0; extra == "dev"
-Requires-Dist: twine>=4.0.0; extra == "dev"
-Dynamic: license-file
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
 
-# Terradev CLI v3.5.2
+# Terradev CLI v3.7.0
 
-**Compare GPU prices across 19 clouds. Provision the cheapest one in one command.**
+**Compare GPU prices across 20 clouds. Provision the cheapest one in one command. Arize Phoenix observability, NeMo Guardrails safety, Qdrant vector DB, and automatic vLLM optimization included.**
 
 <p align="center">
   <img src="https://raw.githubusercontent.com/theoddden/Terradev/main/demo/terradev-demo.gif" alt="Terradev CLI Demo" width="800">
 </p>
 
-## Why Terradev?
+## 🚀 What's New in v3.7.0
+
+### 🧠 Automatic vLLM Workload-Based Optimization
+The most advanced vLLM optimization system - automatically analyzes your workload patterns and optimizes the 6 critical knobs most teams never touch:
 
-Developers overpay by only accessing single-cloud workflows, hopping across switches out of NUMA alignment, or using sequential provisioning with inefficient egress + rate-limiting.
+```bash
+# Auto-optimize based on your actual workload
+terradev vllm auto-optimize -s workload.json -m meta-llama/Llama-2-7b-hf -g 4
+
+# Analyze running server for optimization
+terradev vllm analyze -e http://localhost:8000
 
-Terradev is a cross-cloud compute-provisioning CLI that compresses + stages datasets, provisions optimal instances + nodes, and deploys faster and cheaper than sequential provisioning.
+# Generate optimized configurations
+terradev vllm optimize -m mistralai/Mistral-7B-v0.1 -t throughput -o helm
+```
+
+**Performance Gains:**
+- **2-8x throughput improvements** typical
+- **10-20% latency reduction** for sensitive workloads  
+- **Zero manual tuning required** - intelligent workload analysis
+- **Dynamic adaptation** - adjusts as patterns change
+
+### 🔧 The 6 Critical vLLM Knobs
+1. **`--max-num-batched-tokens`** - 2048→16384 (8x throughput)
+2. **`--gpu-memory-utilization`** - 0.90→0.95 (5% more VRAM)
+3. **`--max-num-seqs`** - 256/1024→512-2048 (prevent queuing)
+4. **`--enable-prefix-caching`** - OFF→ON (free throughput win)
+5. **`--enable-chunked-prefill`** - OFF→ON (better prefill)
+6. **CPU Core Allocation** - 2 + #GPUs (prevent starvation)
+
+### 🎯 Smart Workload Analysis
+- **QPS Pattern Recognition** - High traffic → larger batches
+- **Prompt Length Analysis** - Long prompts → chunked prefill
+- **Concurrency Detection** - Multi-user → higher sequence limits
+- **Latency Sensitivity** - Balanced vs aggressive optimization
+- **Memory Pressure** - Conservative vs aggressive GPU utilization
 
 ## GitOps Automation
 
@@ -320,13 +316,14 @@ terradev ml sglang --start --instance-ip <IP> \
 ## Installation
 
 ```bash
-pip install terradev-cli
+pip install terradev-cli==3.6.2
 ```
 
-With HF Spaces support:
+With vLLM optimization and HF Spaces support:
 ```bash
+pip install terradev-cli[vllm]      # vLLM optimization features
 pip install terradev-cli[hf]        # HuggingFace Spaces deployment
-pip install terradev-cli[all]        # All cloud providers + ML services + HF Spaces
+pip install terradev-cli[all]        # All cloud providers + ML services + HF Spaces + vLLM
 ```
 
 ## Quick Start
@@ -346,25 +343,30 @@ terradev hf-space my-llama --model-id meta-llama/Llama-2-7b-hf --template llm
 terradev hf-space my-embeddings --model-id sentence-transformers/all-MiniLM-L6-v2 --template embedding
 terradev hf-space my-image --model-id runwayml/stable-diffusion-v1-5 --template image
 
-# 4. Get enhanced quotes with conversion prompts
+# 4. Optimize vLLM automatically (NEW!)
+terradev vllm auto-optimize -s workload.json -m meta-llama/Llama-2-7b-hf -g 4
+terradev vllm analyze -e http://localhost:8000  # Analyze running server
+terradev vllm benchmark -e http://localhost:8000 -c 10  # Performance test
+
+# 5. Get enhanced quotes with conversion prompts
 terradev quote -g A100
 terradev quote -g A100 --quick  # Quick provision best quote
 
-# 5. Provision the cheapest instance (real API call)
+# 6. Provision the cheapest instance (real API call)
 terradev provision -g A100
 
-# 6. Configure ML services
+# 7. Configure ML services
 terradev configure --provider wandb --dashboard-enabled true
 terradev configure --provider langchain --tracing-enabled true
 
-# 7. Use ML services
+# 8. Use ML services
 terradev ml wandb --test
 terradev ml langchain --create-workflow my-workflow
 
-# 8. View analytics
+# 9. View analytics
 python user_analytics.py
 
-# 9. Provision 4x H100s in parallel across multiple clouds
+# 10. Provision 4x H100s in parallel across multiple clouds
 terradev provision -g H100 -n 4 --parallel 6
 
 # 10. Dry-run to see the allocation plan without launching

{terradev_cli-3.5.2 → terradev_cli-3.7.0}/README.md

@@ -1,16 +1,47 @@
-# Terradev CLI v3.5.2
+# Terradev CLI v3.7.0
 
-**Compare GPU prices across 19 clouds. Provision the cheapest one in one command.**
+**Compare GPU prices across 20 clouds. Provision the cheapest one in one command. Arize Phoenix observability, NeMo Guardrails safety, Qdrant vector DB, and automatic vLLM optimization included.**
 
 <p align="center">
   <img src="https://raw.githubusercontent.com/theoddden/Terradev/main/demo/terradev-demo.gif" alt="Terradev CLI Demo" width="800">
 </p>
 
-## Why Terradev?
+## 🚀 What's New in v3.7.0
 
-Developers overpay by only accessing single-cloud workflows, hopping across switches out of NUMA alignment, or using sequential provisioning with inefficient egress + rate-limiting.
+### 🧠 Automatic vLLM Workload-Based Optimization
+The most advanced vLLM optimization system - automatically analyzes your workload patterns and optimizes the 6 critical knobs most teams never touch:
 
-Terradev is a cross-cloud compute-provisioning CLI that compresses + stages datasets, provisions optimal instances + nodes, and deploys faster and cheaper than sequential provisioning.
+```bash
+# Auto-optimize based on your actual workload
+terradev vllm auto-optimize -s workload.json -m meta-llama/Llama-2-7b-hf -g 4
+
+# Analyze running server for optimization
+terradev vllm analyze -e http://localhost:8000
+
+# Generate optimized configurations
+terradev vllm optimize -m mistralai/Mistral-7B-v0.1 -t throughput -o helm
+```
+
+**Performance Gains:**
+- **2-8x throughput improvements** typical
+- **10-20% latency reduction** for sensitive workloads  
+- **Zero manual tuning required** - intelligent workload analysis
+- **Dynamic adaptation** - adjusts as patterns change
+
+### 🔧 The 6 Critical vLLM Knobs
+1. **`--max-num-batched-tokens`** - 2048→16384 (8x throughput)
+2. **`--gpu-memory-utilization`** - 0.90→0.95 (5% more VRAM)
+3. **`--max-num-seqs`** - 256/1024→512-2048 (prevent queuing)
+4. **`--enable-prefix-caching`** - OFF→ON (free throughput win)
+5. **`--enable-chunked-prefill`** - OFF→ON (better prefill)
+6. **CPU Core Allocation** - 2 + #GPUs (prevent starvation)
+
+### 🎯 Smart Workload Analysis
+- **QPS Pattern Recognition** - High traffic → larger batches
+- **Prompt Length Analysis** - Long prompts → chunked prefill
+- **Concurrency Detection** - Multi-user → higher sequence limits
+- **Latency Sensitivity** - Balanced vs aggressive optimization
+- **Memory Pressure** - Conservative vs aggressive GPU utilization
 
 ## GitOps Automation
 
@@ -234,13 +265,14 @@ terradev ml sglang --start --instance-ip <IP> \
 ## Installation
 
 ```bash
-pip install terradev-cli
+pip install terradev-cli==3.6.2
 ```
 
-With HF Spaces support:
+With vLLM optimization and HF Spaces support:
 ```bash
+pip install terradev-cli[vllm]      # vLLM optimization features
 pip install terradev-cli[hf]        # HuggingFace Spaces deployment
-pip install terradev-cli[all]        # All cloud providers + ML services + HF Spaces
+pip install terradev-cli[all]        # All cloud providers + ML services + HF Spaces + vLLM
 ```
 
 ## Quick Start
@@ -260,25 +292,30 @@ terradev hf-space my-llama --model-id meta-llama/Llama-2-7b-hf --template llm
 terradev hf-space my-embeddings --model-id sentence-transformers/all-MiniLM-L6-v2 --template embedding
 terradev hf-space my-image --model-id runwayml/stable-diffusion-v1-5 --template image
 
-# 4. Get enhanced quotes with conversion prompts
+# 4. Optimize vLLM automatically (NEW!)
+terradev vllm auto-optimize -s workload.json -m meta-llama/Llama-2-7b-hf -g 4
+terradev vllm analyze -e http://localhost:8000  # Analyze running server
+terradev vllm benchmark -e http://localhost:8000 -c 10  # Performance test
+
+# 5. Get enhanced quotes with conversion prompts
 terradev quote -g A100
 terradev quote -g A100 --quick  # Quick provision best quote
 
-# 5. Provision the cheapest instance (real API call)
+# 6. Provision the cheapest instance (real API call)
 terradev provision -g A100
 
-# 6. Configure ML services
+# 7. Configure ML services
 terradev configure --provider wandb --dashboard-enabled true
 terradev configure --provider langchain --tracing-enabled true
 
-# 7. Use ML services
+# 8. Use ML services
 terradev ml wandb --test
 terradev ml langchain --create-workflow my-workflow
 
-# 8. View analytics
+# 9. View analytics
 python user_analytics.py
 
-# 9. Provision 4x H100s in parallel across multiple clouds
+# 10. Provision 4x H100s in parallel across multiple clouds
 terradev provision -g H100 -n 4 --parallel 6
 
 # 10. Dry-run to see the allocation plan without launching