FAI-RL 0.1.0__tar.gz

fai_rl-0.1.0/FAI_RL.egg-info/PKG-INFO

@@ -0,0 +1,203 @@
+Metadata-Version: 2.4
+Name: FAI-RL
+Version: 0.1.0
+Summary: Foundation of AI - Reinforcement learning Library
+Author-email: Roblox <ylim@roblox.com>, Roblox <mnandwana@roblox.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/Roblox/FAI-RL
+Project-URL: Documentation, https://github.com/Roblox/FAI-RL#readme
+Project-URL: Repository, https://github.com/Roblox/FAI-RL
+Project-URL: Issues, https://github.com/Roblox/FAI-RL/issues
+Keywords: reinforcement learning,language models,transformers,rlhf,dpo,ppo,sft
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Requires-Dist: torch>=2.7.0
+Requires-Dist: torchvision>=0.22.0
+Requires-Dist: torchaudio>=2.7.0
+Requires-Dist: datasets>=4.0.0
+Requires-Dist: transformers>=4.56.0
+Requires-Dist: trl>=0.23.0
+Requires-Dist: wandb>=0.21.0
+Requires-Dist: bitsandbytes>=0.46.0
+Requires-Dist: peft>=0.17.0
+Requires-Dist: deepspeed>=0.17.0
+Requires-Dist: ipykernel>=6.30.0
+Requires-Dist: ipywidgets>=8.1.0
+Requires-Dist: fsspec>=2025.3.0
+Requires-Dist: huggingface_hub>=0.34.0
+Requires-Dist: mpi4py>=4.1.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: black>=22.0.0; extra == "dev"
+Requires-Dist: flake8>=4.0.0; extra == "dev"
+Requires-Dist: mypy>=0.950; extra == "dev"
+
+# FAI-RL: Foundation of AI - Reinforcement learning Library
+
+A modular, production-ready library designed for **easy training, inference, and evaluation** of language models using reinforcement learning methods. Currently supports: 
+- SFT (Supervised Fine-Tuning)
+- DPO (Direct Preference Optimization)
+- PPO (Proximal Policy Optimization)
+- GRPO (Group Relative Preference Optimization)
+- GSPO (Group Sequence Policy Optimization)
+
+### Flexible Configuration System
+* YAML-based configuration for all training parameters
+* Pre-configured recipes for popular models
+* DeepSpeed ZeRO-3 integration for distributed training
+
+## 🚀 Quick Start
+
+Get started with installation, training, inference, and evaluation in just a few commands:
+
+### 📦 Installation
+
+#### Option 1: Install from PyPI (Recommended)
+
+```bash
+pip install FAI-RL
+```
+
+#### Option 2: Install from source
+
+```bash
+# Clone the repository
+git clone https://github.com/Roblox/FAI-RL.git
+cd FAI-RL
+
+# Install in development mode
+pip install -e .
+```
+
+#### Option 3: Manual setup with virtual environment
+
+```bash
+# Clone the repository
+git clone https://github.com/Roblox/FAI-RL.git
+cd FAI-RL
+
+# Create virtual environment
+python -m venv venv_fai_rl
+source venv_fai_rl/bin/activate
+
+# Install the package
+pip install -e .
+```
+
+### Training
+
+Train a model using SFT, DPO, PPO, GRPO, or GSPO:
+
+```bash
+# Single GPU training
+./scripts/run_training.sh \
+    --config configs/training/dpo/llama3_3B_recipe.yaml \
+    --num-gpus 1
+
+# Multi-GPU training (8 GPUs)
+./scripts/run_training.sh \
+    --config configs/training/dpo/llama3_3B_recipe.yaml \
+    --num-gpus 8 \
+    --nohup  # Run in background
+```
+
+### Inference
+
+Generate responses from your trained models:
+
+```bash
+# Run inference on trained model
+./scripts/run_inference.sh \
+    --config configs/inference/llama3_3B_recipe.yaml
+
+# Run inference with debug mode
+./scripts/run_inference.sh \
+    --config configs/inference/llama3_3B_recipe.yaml \
+    --debug
+```
+
+### Evaluation
+
+Evaluate model performance on benchmarks:
+
+```bash
+# Evaluate on MMLU benchmark
+./scripts/run_evaluation.sh \
+    --config configs/evaluation/mmlu/llama3_3B_recipe.yaml
+
+# Evaluate with debug output
+./scripts/run_evaluation.sh \
+    --config configs/evaluation/mmlu/llama3_3B_recipe.yaml \
+    --debug
+```
+
+-----
+
+## 📁 Project Structure
+
+```
+FAI-RL/
+├── core/                      # Core framework components
+├── trainers/                  # Training method implementations
+├── inference/                 # Inference components
+├── evaluations/               # Evaluation system
+├── configs/                   # Configuration files
+│   ├── training/              # Training configurations
+│   ├── inference/             # Inference configurations
+│   ├── evaluation/            # Evaluation configurations
+│   └── deepspeed/             # DeepSpeed ZeRO configurations
+├── utils/                     # Utility modules
+├── scripts/                   # Scripts
+├── logs/                      # Training logs (auto-generated)
+└── outputs/                   # Inference output (auto-generated)
+```
+
+-----
+
+## 🔗 Quick Links
+
+* **[Training Guide](./trainers/README.md)** - Comprehensive guide to configuring and running model training with detailed parameter explanations
+* **[Inference Guide](./inference/README.md)** - Running model inference and text generation
+* **[Evaluation Guide](./evaluations/README.md)** - Evaluating model performance on standard benchmarks
+
+## Algorithm Selection Guide
+
+Choose the right algorithm for your use case:
+
+| Algorithm | Best For | Requirements | Key Benefits |
+|-----------|----------|--------------|--------------|
+| **SFT** | Initial instruction tuning, domain adaptation | Prompt-response pairs | Simple, fast, establishes baseline |
+| **DPO** | Aligning to human preferences | Preference pairs (chosen/rejected) | No reward model needed, stable training |
+| **PPO** | Complex sequential tasks, agentic workflows | Preference pairs + reward model | Most flexible, handles multi-turn interactions |
+| **GRPO** | Math reasoning, efficiency-focused tasks | Question-answer pairs | No critic model, faster training |
+| **GSPO** | Multi-turn RL, stable sequence-level optimization | Question-answer pairs | Better stability than GRPO |
+
+## Memory Optimization
+
+FAI-RL supports various techniques to train large models efficiently:
+
+* **Full Fine-tuning:** Train all model parameters (requires most memory)
+* **LoRA:** Parameter-efficient training (~10% memory of full fine-tuning)
+* **QLoRA:** 4-bit quantized LoRA (train 7B+ models on single consumer GPU)
+* **DeepSpeed ZeRO-3:** Distributed training for models that don't fit on single GPU
+
+## 🧪 Tested Environment
+
+This framework has been validated on:
+
+* **Instance:** AWS EC2 p4d.24xlarge
+* **GPUs:** 8 x NVIDIA A100-SXM4-80GB (80GB VRAM each)
+* **CPU:** 96 vCPUs
+* **Memory:** 1152 GiB
+* **Storage:** 8TB NVMe SSD
+* **Network:** 400 Gbps

fai_rl-0.1.0/FAI_RL.egg-info/SOURCES.txt

@@ -0,0 +1,59 @@
+README.md
+pyproject.toml
+setup.py
+FAI_RL.egg-info/PKG-INFO
+FAI_RL.egg-info/SOURCES.txt
+FAI_RL.egg-info/dependency_links.txt
+FAI_RL.egg-info/entry_points.txt
+FAI_RL.egg-info/requires.txt
+FAI_RL.egg-info/top_level.txt
+configs/__init__.py
+configs/deepspeed/zero3_config_gpu1.json
+configs/deepspeed/zero3_config_gpu2.json
+configs/deepspeed/zero3_config_gpu4.json
+configs/deepspeed/zero3_config_gpu8.json
+configs/evaluation/mmlu/llama3_3B_recipe.yaml
+configs/inference/llama3_3B_recipe.yaml
+configs/training/dpo/llama3_3B_full_recipe.yaml
+configs/training/dpo/llama3_3B_lora_recipe.yaml
+configs/training/dpo/llama3_3B_qlora_recipe.yaml
+configs/training/grpo/llama3_3B_full_recipe.yaml
+configs/training/grpo/llama3_3B_lora_recipe.yaml
+configs/training/gspo/llama3_3B_full_recipe.yaml
+configs/training/gspo/llama3_3B_lora_recipe.yaml
+configs/training/ppo/llama3_3B_full_recipe.yaml
+configs/training/ppo/llama3_3B_lora_recipe.yaml
+configs/training/ppo/llama3_3B_qlora_recipe.yaml
+configs/training/sft/llama3_3B_full_recipe.yaml
+configs/training/sft/llama3_3B_lora_recipe.yaml
+configs/training/sft/llama3_3B_qlora_recipe.yaml
+core/__init__.py
+core/config.py
+core/model_utils.py
+core/trainer_base.py
+evaluations/README.md
+evaluations/__init__.py
+evaluations/eval.py
+inference/README.md
+inference/__init__.py
+inference/inference.py
+scripts/run_evaluation.sh
+scripts/run_inference.py
+scripts/run_inference.sh
+scripts/run_training.sh
+scripts/train.py
+trainers/README.md
+trainers/__init__.py
+trainers/dpo_trainer.py
+trainers/grpo_trainer.py
+trainers/gspo_trainer.py
+trainers/ppo_trainer.py
+trainers/sft_trainer.py
+trainers/rewards/__init__.py
+trainers/rewards/accuracy_rewards.py
+trainers/rewards/format_rewards.py
+trainers/templates/__init__.py
+trainers/templates/gsm8k_template.py
+trainers/templates/openmathinstruct_template.py
+utils/__init__.py
+utils/logging_utils.py

fai_rl-0.1.0/FAI_RL.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

fai_rl-0.1.0/FAI_RL.egg-info/entry_points.txt

@@ -0,0 +1,4 @@
+[console_scripts]
+fai-rl-eval = evaluations.eval:main
+fai-rl-inference = scripts.run_inference:main
+fai-rl-train = scripts.train:main

fai_rl-0.1.0/FAI_RL.egg-info/requires.txt

@@ -0,0 +1,21 @@
+torch>=2.7.0
+torchvision>=0.22.0
+torchaudio>=2.7.0
+datasets>=4.0.0
+transformers>=4.56.0
+trl>=0.23.0
+wandb>=0.21.0
+bitsandbytes>=0.46.0
+peft>=0.17.0
+deepspeed>=0.17.0
+ipykernel>=6.30.0
+ipywidgets>=8.1.0
+fsspec>=2025.3.0
+huggingface_hub>=0.34.0
+mpi4py>=4.1.0
+
+[dev]
+pytest>=7.0.0
+black>=22.0.0
+flake8>=4.0.0
+mypy>=0.950

fai_rl-0.1.0/FAI_RL.egg-info/top_level.txt

@@ -0,0 +1,8 @@
+configs
+core
+dist
+evaluations
+inference
+scripts
+trainers
+utils

fai_rl-0.1.0/PKG-INFO

@@ -0,0 +1,203 @@
+Metadata-Version: 2.4
+Name: FAI-RL
+Version: 0.1.0
+Summary: Foundation of AI - Reinforcement learning Library
+Author-email: Roblox <ylim@roblox.com>, Roblox <mnandwana@roblox.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/Roblox/FAI-RL
+Project-URL: Documentation, https://github.com/Roblox/FAI-RL#readme
+Project-URL: Repository, https://github.com/Roblox/FAI-RL
+Project-URL: Issues, https://github.com/Roblox/FAI-RL/issues
+Keywords: reinforcement learning,language models,transformers,rlhf,dpo,ppo,sft
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Requires-Dist: torch>=2.7.0
+Requires-Dist: torchvision>=0.22.0
+Requires-Dist: torchaudio>=2.7.0
+Requires-Dist: datasets>=4.0.0
+Requires-Dist: transformers>=4.56.0
+Requires-Dist: trl>=0.23.0
+Requires-Dist: wandb>=0.21.0
+Requires-Dist: bitsandbytes>=0.46.0
+Requires-Dist: peft>=0.17.0
+Requires-Dist: deepspeed>=0.17.0
+Requires-Dist: ipykernel>=6.30.0
+Requires-Dist: ipywidgets>=8.1.0
+Requires-Dist: fsspec>=2025.3.0
+Requires-Dist: huggingface_hub>=0.34.0
+Requires-Dist: mpi4py>=4.1.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: black>=22.0.0; extra == "dev"
+Requires-Dist: flake8>=4.0.0; extra == "dev"
+Requires-Dist: mypy>=0.950; extra == "dev"
+
+# FAI-RL: Foundation of AI - Reinforcement learning Library
+
+A modular, production-ready library designed for **easy training, inference, and evaluation** of language models using reinforcement learning methods. Currently supports: 
+- SFT (Supervised Fine-Tuning)
+- DPO (Direct Preference Optimization)
+- PPO (Proximal Policy Optimization)
+- GRPO (Group Relative Preference Optimization)
+- GSPO (Group Sequence Policy Optimization)
+
+### Flexible Configuration System
+* YAML-based configuration for all training parameters
+* Pre-configured recipes for popular models
+* DeepSpeed ZeRO-3 integration for distributed training
+
+## 🚀 Quick Start
+
+Get started with installation, training, inference, and evaluation in just a few commands:
+
+### 📦 Installation
+
+#### Option 1: Install from PyPI (Recommended)
+
+```bash
+pip install FAI-RL
+```
+
+#### Option 2: Install from source
+
+```bash
+# Clone the repository
+git clone https://github.com/Roblox/FAI-RL.git
+cd FAI-RL
+
+# Install in development mode
+pip install -e .
+```
+
+#### Option 3: Manual setup with virtual environment
+
+```bash
+# Clone the repository
+git clone https://github.com/Roblox/FAI-RL.git
+cd FAI-RL
+
+# Create virtual environment
+python -m venv venv_fai_rl
+source venv_fai_rl/bin/activate
+
+# Install the package
+pip install -e .
+```
+
+### Training
+
+Train a model using SFT, DPO, PPO, GRPO, or GSPO:
+
+```bash
+# Single GPU training
+./scripts/run_training.sh \
+    --config configs/training/dpo/llama3_3B_recipe.yaml \
+    --num-gpus 1
+
+# Multi-GPU training (8 GPUs)
+./scripts/run_training.sh \
+    --config configs/training/dpo/llama3_3B_recipe.yaml \
+    --num-gpus 8 \
+    --nohup  # Run in background
+```
+
+### Inference
+
+Generate responses from your trained models:
+
+```bash
+# Run inference on trained model
+./scripts/run_inference.sh \
+    --config configs/inference/llama3_3B_recipe.yaml
+
+# Run inference with debug mode
+./scripts/run_inference.sh \
+    --config configs/inference/llama3_3B_recipe.yaml \
+    --debug
+```
+
+### Evaluation
+
+Evaluate model performance on benchmarks:
+
+```bash
+# Evaluate on MMLU benchmark
+./scripts/run_evaluation.sh \
+    --config configs/evaluation/mmlu/llama3_3B_recipe.yaml
+
+# Evaluate with debug output
+./scripts/run_evaluation.sh \
+    --config configs/evaluation/mmlu/llama3_3B_recipe.yaml \
+    --debug
+```
+
+-----
+
+## 📁 Project Structure
+
+```
+FAI-RL/
+├── core/                      # Core framework components
+├── trainers/                  # Training method implementations
+├── inference/                 # Inference components
+├── evaluations/               # Evaluation system
+├── configs/                   # Configuration files
+│   ├── training/              # Training configurations
+│   ├── inference/             # Inference configurations
+│   ├── evaluation/            # Evaluation configurations
+│   └── deepspeed/             # DeepSpeed ZeRO configurations
+├── utils/                     # Utility modules
+├── scripts/                   # Scripts
+├── logs/                      # Training logs (auto-generated)
+└── outputs/                   # Inference output (auto-generated)
+```
+
+-----
+
+## 🔗 Quick Links
+
+* **[Training Guide](./trainers/README.md)** - Comprehensive guide to configuring and running model training with detailed parameter explanations
+* **[Inference Guide](./inference/README.md)** - Running model inference and text generation
+* **[Evaluation Guide](./evaluations/README.md)** - Evaluating model performance on standard benchmarks
+
+## Algorithm Selection Guide
+
+Choose the right algorithm for your use case:
+
+| Algorithm | Best For | Requirements | Key Benefits |
+|-----------|----------|--------------|--------------|
+| **SFT** | Initial instruction tuning, domain adaptation | Prompt-response pairs | Simple, fast, establishes baseline |
+| **DPO** | Aligning to human preferences | Preference pairs (chosen/rejected) | No reward model needed, stable training |
+| **PPO** | Complex sequential tasks, agentic workflows | Preference pairs + reward model | Most flexible, handles multi-turn interactions |
+| **GRPO** | Math reasoning, efficiency-focused tasks | Question-answer pairs | No critic model, faster training |
+| **GSPO** | Multi-turn RL, stable sequence-level optimization | Question-answer pairs | Better stability than GRPO |
+
+## Memory Optimization
+
+FAI-RL supports various techniques to train large models efficiently:
+
+* **Full Fine-tuning:** Train all model parameters (requires most memory)
+* **LoRA:** Parameter-efficient training (~10% memory of full fine-tuning)
+* **QLoRA:** 4-bit quantized LoRA (train 7B+ models on single consumer GPU)
+* **DeepSpeed ZeRO-3:** Distributed training for models that don't fit on single GPU
+
+## 🧪 Tested Environment
+
+This framework has been validated on:
+
+* **Instance:** AWS EC2 p4d.24xlarge
+* **GPUs:** 8 x NVIDIA A100-SXM4-80GB (80GB VRAM each)
+* **CPU:** 96 vCPUs
+* **Memory:** 1152 GiB
+* **Storage:** 8TB NVMe SSD
+* **Network:** 400 Gbps

fai_rl-0.1.0/README.md

@@ -0,0 +1,159 @@
+# FAI-RL: Foundation of AI - Reinforcement learning Library
+
+A modular, production-ready library designed for **easy training, inference, and evaluation** of language models using reinforcement learning methods. Currently supports: 
+- SFT (Supervised Fine-Tuning)
+- DPO (Direct Preference Optimization)
+- PPO (Proximal Policy Optimization)
+- GRPO (Group Relative Preference Optimization)
+- GSPO (Group Sequence Policy Optimization)
+
+### Flexible Configuration System
+* YAML-based configuration for all training parameters
+* Pre-configured recipes for popular models
+* DeepSpeed ZeRO-3 integration for distributed training
+
+## 🚀 Quick Start
+
+Get started with installation, training, inference, and evaluation in just a few commands:
+
+### 📦 Installation
+
+#### Option 1: Install from PyPI (Recommended)
+
+```bash
+pip install FAI-RL
+```
+
+#### Option 2: Install from source
+
+```bash
+# Clone the repository
+git clone https://github.com/Roblox/FAI-RL.git
+cd FAI-RL
+
+# Install in development mode
+pip install -e .
+```
+
+#### Option 3: Manual setup with virtual environment
+
+```bash
+# Clone the repository
+git clone https://github.com/Roblox/FAI-RL.git
+cd FAI-RL
+
+# Create virtual environment
+python -m venv venv_fai_rl
+source venv_fai_rl/bin/activate
+
+# Install the package
+pip install -e .
+```
+
+### Training
+
+Train a model using SFT, DPO, PPO, GRPO, or GSPO:
+
+```bash
+# Single GPU training
+./scripts/run_training.sh \
+    --config configs/training/dpo/llama3_3B_recipe.yaml \
+    --num-gpus 1
+
+# Multi-GPU training (8 GPUs)
+./scripts/run_training.sh \
+    --config configs/training/dpo/llama3_3B_recipe.yaml \
+    --num-gpus 8 \
+    --nohup  # Run in background
+```
+
+### Inference
+
+Generate responses from your trained models:
+
+```bash
+# Run inference on trained model
+./scripts/run_inference.sh \
+    --config configs/inference/llama3_3B_recipe.yaml
+
+# Run inference with debug mode
+./scripts/run_inference.sh \
+    --config configs/inference/llama3_3B_recipe.yaml \
+    --debug
+```
+
+### Evaluation
+
+Evaluate model performance on benchmarks:
+
+```bash
+# Evaluate on MMLU benchmark
+./scripts/run_evaluation.sh \
+    --config configs/evaluation/mmlu/llama3_3B_recipe.yaml
+
+# Evaluate with debug output
+./scripts/run_evaluation.sh \
+    --config configs/evaluation/mmlu/llama3_3B_recipe.yaml \
+    --debug
+```
+
+-----
+
+## 📁 Project Structure
+
+```
+FAI-RL/
+├── core/                      # Core framework components
+├── trainers/                  # Training method implementations
+├── inference/                 # Inference components
+├── evaluations/               # Evaluation system
+├── configs/                   # Configuration files
+│   ├── training/              # Training configurations
+│   ├── inference/             # Inference configurations
+│   ├── evaluation/            # Evaluation configurations
+│   └── deepspeed/             # DeepSpeed ZeRO configurations
+├── utils/                     # Utility modules
+├── scripts/                   # Scripts
+├── logs/                      # Training logs (auto-generated)
+└── outputs/                   # Inference output (auto-generated)
+```
+
+-----
+
+## 🔗 Quick Links
+
+* **[Training Guide](./trainers/README.md)** - Comprehensive guide to configuring and running model training with detailed parameter explanations
+* **[Inference Guide](./inference/README.md)** - Running model inference and text generation
+* **[Evaluation Guide](./evaluations/README.md)** - Evaluating model performance on standard benchmarks
+
+## Algorithm Selection Guide
+
+Choose the right algorithm for your use case:
+
+| Algorithm | Best For | Requirements | Key Benefits |
+|-----------|----------|--------------|--------------|
+| **SFT** | Initial instruction tuning, domain adaptation | Prompt-response pairs | Simple, fast, establishes baseline |
+| **DPO** | Aligning to human preferences | Preference pairs (chosen/rejected) | No reward model needed, stable training |
+| **PPO** | Complex sequential tasks, agentic workflows | Preference pairs + reward model | Most flexible, handles multi-turn interactions |
+| **GRPO** | Math reasoning, efficiency-focused tasks | Question-answer pairs | No critic model, faster training |
+| **GSPO** | Multi-turn RL, stable sequence-level optimization | Question-answer pairs | Better stability than GRPO |
+
+## Memory Optimization
+
+FAI-RL supports various techniques to train large models efficiently:
+
+* **Full Fine-tuning:** Train all model parameters (requires most memory)
+* **LoRA:** Parameter-efficient training (~10% memory of full fine-tuning)
+* **QLoRA:** 4-bit quantized LoRA (train 7B+ models on single consumer GPU)
+* **DeepSpeed ZeRO-3:** Distributed training for models that don't fit on single GPU
+
+## 🧪 Tested Environment
+
+This framework has been validated on:
+
+* **Instance:** AWS EC2 p4d.24xlarge
+* **GPUs:** 8 x NVIDIA A100-SXM4-80GB (80GB VRAM each)
+* **CPU:** 96 vCPUs
+* **Memory:** 1152 GiB
+* **Storage:** 8TB NVMe SSD
+* **Network:** 400 Gbps

fai_rl-0.1.0/configs/__init__.py

@@ -0,0 +1 @@
+"""Configuration files and templates."""

fai_rl-0.1.0/configs/deepspeed/zero3_config_gpu1.json

@@ -0,0 +1,31 @@
+{
+    "zero_optimization": {
+        "stage": 3,
+        "offload_optimizer": {
+            "device": "cpu",
+            "pin_memory": true
+        },
+        "offload_param": {
+            "device": "cpu",
+            "pin_memory": true
+        },
+        "overlap_comm": true,
+        "contiguous_gradients": true,
+        "reduce_bucket_size": 5e8,
+        "stage3_prefetch_bucket_size": 5e8,
+        "stage3_param_persistence_threshold": 1e6,
+        "sub_group_size": 1e9,
+        "stage3_max_live_parameters": 1e9,
+        "stage3_max_reuse_distance": 1e9,
+        "stage3_gather_16bit_weights_on_model_save": true
+    },
+    "gradient_accumulation_steps": 16,
+    "gradient_clipping": 1.0,
+    "steps_per_print": 10,
+    "train_batch_size": 16,
+    "train_micro_batch_size_per_gpu": 1,
+    "wall_clock_breakdown": false,
+    "bf16": {
+        "enabled": true
+    }
+}