@synsci/cli-darwin-x64 1.1.49

package/bin/skills/deepspeed/references/2023.md

@@ -0,0 +1,279 @@
+# Deepspeed - 2023
+
+**Pages:** 21
+
+---
+
+## DeepSpeed-VisualChat: Improve Your Chat Experience with Multi-Round Multi-Image Inputs
+
+**URL:** https://www.deepspeed.ai/2023/10/03/deepspeed-visualchat.html
+
+**Contents:**
+- DeepSpeed-VisualChat: Improve Your Chat Experience with Multi-Round Multi-Image Inputs
+    - Contents
+
+Updated: October 3, 2023
+
+---
+
+## DeepSpeed4Science：利用先进的AI系统优化技术实现科学发现
+
+**URL:** https://www.deepspeed.ai/2023/09/18/deepspeed4science-chinese.html
+
+**Contents:**
+- DeepSpeed4Science：利用先进的AI系统优化技术实现科学发现
+    - Contents
+
+Updated: September 18, 2023
+
+---
+
+## DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models
+
+**URL:** https://www.deepspeed.ai/2023/08/23/ulysses.html
+
+**Contents:**
+- DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models
+    - Contents
+
+Updated: August 23, 2023
+
+---
+
+## DeepSpeed Ulysses: 训练极长序列Transformer模型的系统优化
+
+**URL:** https://www.deepspeed.ai/2023/08/23/ulysses-chinese.html
+
+**Contents:**
+- DeepSpeed Ulysses: 训练极长序列Transformer模型的系统优化
+    - Contents
+
+Updated: August 23, 2023
+
+---
+
+## DeepSpeed Chat: 一键式RLHF训练，让你的类ChatGPT千亿大模型提速省钱15倍
+
+**URL:** https://www.deepspeed.ai/2023/04/23/deepspeed-chat-chinese.html
+
+**Contents:**
+- DeepSpeed Chat: 一键式RLHF训练，让你的类ChatGPT千亿大模型提速省钱15倍
+    - Contents
+
+Updated: April 23, 2023
+
+---
+
+## DeepSpeed ZeRO++: LLMやチャットモデルの訓練を劇的に高速化 – 通信オーバヘッドを1/4に大幅削減 -
+
+**URL:** https://www.deepspeed.ai/2023/06/21/zeropp-japanese.html
+
+**Contents:**
+- DeepSpeed ZeRO++: LLMやチャットモデルの訓練を劇的に高速化 – 通信オーバヘッドを1/4に大幅削減 -
+    - Contents
+
+Updated: June 21, 2023
+
+---
+
+## DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference
+
+**URL:** https://www.deepspeed.ai/2023/11/05/deepspeed-fastgen.html
+
+**Contents:**
+- DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference
+    - Contents
+
+Updated: November 5, 2023
+
+---
+
+## DeepSpeed-VisualChat: 複数ラウンド・複数画像の入力が可能なAIチャット体験を実現
+
+**URL:** https://www.deepspeed.ai/2023/10/03/deepspeed-visualchat-japanese.html
+
+**Contents:**
+- DeepSpeed-VisualChat: 複数ラウンド・複数画像の入力が可能なAIチャット体験を実現
+    - Contents
+
+Updated: October 3, 2023
+
+---
+
+## DeepSpeed-FastGen: MIIとDeepSpeed-InferenceによるLLMのための高速なテキスト生成
+
+**URL:** https://www.deepspeed.ai/2023/11/05/deepspeed-fastgen-japanese.html
+
+**Contents:**
+- DeepSpeed-FastGen: MIIとDeepSpeed-InferenceによるLLMのための高速なテキスト生成
+    - Contents
+
+Updated: November 5, 2023
+
+---
+
+## Zero Inference
+
+**URL:** https://www.deepspeed.ai/2023/09/12/ZeRO-Inference.html
+
+**Contents:**
+- Zero Inference
+    - Contents
+
+title: “ZeRO-Inference: 20X faster inference through weight quantization and KV cache offloading” excerpt: “” link: https://github.com/deepspeedai/DeepSpeedExamples/blob/master/inference/huggingface/zero_inference/README.md date: 2023-09-12 00:09:00 tags: inference ZeRO quantization English —
+
+Updated: September 12, 2023
+
+---
+
+## DeepSpeed Ulysses: Transformerモデルを非常に長いシーケンスで訓練するための最適化
+
+**URL:** https://www.deepspeed.ai/2023/08/23/ulysses-japanese.html
+
+**Contents:**
+- DeepSpeed Ulysses: Transformerモデルを非常に長いシーケンスで訓練するための最適化
+    - Contents
+
+Updated: August 23, 2023
+
+---
+
+## DeepSpeed-VisualChat：多轮图像+文字，为你展现不一样的AI聊天魅力
+
+**URL:** https://www.deepspeed.ai/2023/10/03/deepspeed-visualchat-chinese.html
+
+**Contents:**
+- DeepSpeed-VisualChat：多轮图像+文字，为你展现不一样的AI聊天魅力
+    - Contents
+
+Updated: October 3, 2023
+
+---
+
+## DeepSpeed ZeRO++: A leap in speed for LLM and chat model training with 4X less communication
+
+**URL:** https://www.deepspeed.ai/2023/06/21/zeropp.html
+
+**Contents:**
+- DeepSpeed ZeRO++: A leap in speed for LLM and chat model training with 4X less communication
+    - Contents
+
+Updated: June 21, 2023
+
+---
+
+## Announcing the DeepSpeed4Science Initiative: Enabling large-scale scientific discovery through sophisticated AI system technologies
+
+**URL:** https://www.deepspeed.ai/2023/09/18/deepspeed4science.html
+
+**Contents:**
+- Announcing the DeepSpeed4Science Initiative: Enabling large-scale scientific discovery through sophisticated AI system technologies
+    - Contents
+
+Updated: September 18, 2023
+
+---
+
+## Scaling Large-Scale Generative Mixture-of-Expert Multimodal Model With VL-MoE
+
+**URL:** https://www.deepspeed.ai/2023/03/30/multi-modal.html
+
+**Contents:**
+- Scaling Large-Scale Generative Mixture-of-Expert Multimodal Model With VL-MoE
+    - Contents
+
+The field of Artificial Intelligence-Generated Content (AIGC) is rapidly growing, with the goal of making content creation more efficient and accessible. One of the most exciting areas of AIGC is the development of large-scale multi-modal models like Flamingo, BLIP, and GPT4, which can accept inputs from multiple resources, e.g., image, text, audio, etc., and generate a variety of formats as outputs. For example, image creation can be made through stable diffusion and DALLE using the prompt text, and the new feature in the coming Office can create slides with texts, images, animations, etc., by leveraging the power of the new Microsoft Office Copilot.
+
+Scaling up the model size is one common approach to boost usability and capability of AIGC tasks. However, simply scaling up dense architectures (e.g., from GPT-1 to GPT-3) is usually extremely resource-intensive and time-consuming for both model training and inference. One effective way to tackle this challenge is to apply mixture of experts (MoE). In particular, recent text-based MoE and vision-based MoE studies have demonstrated that MoE models can significantly reduce the training and resource cost as compared to a quality-equivalent dense model, or produce a higher quality model under the same training budget. Up to now, the effectiveness of jointly training MoE for multi-modal models remains not well understood. To explore this important capability, DeepSpeed team is proud to announce our first large-scale generative mixture-of-expert (MoE) multimodal model, named VL-MoE.
+
+Figure 1: The new encoding process in our VL-MoE for various modality inputs, for which gray and colored blocks indicate non-activated and activated modules, respectively.
+
+Specifically, we incorporate the MoE structure into the classical single-tower multi-modal model by comprising of the following components: (1) a shared self-attention module across modalities, (2) a pool of modality-specific experts in the feed-forward network (FFN), and (3) a sparse gated MoE extended from the dense FFN. Subsequently, under the same amount of training resources as that used in VLMO (200k training steps), we demonstrate VL-MoE’s advantages over the state-of-the-art dense counterparts in the following two aspects:
+
+(1) VL-MoE can achieve significant accuracy improvement in comparison to its dense counterparts. Table 1 demonstrates that under the same training budget (i.e., have the same number of activated parameters for each token), VL-MoE Base with 32 experts achieves better accuracy than the VLMO-Base dense model on all four vision-language datasets.
+
+(2) VL-MoE achieves similar model quality with a much smaller activated number of parameters compared to its dense counterparts. Our results show that the finetuning performance of our VL-MoE is similar to that of the 3.1X larger VLMO-Large dense model (i.e., 3.1X more activated number of parameters per token). This can directly translate to approximately 3.1X training cost reduction as the training FLOPs for transformers are proportional to the activated model size per token.
+
+Table 1: Comparison of finetuning accuracy results for different models used in vision-language classification tasks and image-text retrieval tasks.
+
+A sophisticated MoE model design requires a highly efficient and scalable training system that can support multi-dimensional parallelism and efficient memory management. DeepSpeed MoE training system offers such advanced capabilities including easy-to-use APIs enabling flexible combinations of data, tensor, and expert parallelism. Furthermore, DeepSpeed MoE enables larger model scale than state-of-the-art systems by exploiting expert parallelism and ZeRO optimizations together. By leveraging the DeepSpeed MoE system, VL-MoE Base with 32 experts achieves similar model quality as VLMO-dense Large with about 2.5x training speedup.
+
+DeepSpeed MoE system is already open-sourced and can be easily used as plug-and-play component to achieve high-performance low-cost training for any large-scale MoE models. The tutorial of how to use DeepSpeed MoE is available here. VL-MoE is currently in the process of being integrated as a model example of DeepSpeed Examples. Please stay tuned for our upcoming updates on this thread.
+
+Updated: March 30, 2023
+
+---
+
+## DeepSpeed-FastGen：通过 MII 和 DeepSpeed-Inference 实现 LLM 高吞吐量文本生成
+
+**URL:** https://www.deepspeed.ai/2023/11/05/deepspeed-fastgen-chinese.html
+
+**Contents:**
+- DeepSpeed-FastGen：通过 MII 和 DeepSpeed-Inference 实现 LLM 高吞吐量文本生成
+    - Contents
+
+Updated: November 5, 2023
+
+---
+
+## DeepSpeed4Scienceイニシアティブ: 洗練されたAIシステムのテクノロジーにより大規模な科学的発見を可能に
+
+**URL:** https://www.deepspeed.ai/2023/09/18/deepspeed4science-japanese.html
+
+**Contents:**
+- DeepSpeed4Scienceイニシアティブ: 洗練されたAIシステムのテクノロジーにより大規模な科学的発見を可能に
+    - Contents
+
+Updated: September 18, 2023
+
+---
+
+## DeepSpeed Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales
+
+**URL:** https://www.deepspeed.ai/2023/04/23/deepspeed-chat.html
+
+**Contents:**
+- DeepSpeed Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales
+    - Contents
+
+Updated: April 23, 2023
+
+---
+
+## DeepSpeed ZeRO++：降低4倍网络通信，显著提高大模型及类ChatGPT模型训练效率
+
+**URL:** https://www.deepspeed.ai/2023/06/21/zeropp-chinese.html
+
+**Contents:**
+- DeepSpeed ZeRO++：降低4倍网络通信，显著提高大模型及类ChatGPT模型训练效率
+    - Contents
+
+Updated: June 21, 2023
+
+---
+
+## DeepSpeed主要技術の概要紹介
+
+**URL:** https://www.deepspeed.ai/2023/06/06/deepspeed-overview-japanese.html
+
+**Contents:**
+- DeepSpeed主要技術の概要紹介
+    - Contents
+
+我々が研究開発しているDeepSpeedについて、主要技術を日本語で説明した資料を公開しました。GPT3やChatGPTのような生成型AIのための大規模言語モデルを含む、様々な深層学習の訓練や推論に容易に適用でき、モデルの大規模化、高速化、コスト削減を可能にします。こちらよりダウンロードしてください。
+
+Updated: June 6, 2023
+
+---
+
+## DeepSpeed Chat: ChatGPTライクなモデルを簡単・高速・低コストに、あらゆるスケールで学習
+
+**URL:** https://www.deepspeed.ai/2023/04/23/deepspeed-chat-japanese.html
+
+**Contents:**
+- DeepSpeed Chat: ChatGPTライクなモデルを簡単・高速・低コストに、あらゆるスケールで学習
+    - Contents
+
+Updated: April 23, 2023
+
+---

package/bin/skills/deepspeed/references/assets.md

@@ -0,0 +1,179 @@
+# Deepspeed - Assets
+
+**Pages:** 29
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/zero1_dp8_1.5B_log.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/mii/bert.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/files/DeepSpeed_Overview_Japanese_2023Jun7th.pdf
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/zero_offload_dp1_10B_smi.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/zero3-offload-512-v100.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/data_efficiency/data_efficiecy_fig1.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/zeropp/ZeRO-baseline.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/mii/azure-cost.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/data_efficiency/data_efficiecy_fig0.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/zero3-offload-200B-scalability.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/mii/hero.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/zero_offload_dp1_10B_log.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/zero2_dp32_10B_smi.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/data_efficiency/data_efficiecy_fig3.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/mii/roberta.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/zero_offload_dp1_10B_cpu.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/oom_dp8_1.5B_log.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/data_efficiency/data_efficiecy_fig2.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/vl_moe.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/zero3-offload-1-v100.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/zero3-offload-memory-overview.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/mii/opt-bloom.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/zero1_dp8_1.5B_smi.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/mii/tput-llms.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/mii/llm-latency-sd-latency-zoom.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/mii/gpt.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/zeropp/ZeROpp.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/mii/mii-arch.png
+
+---
+
+## 
+
+**URL:** https://www.deepspeed.ai/assets/images/zero2_dp32_10B_log.png
+
+---

package/bin/skills/deepspeed/references/index.md

@@ -0,0 +1,35 @@
+# Deepspeed Documentation Index
+
+## Categories
+
+### 08
+**File:** `08.md`
+**Pages:** 1
+
+### 09
+**File:** `09.md`
+**Pages:** 2
+
+### 2020
+**File:** `2020.md`
+**Pages:** 16
+
+### 2023
+**File:** `2023.md`
+**Pages:** 21
+
+### Assets
+**File:** `assets.md`
+**Pages:** 29
+
+### Mii
+**File:** `mii.md`
+**Pages:** 1
+
+### Other
+**File:** `other.md`
+**Pages:** 15
+
+### Tutorials
+**File:** `tutorials.md`
+**Pages:** 59

package/bin/skills/deepspeed/references/mii.md

@@ -0,0 +1,118 @@
+# Deepspeed - Mii
+
+**Pages:** 1
+
+---
+
+## DeepSpeed-MII: instant speedup on 24,000+ open-source DL models with up to 40x cheaper inference
+
+**URL:** https://www.deepspeed.ai/2022/10/10/mii.html
+
+**Contents:**
+- DeepSpeed-MII: instant speedup on 24,000+ open-source DL models with up to 40x cheaper inference
+    - Contents
+- How does MII work?
+- Supported Models and Tasks
+- Inference Optimizations with MII
+- MII-Public and MII-Azure
+- Quantifying Latency and Cost Reduction
+- Latency Critical Scenarios
+- Cost Sensitive Scenarios
+- Deployment Options
+
+The Deep Learning (DL) open-source community has seen tremendous growth in the last few months. Incredibly powerful text generation models such as the Bloom 176B, or image generation models such as Stable Diffusion are now available to anyone with access to a handful or even a single GPU through platforms such as Hugging Face. While open-sourcing has democratized access to AI capabilities, their application is still restricted by two critical factors: 1) inference latency and 2) cost.
+
+There has been significant progress in system optimizations for DL model inference that can drastically reduce both latency and cost, but those are not easily accessible. The main reason for this limited accessibility is that the DL model inference landscape is diverse with models varying in size, architecture, system performance characteristics, hardware requirements, etc. Identifying the appropriate set of system optimizations applicable to a given model and applying them correctly is often beyond the scope of most data scientists, making low latency and low-cost inference mostly inaccessible.
+
+DeepSpeed Model Implementations for Inference (MII) is a new open-source python library from DeepSpeed, aimed towards making low-latency, low-cost inference of powerful models not only feasible but also easily accessible.
+
+Figure 1: MII Architecture, showing how MII automatically optimizes OSS models using DS-Inference before deploying them on-premises using GRPC, or on Microsoft Azure using AML Inference.
+
+Under-the-hood MII is powered by DeepSpeed-Inference. Based on the model type, model size, batch size, and available hardware resources, MII automatically applies the appropriate set of system optimizations from DeepSpeed-Inference to minimize latency and maximize throughput. It does so by using one of many pre-specified model injection policies, that allows MII and DeepSpeed-Inference to identify the underlying PyTorch model architecture and replace it with an optimized implementation (see Figure 1). In doing so, MII makes the expansive set of optimizations in DeepSpeed-Inference automatically available for thousands of popular models that it supports.
+
+MII supports a growing list of tasks such as text generation, question-answering, text classification, etc, across thousands of transformer models available through multiple open-sourced model repositories such as Hugging Face, FairSeq, EluetherAI, etc. It supports dense models based on BERT, RoBERTa, GPT, OPT, and BLOOM architectures ranging from a few hundred million parameters in size to hundreds of billions of parameters in size. At the same time, it supports recent image generation models such as Stable Diffusion.
+
+See the MII GitHub repo for an up-to-date list of models and tasks supported by MII.
+
+Here we provide a summary of the expansive set of optimizations from DeepSpeed-inference made available via MII. For more details, please refer to [1, 2]:
+
+DeepFusion for Transformers: For transformer-based models such as Bert, Roberta, GPT-2, and GPT-J, MII leverages the transformer kernels in DeepSpeed-Inference that are optimized to achieve low latency at small batch sizes and high throughput at large batch sizes using DeepFusion.
+
+Multi-GPU Inference with Tensor-Slicing: For massive models such as Bloom 176B, MII automatically enables tensor-parallelism within a node to leverage aggregate memory bandwidth and compute across multiple GPUs to achieve the lowest latency and throughput compared to anything else that is currently available.
+
+INT8 Inference with ZeroQuant: For massive models with tens or hundreds of billions of parameters, MII supports INT8 Inference with ZeroQuant. Using this feature not only reduces the memory footprint and the number of GPUs required for inference but also increases the inference throughput by supporting larger batch sizes and using INT8 compute, thus lowering cost compared to FP16.
+
+ZeRO-Inference for Resource Constrained Systems: Models such as Bloom 176B, require over 176 GB of memory to just fit the model even with INT8 support. In the absence of the aggregate GPU memory across multiple GPUs required to deploy such models, MII enables ZeRO-Inference that can leverage the system CPU memory to deploy these massive models with a single GPU with limited memory.
+
+Compiler Optimizations: When applicable, MII automatically applies compiler-based optimizations via TorchScript, nvFuser, and CUDA graph, in addition to the above optimizations, to further lower latency and improve throughput.
+
+MII can work with two variations of DeepSpeed-Inference. The first, referred to as ds-public, contains most of the optimizations discussed above and is also available via our open-source DeepSpeed library. The second referred to as ds-azure, offers tighter integration with Azure, and is available via MII to all Microsoft Azure customers. We refer to MII running the two DeepSpeed-Inference variants as MII-Public and MII-Azure, respectively.
+
+Both MII-Public and MII-Azure offer significant latency and cost reduction compared to open-sourced PyTorch implementation (Baseline). However for certain generative workloads, they can have differentiated performance: MII-Azure provides further improvements beyond MII-Public. We quantify the latency and cost reduction for both variations in the next section.
+
+Inference workloads can be either latency critical, where the primary objective is to minimize latency, or cost sensitive, where the primary objective is to minimize cost. In this section, we quantify the benefits of using MII for both latency-critical and cost-sensitive scenarios.
+
+For latency-critical scenarios, where a small batch size of 1 is often used, MII can reduce the latency by up to 6x for a wide range of open-source models, across multiple tasks. More specifically, we show model latency reduction of 1:
+
+Up to 5.7x for multi-GPU inference for text generation using massive models such as Big Science Bloom, Facebook OPT, and EluetherAI NeoX (Figure 2 (left))
+
+Up to 1.9x for image generation tasks model using Stable Diffusion (Figure 2 (right))
+
+Up to 3x for relatively smaller text generation models (up to 7B parameters) based on OPT, BLOOM, and GPT architectures, running on a single GPU (Figures 3 and 4)
+
+Up to 9x for various text representation tasks like fill-mask, text classification, question answering, and token classification using RoBERTa- and BERT- based models (Figures 5 and 6).
+
+Figure 2: (left) Best achievable latency for large models. MII-Azure (int8) offers 5.7X lower latency compared to Baseline for Bloom-176B. (right) Stable Diffusion text to image generation latency comparison.
+
+Figure 3: Latency comparison for OPT and BLOOM models. MII-Azure is up to 2.8x faster than baseline.
+
+Figure 4: Latency comparison for GPT models. MII-Azure is up to 3x faster than baseline.
+
+Figure 5: Latency comparison for RoBERTa models. MII offers up to 9x lower model latency and up to 3x lower end-to-end latency than baseline on several tasks and RoBERTa variants 1.
+
+Figure 6: Latency comparison for BERT models. MII offers up to 8.9x lower model latency and up to 4.5x end-to-end latency across several tasks and BERT variants1.
+
+MII can significantly reduce the inference cost of very expensive language models like Bloom, OPT, etc. To get the lowest cost, we use a large batch size that maximizes throughput for both baseline and MII. Here we look at the cost reduction from MII using two different metrics: i) tokens generated per second per GPU, and ii) dollars per million tokens generated.
+
+Figures 7 and 8 show that MII-Public offers over 10x throughput improvement and cost reduction compared to the baseline, respectively. Furthermore, MII-Azure offers over 30x improvement in throughput and cost compared to the baseline.
+
+Figure 7: Throughput comparison per A100-80GB GPU for large models. MII-Public offers over 15x throughput improvement while MII-Azure offers over 40x throughput improvement.
+
+Figure 8: Cost of generating 1 million tokens on Azure with different model types. MII-Azure reduces the cost of generation by over 40x.
+
+MII supported models can be deployed in two different ways as shown in Figure 1 with just a few lines of code.
+
+MII-Public can be deployed on-premises or on any cloud offering. MII creates a lightweight GRPC server to support this form of deployment and provides a GRPC inference endpoint for queries. The code below shows how a supported model can be deployed with MII-Public Deployment.
+
+MII supports deployment on Azure via AML Inference. To enable this, MII generates AML deployment assets for a given model that can be deployed using the Azure-CLI, as shown in the code below. Furthermore, deploying on Azure, allows MII to leverage DeepSpeed-Azure as its optimization backend, which offers better latency and cost reduction than DeepSpeed-Public.
+
+To learn more about these deployment options and get started with MII, please the MII getting started guide.
+
+We are very excited to share MII with the community and improve it with your feedback. We will continue to add support for more models in MII as well as enhance both MII-Public and MII-Azure for both on-premise and Azure users. Our hope is that while open sourcing has made powerful AI capabilities accessible to many, MII will allow for a wider infusion of these capabilities into a diverse set of applications and product offerings by instantly reducing the latency and cost of inferencing.
+
+The table below shows the mapping between model aliases used in Figures 3, 4, 5, and 6 and real model names.
+
+The end-to-end latency of an inference workload is comprised of two components: i) actual model execution, and ii) pre-/post-processing before and after the model execution. MII optimizes the actual model execution but leaves the pre-/post-processing pipeline for future optimizations. We notice that text representation tasks have significant pre-/post-processing overhead (Figures G and H). We plan to address those in a future update. ↩ ↩2 ↩3
+
+Updated: October 10, 2022
+
+**Examples:**
+
+Example 1 (unknown):
+```unknown
+import mii
+mii.deploy(task="text-to-image",
+           model="CompVis/stable-diffusion-v1-4",
+           deployment_name="sd-deployment")
+```
+
+Example 2 (unknown):
+```unknown
+import mii
+mii.deploy(task="text-to-image",
+           model="CompVis/stable-diffusion-v1-4",
+           deployment_name="sd-deployment",
+           deployment_type=DeploymentType.AML)
+```
+
+---