neural-compressor 2.2.1__tar.gz → 2.3__tar.gz

neural_compressor-2.3/PKG-INFO

@@ -0,0 +1,191 @@
+Metadata-Version: 2.1
+Name: neural_compressor
+Version: 2.3
+Summary: Repository of Intel® Neural Compressor
+Home-page: https://github.com/intel/neural-compressor
+Author: Intel AIA Team
+Author-email: feng.tian@intel.com, haihao.shen@intel.com, suyue.chen@intel.com
+License: Apache 2.0
+Keywords: quantization,auto-tuning,post-training static quantization,post-training dynamic quantization,quantization-aware training
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: License :: OSI Approved :: Apache Software License
+Requires-Python: >=3.6.0
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: deprecated>=1.2.13
+Requires-Dist: numpy
+Requires-Dist: opencv-python-headless
+Requires-Dist: pandas
+Requires-Dist: Pillow
+Requires-Dist: prettytable
+Requires-Dist: psutil
+Requires-Dist: py-cpuinfo
+Requires-Dist: pycocotools; sys_platform != "win32" or python_version > "3.8"
+Requires-Dist: pycocotools-windows; sys_platform == "win32" and python_version <= "3.8"
+Requires-Dist: pyyaml
+Requires-Dist: requests
+Requires-Dist: schema
+Requires-Dist: scikit-learn
+
+<div align="center">
+
+Intel® Neural Compressor
+===========================
+<h3> An open-source Python library supporting popular model compression techniques on all mainstream deep learning frameworks (TensorFlow, PyTorch, ONNX Runtime, and MXNet)</h3>
+
+[![python](https://img.shields.io/badge/python-3.7%2B-blue)](https://github.com/intel/neural-compressor)
+[![version](https://img.shields.io/badge/release-2.3-green)](https://github.com/intel/neural-compressor/releases)
+[![license](https://img.shields.io/badge/license-Apache%202-blue)](https://github.com/intel/neural-compressor/blob/master/LICENSE)
+[![coverage](https://img.shields.io/badge/coverage-85%25-green)](https://github.com/intel/neural-compressor)
+[![Downloads](https://static.pepy.tech/personalized-badge/neural-compressor?period=total&units=international_system&left_color=grey&right_color=green&left_text=downloads)](https://pepy.tech/project/neural-compressor)
+
+[Architecture](./docs/source/design.md#architecture)&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;[Workflow](./docs/source/design.md#workflow)&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;[Results](./docs/source/validated_model_list.md)&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;[Examples](./examples/README.md)&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;[Documentations](https://intel.github.io/neural-compressor)
+
+---
+<div align="left">
+
+Intel® Neural Compressor aims to provide popular model compression techniques such as quantization, pruning (sparsity), distillation, and neural architecture search on mainstream frameworks such as [TensorFlow](https://www.tensorflow.org/), [PyTorch](https://pytorch.org/), [ONNX Runtime](https://onnxruntime.ai/), and [MXNet](https://mxnet.apache.org/),
+as well as Intel extensions such as [Intel Extension for TensorFlow](https://github.com/intel/intel-extension-for-tensorflow) and [Intel Extension for PyTorch](https://github.com/intel/intel-extension-for-pytorch).
+In particular, the tool provides the key features, typical examples, and open collaborations as below:
+
+* Support a wide range of Intel hardware such as [Intel Xeon Scalable processor](https://www.intel.com/content/www/us/en/products/details/processors/xeon/scalable.html), [Intel Xeon CPU Max Series](https://www.intel.com/content/www/us/en/products/details/processors/xeon/max-series.html), [Intel Data Center GPU Flex Series](https://www.intel.com/content/www/us/en/products/details/discrete-gpus/data-center-gpu/flex-series.html), and [Intel Data Center GPU Max Series](https://www.intel.com/content/www/us/en/products/details/discrete-gpus/data-center-gpu/max-series.html) with extensive testing; support AMD CPU, ARM CPU, and NVidia GPU through ONNX Runtime with limited testing
+
+* Validate popular LLMs such as LLama2, [LLama](examples/onnxrt/nlp/huggingface_model/text_generation/llama/quantization/ptq_static), [MPT](https://github.com/intel/intel-extension-for-transformers/blob/main/examples/huggingface/pytorch/text-generation/quantization/README.md), [Falcon](https://github.com/intel/intel-extension-for-transformers/blob/main/examples/huggingface/pytorch/language-modeling/quantization/README.md), [GPT-J](/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/ptq_static/fx), [Bloom](/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/ptq_static/ipex/smooth_quant), [OPT](/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/ptq_static/ipex/smooth_quant), and more than 10,000 broad models such as [Stable Diffusion](/examples/pytorch/nlp/huggingface_models/text-to-image/quantization), [BERT-Large](/examples/pytorch/nlp/huggingface_models/text-classification/quantization/ptq_static/fx), and [ResNet50](/examples/pytorch/image_recognition/torchvision_models/quantization/ptq/cpu/fx) from popular model hubs such as [Hugging Face](https://huggingface.co/), [Torch Vision](https://pytorch.org/vision/stable/index.html), and [ONNX Model Zoo](https://github.com/onnx/models#models), by leveraging zero-code optimization solution [Neural Coder](/neural_coder#what-do-we-offer) and automatic [accuracy-driven](/docs/source/design.md#workflow) quantization strategies
+
+* Collaborate with cloud marketplace such as [Google Cloud Platform](https://console.cloud.google.com/marketplace/product/bitnami-launchpad/inc-tensorflow-intel?project=verdant-sensor-286207), [Amazon Web Services](https://aws.amazon.com/marketplace/pp/prodview-yjyh2xmggbmga#pdp-support), and [Azure](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/bitnami.inc-tensorflow-intel), software platforms such as [Alibaba Cloud](https://www.intel.com/content/www/us/en/developer/articles/technical/quantize-ai-by-oneapi-analytics-on-alibaba-cloud.html), [Tencent TACO](https://new.qq.com/rain/a/20221202A00B9S00) and [Microsoft Olive](https://github.com/microsoft/Olive), and open AI ecosystem such as [Hugging Face](https://huggingface.co/blog/intel), [PyTorch](https://pytorch.org/tutorials/recipes/intel_neural_compressor_for_pytorch.html), [ONNX](https://github.com/onnx/models#models), [ONNX Runtime](https://github.com/microsoft/onnxruntime), and [Lightning AI](https://github.com/Lightning-AI/lightning/blob/master/docs/source-pytorch/advanced/post_training_quantization.rst)
+
+## Installation
+
+### Install from pypi
+```Shell
+pip install neural-compressor
+```
+> More installation methods can be found at [Installation Guide](./docs/source/installation_guide.md). Please check out our [FAQ](./docs/source/faq.md) for more details.
+
+## Getting Started
+### Quantization with Python API
+
+```shell
+# Install Intel Neural Compressor and TensorFlow
+pip install neural-compressor
+pip install tensorflow
+# Prepare fp32 model
+wget https://storage.googleapis.com/intel-optimized-tensorflow/models/v1_6/mobilenet_v1_1.0_224_frozen.pb
+```
+```python
+from neural_compressor.data import DataLoader, Datasets
+from neural_compressor.config import PostTrainingQuantConfig
+
+dataset = Datasets("tensorflow")["dummy"](shape=(1, 224, 224, 3))
+dataloader = DataLoader(framework="tensorflow", dataset=dataset)
+
+from neural_compressor.quantization import fit
+
+q_model = fit(
+    model="./mobilenet_v1_1.0_224_frozen.pb",
+    conf=PostTrainingQuantConfig(),
+    calib_dataloader=dataloader,
+)
+```
+
+## Documentation
+
+<table class="docutils">
+  <thead>
+  <tr>
+    <th colspan="8">Overview</th>
+  </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td colspan="2" align="center"><a href="./docs/source/design.md#architecture">Architecture</a></td>
+      <td colspan="2" align="center"><a href="./docs/source/design.md#workflow">Workflow</a></td>
+      <td colspan="2" align="center"><a href="examples/README.md">Examples</a></td>
+      <td colspan="2" align="center"><a href="https://intel.github.io/neural-compressor/latest/docs/source/api-doc/apis.html">APIs</a></td>
+    </tr>
+  </tbody>
+  <thead>
+    <tr>
+      <th colspan="8">Python-based APIs</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+        <td colspan="2" align="center"><a href="./docs/source/quantization.md">Quantization</a></td>
+        <td colspan="2" align="center"><a href="./docs/source/mixed_precision.md">Advanced Mixed Precision</a></td>
+        <td colspan="2" align="center"><a href="./docs/source/pruning.md">Pruning (Sparsity)</a></td>
+        <td colspan="2" align="center"><a href="./docs/source/distillation.md">Distillation</a></td>
+    </tr>
+    <tr>
+        <td colspan="2" align="center"><a href="./docs/source/orchestration.md">Orchestration</a></td>
+        <td colspan="2" align="center"><a href="./docs/source/benchmark.md">Benchmarking</a></td>
+        <td colspan="2" align="center"><a href="./docs/source/distributed.md">Distributed Compression</a></td>
+        <td colspan="2" align="center"><a href="./docs/source/export.md">Model Export</a></td>
+    </tr>
+  </tbody>
+  <thead>
+    <tr>
+      <th colspan="8">Neural Coder (Zero-code Optimization)</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+        <td colspan="2" align="center"><a href="./neural_coder/docs/PythonLauncher.md">Launcher</a></td>
+        <td colspan="2" align="center"><a href="./neural_coder/extensions/neural_compressor_ext_lab/README.md">JupyterLab Extension</a></td>
+        <td colspan="2" align="center"><a href="./neural_coder/extensions/neural_compressor_ext_vscode/README.md">Visual Studio Code Extension</a></td>
+        <td colspan="2" align="center"><a href="./neural_coder/docs/SupportMatrix.md">Supported Matrix</a></td>
+    </tr>
+  </tbody>
+  <thead>
+      <tr>
+        <th colspan="8">Advanced Topics</th>
+      </tr>
+  </thead>
+  <tbody>
+      <tr>
+          <td colspan="2" align="center"><a href="./docs/source/adaptor.md">Adaptor</a></td>
+          <td colspan="2" align="center"><a href="./docs/source/tuning_strategies.md">Strategy</a></td>
+          <td colspan="2" align="center"><a href="./docs/source/distillation_quantization.md">Distillation for Quantization</a></td>
+          <td colspan="2" align="center"><a href="./docs/source/smooth_quant.md">SmoothQuant</td>
+      </tr>
+      <tr>
+          <td colspan="8" align="center"><a href="./docs/source/quantization_weight_only.md">Weight-Only Quantization (INT8/INT4/FP4/NF4) </td>
+      </tr>
+  </tbody>
+  <thead>
+      <tr>
+        <th colspan="8">Innovations for Productivity</th>
+      </tr>
+  </thead>
+  <tbody>
+      <tr>
+          <td colspan="4" align="center"><a href="./neural_insights/README.md">Neural Insights</a></td>
+          <td colspan="4" align="center"><a href="./neural_solution/README.md">Neural Solution</a></td>
+      </tr>
+  </tbody>
+</table>
+
+> More documentations can be found at [User Guide](./docs/source/user_guide.md).
+
+## Selected Publications/Events
+* arXiv: [Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs](https://arxiv.org/abs/2309.05516) (Sep 2023)
+* Post on Social Media: [ONNXCommunityMeetup2023: INT8 Quantization for Large Language Models with Intel Neural Compressor](https://www.youtube.com/watch?v=luYBWA1Q5pQ)  (July 2023)
+* Blog by Intel: [Accelerate Llama 2 with Intel AI Hardware and Software Optimizations](https://www.intel.com/content/www/us/en/developer/articles/news/llama2.html) (July 2023)
+* NeurIPS'2022: [Fast Distilbert on CPUs](https://arxiv.org/abs/2211.07715) (Oct 2022)
+* NeurIPS'2022: [QuaLA-MiniLM: a Quantized Length Adaptive MiniLM](https://arxiv.org/abs/2210.17114) (Oct 2022)
+
+> View [Full Publication List](./docs/source/publication_list.md).
+
+## Additional Content
+
+* [Release Information](./docs/source/releases_info.md)
+* [Contribution Guidelines](./docs/source/CONTRIBUTING.md)
+* [Legal Information](./docs/source/legal_information.md)
+* [Security Policy](SECURITY.md)
+
+## Communication 
+- [GitHub Issues](https://github.com/intel/neural-compressor/issues): mainly for bugs report, new feature request, question asking, etc.
+- [Email](mailto:inc.maintainers@intel.com): welcome to raise any interesting research ideas on model compression techniques by email for collaborations.  
+- [WeChat group](/docs/source/imgs/wechat_group.jpg): scan the QA code to join the technical discussion.

neural_compressor-2.3/README.md

@@ -0,0 +1,160 @@
+<div align="center">
+
+Intel® Neural Compressor
+===========================
+<h3> An open-source Python library supporting popular model compression techniques on all mainstream deep learning frameworks (TensorFlow, PyTorch, ONNX Runtime, and MXNet)</h3>
+
+[![python](https://img.shields.io/badge/python-3.7%2B-blue)](https://github.com/intel/neural-compressor)
+[![version](https://img.shields.io/badge/release-2.3-green)](https://github.com/intel/neural-compressor/releases)
+[![license](https://img.shields.io/badge/license-Apache%202-blue)](https://github.com/intel/neural-compressor/blob/master/LICENSE)
+[![coverage](https://img.shields.io/badge/coverage-85%25-green)](https://github.com/intel/neural-compressor)
+[![Downloads](https://static.pepy.tech/personalized-badge/neural-compressor?period=total&units=international_system&left_color=grey&right_color=green&left_text=downloads)](https://pepy.tech/project/neural-compressor)
+
+[Architecture](./docs/source/design.md#architecture)&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;[Workflow](./docs/source/design.md#workflow)&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;[Results](./docs/source/validated_model_list.md)&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;[Examples](./examples/README.md)&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;[Documentations](https://intel.github.io/neural-compressor)
+
+---
+<div align="left">
+
+Intel® Neural Compressor aims to provide popular model compression techniques such as quantization, pruning (sparsity), distillation, and neural architecture search on mainstream frameworks such as [TensorFlow](https://www.tensorflow.org/), [PyTorch](https://pytorch.org/), [ONNX Runtime](https://onnxruntime.ai/), and [MXNet](https://mxnet.apache.org/),
+as well as Intel extensions such as [Intel Extension for TensorFlow](https://github.com/intel/intel-extension-for-tensorflow) and [Intel Extension for PyTorch](https://github.com/intel/intel-extension-for-pytorch).
+In particular, the tool provides the key features, typical examples, and open collaborations as below:
+
+* Support a wide range of Intel hardware such as [Intel Xeon Scalable processor](https://www.intel.com/content/www/us/en/products/details/processors/xeon/scalable.html), [Intel Xeon CPU Max Series](https://www.intel.com/content/www/us/en/products/details/processors/xeon/max-series.html), [Intel Data Center GPU Flex Series](https://www.intel.com/content/www/us/en/products/details/discrete-gpus/data-center-gpu/flex-series.html), and [Intel Data Center GPU Max Series](https://www.intel.com/content/www/us/en/products/details/discrete-gpus/data-center-gpu/max-series.html) with extensive testing; support AMD CPU, ARM CPU, and NVidia GPU through ONNX Runtime with limited testing
+
+* Validate popular LLMs such as LLama2, [LLama](examples/onnxrt/nlp/huggingface_model/text_generation/llama/quantization/ptq_static), [MPT](https://github.com/intel/intel-extension-for-transformers/blob/main/examples/huggingface/pytorch/text-generation/quantization/README.md), [Falcon](https://github.com/intel/intel-extension-for-transformers/blob/main/examples/huggingface/pytorch/language-modeling/quantization/README.md), [GPT-J](/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/ptq_static/fx), [Bloom](/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/ptq_static/ipex/smooth_quant), [OPT](/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/ptq_static/ipex/smooth_quant), and more than 10,000 broad models such as [Stable Diffusion](/examples/pytorch/nlp/huggingface_models/text-to-image/quantization), [BERT-Large](/examples/pytorch/nlp/huggingface_models/text-classification/quantization/ptq_static/fx), and [ResNet50](/examples/pytorch/image_recognition/torchvision_models/quantization/ptq/cpu/fx) from popular model hubs such as [Hugging Face](https://huggingface.co/), [Torch Vision](https://pytorch.org/vision/stable/index.html), and [ONNX Model Zoo](https://github.com/onnx/models#models), by leveraging zero-code optimization solution [Neural Coder](/neural_coder#what-do-we-offer) and automatic [accuracy-driven](/docs/source/design.md#workflow) quantization strategies
+
+* Collaborate with cloud marketplace such as [Google Cloud Platform](https://console.cloud.google.com/marketplace/product/bitnami-launchpad/inc-tensorflow-intel?project=verdant-sensor-286207), [Amazon Web Services](https://aws.amazon.com/marketplace/pp/prodview-yjyh2xmggbmga#pdp-support), and [Azure](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/bitnami.inc-tensorflow-intel), software platforms such as [Alibaba Cloud](https://www.intel.com/content/www/us/en/developer/articles/technical/quantize-ai-by-oneapi-analytics-on-alibaba-cloud.html), [Tencent TACO](https://new.qq.com/rain/a/20221202A00B9S00) and [Microsoft Olive](https://github.com/microsoft/Olive), and open AI ecosystem such as [Hugging Face](https://huggingface.co/blog/intel), [PyTorch](https://pytorch.org/tutorials/recipes/intel_neural_compressor_for_pytorch.html), [ONNX](https://github.com/onnx/models#models), [ONNX Runtime](https://github.com/microsoft/onnxruntime), and [Lightning AI](https://github.com/Lightning-AI/lightning/blob/master/docs/source-pytorch/advanced/post_training_quantization.rst)
+
+## Installation
+
+### Install from pypi
+```Shell
+pip install neural-compressor
+```
+> More installation methods can be found at [Installation Guide](./docs/source/installation_guide.md). Please check out our [FAQ](./docs/source/faq.md) for more details.
+
+## Getting Started
+### Quantization with Python API
+
+```shell
+# Install Intel Neural Compressor and TensorFlow
+pip install neural-compressor
+pip install tensorflow
+# Prepare fp32 model
+wget https://storage.googleapis.com/intel-optimized-tensorflow/models/v1_6/mobilenet_v1_1.0_224_frozen.pb
+```
+```python
+from neural_compressor.data import DataLoader, Datasets
+from neural_compressor.config import PostTrainingQuantConfig
+
+dataset = Datasets("tensorflow")["dummy"](shape=(1, 224, 224, 3))
+dataloader = DataLoader(framework="tensorflow", dataset=dataset)
+
+from neural_compressor.quantization import fit
+
+q_model = fit(
+    model="./mobilenet_v1_1.0_224_frozen.pb",
+    conf=PostTrainingQuantConfig(),
+    calib_dataloader=dataloader,
+)
+```
+
+## Documentation
+
+<table class="docutils">
+  <thead>
+  <tr>
+    <th colspan="8">Overview</th>
+  </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td colspan="2" align="center"><a href="./docs/source/design.md#architecture">Architecture</a></td>
+      <td colspan="2" align="center"><a href="./docs/source/design.md#workflow">Workflow</a></td>
+      <td colspan="2" align="center"><a href="examples/README.md">Examples</a></td>
+      <td colspan="2" align="center"><a href="https://intel.github.io/neural-compressor/latest/docs/source/api-doc/apis.html">APIs</a></td>
+    </tr>
+  </tbody>
+  <thead>
+    <tr>
+      <th colspan="8">Python-based APIs</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+        <td colspan="2" align="center"><a href="./docs/source/quantization.md">Quantization</a></td>
+        <td colspan="2" align="center"><a href="./docs/source/mixed_precision.md">Advanced Mixed Precision</a></td>
+        <td colspan="2" align="center"><a href="./docs/source/pruning.md">Pruning (Sparsity)</a></td>
+        <td colspan="2" align="center"><a href="./docs/source/distillation.md">Distillation</a></td>
+    </tr>
+    <tr>
+        <td colspan="2" align="center"><a href="./docs/source/orchestration.md">Orchestration</a></td>
+        <td colspan="2" align="center"><a href="./docs/source/benchmark.md">Benchmarking</a></td>
+        <td colspan="2" align="center"><a href="./docs/source/distributed.md">Distributed Compression</a></td>
+        <td colspan="2" align="center"><a href="./docs/source/export.md">Model Export</a></td>
+    </tr>
+  </tbody>
+  <thead>
+    <tr>
+      <th colspan="8">Neural Coder (Zero-code Optimization)</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+        <td colspan="2" align="center"><a href="./neural_coder/docs/PythonLauncher.md">Launcher</a></td>
+        <td colspan="2" align="center"><a href="./neural_coder/extensions/neural_compressor_ext_lab/README.md">JupyterLab Extension</a></td>
+        <td colspan="2" align="center"><a href="./neural_coder/extensions/neural_compressor_ext_vscode/README.md">Visual Studio Code Extension</a></td>
+        <td colspan="2" align="center"><a href="./neural_coder/docs/SupportMatrix.md">Supported Matrix</a></td>
+    </tr>
+  </tbody>
+  <thead>
+      <tr>
+        <th colspan="8">Advanced Topics</th>
+      </tr>
+  </thead>
+  <tbody>
+      <tr>
+          <td colspan="2" align="center"><a href="./docs/source/adaptor.md">Adaptor</a></td>
+          <td colspan="2" align="center"><a href="./docs/source/tuning_strategies.md">Strategy</a></td>
+          <td colspan="2" align="center"><a href="./docs/source/distillation_quantization.md">Distillation for Quantization</a></td>
+          <td colspan="2" align="center"><a href="./docs/source/smooth_quant.md">SmoothQuant</td>
+      </tr>
+      <tr>
+          <td colspan="8" align="center"><a href="./docs/source/quantization_weight_only.md">Weight-Only Quantization (INT8/INT4/FP4/NF4) </td>
+      </tr>
+  </tbody>
+  <thead>
+      <tr>
+        <th colspan="8">Innovations for Productivity</th>
+      </tr>
+  </thead>
+  <tbody>
+      <tr>
+          <td colspan="4" align="center"><a href="./neural_insights/README.md">Neural Insights</a></td>
+          <td colspan="4" align="center"><a href="./neural_solution/README.md">Neural Solution</a></td>
+      </tr>
+  </tbody>
+</table>
+
+> More documentations can be found at [User Guide](./docs/source/user_guide.md).
+
+## Selected Publications/Events
+* arXiv: [Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs](https://arxiv.org/abs/2309.05516) (Sep 2023)
+* Post on Social Media: [ONNXCommunityMeetup2023: INT8 Quantization for Large Language Models with Intel Neural Compressor](https://www.youtube.com/watch?v=luYBWA1Q5pQ)  (July 2023)
+* Blog by Intel: [Accelerate Llama 2 with Intel AI Hardware and Software Optimizations](https://www.intel.com/content/www/us/en/developer/articles/news/llama2.html) (July 2023)
+* NeurIPS'2022: [Fast Distilbert on CPUs](https://arxiv.org/abs/2211.07715) (Oct 2022)
+* NeurIPS'2022: [QuaLA-MiniLM: a Quantized Length Adaptive MiniLM](https://arxiv.org/abs/2210.17114) (Oct 2022)
+
+> View [Full Publication List](./docs/source/publication_list.md).
+
+## Additional Content
+
+* [Release Information](./docs/source/releases_info.md)
+* [Contribution Guidelines](./docs/source/CONTRIBUTING.md)
+* [Legal Information](./docs/source/legal_information.md)
+* [Security Policy](SECURITY.md)
+
+## Communication 
+- [GitHub Issues](https://github.com/intel/neural-compressor/issues): mainly for bugs report, new feature request, question asking, etc.
+- [Email](mailto:inc.maintainers@intel.com): welcome to raise any interesting research ideas on model compression techniques by email for collaborations.  
+- [WeChat group](/docs/source/imgs/wechat_group.jpg): scan the QA code to join the technical discussion.

neural_compressor-2.3/neural_coder/__init__.py

@@ -0,0 +1,20 @@
+# Copyright (c) 2022 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from .interface import enable
+from .interface import bench
+from .interface import superbench
+
+# from .interface import superreport
+from .interface import auto_quant

neural_compressor-2.3/neural_coder/backends/nano_bf16.yaml

@@ -0,0 +1,27 @@
+# Copyright (c) 2022 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+transformation:
+  location:
+    - ["insert_below_model_definition_line", "insert_below_input_definition_line"]
+    - "indent_inference_line"
+  content:
+    - |-
+      [+] from bigdl.nano.pytorch import InferenceOptimizer
+      [+] MODEL_NAME = InferenceOptimizer.quantize(MODEL_NAME, precision="bf16", input_sample=INPUT_NAME)
+      [+] with InferenceOptimizer.get_context(MODEL_NAME):
+    - 1
+  order:
+    - below:
+      above:

neural_compressor-2.3/neural_coder/backends/nano_bf16_channels_last.yaml

@@ -0,0 +1,27 @@
+# Copyright (c) 2022 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+transformation:
+  location:
+    - ["insert_below_model_definition_line", "insert_below_input_definition_line"]
+    - "indent_inference_line"
+  content:
+    - |-
+      [+] from bigdl.nano.pytorch import InferenceOptimizer
+      [+] MODEL_NAME = InferenceOptimizer.quantize(MODEL_NAME, precision="bf16", channels_last=True, input_sample=INPUT_NAME)
+      [+] with InferenceOptimizer.get_context(MODEL_NAME):
+    - 1
+  order:
+    - below:
+      above:

neural_compressor-2.3/neural_coder/backends/nano_bf16_ipex.yaml

@@ -0,0 +1,27 @@
+# Copyright (c) 2022 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+transformation:
+  location:
+    - ["insert_below_model_definition_line", "insert_below_input_definition_line"]
+    - "indent_inference_line"
+  content:
+    - |-
+      [+] from bigdl.nano.pytorch import InferenceOptimizer
+      [+] MODEL_NAME = InferenceOptimizer.quantize(MODEL_NAME, precision="bf16", use_ipex=True, input_sample=INPUT_NAME)
+      [+] with InferenceOptimizer.get_context(MODEL_NAME):
+    - 1
+  order:
+    - below:
+      above:

neural_compressor-2.3/neural_coder/backends/nano_bf16_ipex_channels_last.yaml

@@ -0,0 +1,27 @@
+# Copyright (c) 2022 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+transformation:
+  location:
+    - ["insert_below_model_definition_line", "insert_below_input_definition_line"]
+    - "indent_inference_line"
+  content:
+    - |-
+      [+] from bigdl.nano.pytorch import InferenceOptimizer
+      [+] MODEL_NAME = InferenceOptimizer.quantize(MODEL_NAME, precision="bf16", use_ipex=True, channels_last=True, input_sample=INPUT_NAME)
+      [+] with InferenceOptimizer.get_context(MODEL_NAME):
+    - 1
+  order:
+    - below:
+      above:

neural_compressor-2.3/neural_coder/backends/nano_fp32_channels_last.yaml

@@ -0,0 +1,27 @@
+# Copyright (c) 2022 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+transformation:
+  location:
+    - ["insert_below_model_definition_line", "insert_below_input_definition_line"]
+    - "indent_inference_line"
+  content:
+    - |-
+      [+] from bigdl.nano.pytorch import InferenceOptimizer
+      [+] MODEL_NAME = InferenceOptimizer.trace(MODEL_NAME, channels_last=True, input_sample=INPUT_NAME)
+      [+] with InferenceOptimizer.get_context(MODEL_NAME):
+    - 1
+  order:
+    - below:
+      above:

neural_compressor-2.3/neural_coder/backends/nano_fp32_ipex.yaml

@@ -0,0 +1,27 @@
+# Copyright (c) 2022 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+transformation:
+  location:
+    - ["insert_below_model_definition_line", "insert_below_input_definition_line"]
+    - "indent_inference_line"
+  content:
+    - |-
+      [+] from bigdl.nano.pytorch import InferenceOptimizer
+      [+] MODEL_NAME = InferenceOptimizer.trace(MODEL_NAME, use_ipex=True, input_sample=INPUT_NAME)
+      [+] with InferenceOptimizer.get_context(MODEL_NAME):
+    - 1
+  order:
+    - below:
+      above:

neural_compressor-2.3/neural_coder/backends/nano_fp32_ipex_channels_last.yaml

@@ -0,0 +1,27 @@
+# Copyright (c) 2022 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+transformation:
+  location:
+    - ["insert_below_model_definition_line", "insert_below_input_definition_line"]
+    - "indent_inference_line"
+  content:
+    - |-
+      [+] from bigdl.nano.pytorch import InferenceOptimizer
+      [+] MODEL_NAME = InferenceOptimizer.trace(MODEL_NAME, use_ipex=True, channels_last=True, input_sample=INPUT_NAME)
+      [+] with InferenceOptimizer.get_context(MODEL_NAME):
+    - 1
+  order:
+    - below:
+      above:

neural_compressor-2.3/neural_coder/backends/nano_gpu_to_cpu.yaml

@@ -0,0 +1,24 @@
+# Copyright (c) 2022 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+transformation:
+  location:
+    - ["insert_above_model_definition_line", "insert_above_input_definition_line"]
+  content:
+    - |-
+      [+] from bigdl.nano.pytorch import patch_torch
+      [+] patch_torch()
+  order:
+    - below:
+      above:

neural_compressor-2.3/neural_coder/backends/nano_int8.yaml

@@ -0,0 +1,27 @@
+# Copyright (c) 2022 Intel Corporation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+transformation:
+  location:
+    - ["insert_below_model_definition_line", "insert_below_input_definition_line"]
+    - "indent_inference_line"
+  content:
+    - |-
+      [+] from bigdl.nano.pytorch import InferenceOptimizer
+      [+] MODEL_NAME = InferenceOptimizer.quantize(MODEL_NAME, precision="int8", input_sample=INPUT_NAME)
+      [+] with InferenceOptimizer.get_context(MODEL_NAME):
+    - 1
+  order:
+    - below:
+      above: