tico 0.1.0.dev251106__py3-none-any.whl → 0.2.0.dev260122__py3-none-any.whl

tico-0.2.0.dev260122.dist-info/METADATA

@@ -0,0 +1,631 @@
+Metadata-Version: 2.4
+Name: tico
+Version: 0.2.0.dev260122
+Summary: Convert Exported Torch Module To Circle
+License: This file provides full text of licenses used in this project
+        
+        - Apache License 2.0
+        - BSD 3-Clause
+        
+        ...............................................................................
+        
+                                      Apache License
+                                Version 2.0, January 2004
+                             http://www.apache.org/licenses/
+        
+        TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+        
+        1. Definitions.
+        
+           "License" shall mean the terms and conditions for use, reproduction,
+           and distribution as defined by Sections 1 through 9 of this document.
+        
+           "Licensor" shall mean the copyright owner or entity authorized by
+           the copyright owner that is granting the License.
+        
+           "Legal Entity" shall mean the union of the acting entity and all
+           other entities that control, are controlled by, or are under common
+           control with that entity. For the purposes of this definition,
+           "control" means (i) the power, direct or indirect, to cause the
+           direction or management of such entity, whether by contract or
+           otherwise, or (ii) ownership of fifty percent (50%) or more of the
+           outstanding shares, or (iii) beneficial ownership of such entity.
+        
+           "You" (or "Your") shall mean an individual or Legal Entity
+           exercising permissions granted by this License.
+        
+           "Source" form shall mean the preferred form for making modifications,
+           including but not limited to software source code, documentation
+           source, and configuration files.
+        
+           "Object" form shall mean any form resulting from mechanical
+           transformation or translation of a Source form, including but
+           not limited to compiled object code, generated documentation,
+           and conversions to other media types.
+        
+           "Work" shall mean the work of authorship, whether in Source or
+           Object form, made available under the License, as indicated by a
+           copyright notice that is included in or attached to the work
+           (an example is provided in the Appendix below).
+        
+           "Derivative Works" shall mean any work, whether in Source or Object
+           form, that is based on (or derived from) the Work and for which the
+           editorial revisions, annotations, elaborations, or other modifications
+           represent, as a whole, an original work of authorship. For the purposes
+           of this License, Derivative Works shall not include works that remain
+           separable from, or merely link (or bind by name) to the interfaces of,
+           the Work and Derivative Works thereof.
+        
+           "Contribution" shall mean any work of authorship, including
+           the original version of the Work and any modifications or additions
+           to that Work or Derivative Works thereof, that is intentionally
+           submitted to Licensor for inclusion in the Work by the copyright owner
+           or by an individual or Legal Entity authorized to submit on behalf of
+           the copyright owner. For the purposes of this definition, "submitted"
+           means any form of electronic, verbal, or written communication sent
+           to the Licensor or its representatives, including but not limited to
+           communication on electronic mailing lists, source code control systems,
+           and issue tracking systems that are managed by, or on behalf of, the
+           Licensor for the purpose of discussing and improving the Work, but
+           excluding communication that is conspicuously marked or otherwise
+           designated in writing by the copyright owner as "Not a Contribution."
+        
+           "Contributor" shall mean Licensor and any individual or Legal Entity
+           on behalf of whom a Contribution has been received by Licensor and
+           subsequently incorporated within the Work.
+        
+        2. Grant of Copyright License. Subject to the terms and conditions of
+           this License, each Contributor hereby grants to You a perpetual,
+           worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+           copyright license to reproduce, prepare Derivative Works of,
+           publicly display, publicly perform, sublicense, and distribute the
+           Work and such Derivative Works in Source or Object form.
+        
+        3. Grant of Patent License. Subject to the terms and conditions of
+           this License, each Contributor hereby grants to You a perpetual,
+           worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+           (except as stated in this section) patent license to make, have made,
+           use, offer to sell, sell, import, and otherwise transfer the Work,
+           where such license applies only to those patent claims licensable
+           by such Contributor that are necessarily infringed by their
+           Contribution(s) alone or by combination of their Contribution(s)
+           with the Work to which such Contribution(s) was submitted. If You
+           institute patent litigation against any entity (including a
+           cross-claim or counterclaim in a lawsuit) alleging that the Work
+           or a Contribution incorporated within the Work constitutes direct
+           or contributory patent infringement, then any patent licenses
+           granted to You under this License for that Work shall terminate
+           as of the date such litigation is filed.
+        
+        4. Redistribution. You may reproduce and distribute copies of the
+           Work or Derivative Works thereof in any medium, with or without
+           modifications, and in Source or Object form, provided that You
+           meet the following conditions:
+        
+           (a) You must give any other recipients of the Work or
+               Derivative Works a copy of this License; and
+        
+           (b) You must cause any modified files to carry prominent notices
+               stating that You changed the files; and
+        
+           (c) You must retain, in the Source form of any Derivative Works
+               that You distribute, all copyright, patent, trademark, and
+               attribution notices from the Source form of the Work,
+               excluding those notices that do not pertain to any part of
+               the Derivative Works; and
+        
+           (d) If the Work includes a "NOTICE" text file as part of its
+               distribution, then any Derivative Works that You distribute must
+               include a readable copy of the attribution notices contained
+               within such NOTICE file, excluding those notices that do not
+               pertain to any part of the Derivative Works, in at least one
+               of the following places: within a NOTICE text file distributed
+               as part of the Derivative Works; within the Source form or
+               documentation, if provided along with the Derivative Works; or,
+               within a display generated by the Derivative Works, if and
+               wherever such third-party notices normally appear. The contents
+               of the NOTICE file are for informational purposes only and
+               do not modify the License. You may add Your own attribution
+               notices within Derivative Works that You distribute, alongside
+               or as an addendum to the NOTICE text from the Work, provided
+               that such additional attribution notices cannot be construed
+               as modifying the License.
+        
+           You may add Your own copyright statement to Your modifications and
+           may provide additional or different license terms and conditions
+           for use, reproduction, or distribution of Your modifications, or
+           for any such Derivative Works as a whole, provided Your use,
+           reproduction, and distribution of the Work otherwise complies with
+           the conditions stated in this License.
+        
+        5. Submission of Contributions. Unless You explicitly state otherwise,
+           any Contribution intentionally submitted for inclusion in the Work
+           by You to the Licensor shall be under the terms and conditions of
+           this License, without any additional terms or conditions.
+           Notwithstanding the above, nothing herein shall supersede or modify
+           the terms of any separate license agreement you may have executed
+           with Licensor regarding such Contributions.
+        
+        6. Trademarks. This License does not grant permission to use the trade
+           names, trademarks, service marks, or product names of the Licensor,
+           except as required for reasonable and customary use in describing the
+           origin of the Work and reproducing the content of the NOTICE file.
+        
+        7. Disclaimer of Warranty. Unless required by applicable law or
+           agreed to in writing, Licensor provides the Work (and each
+           Contributor provides its Contributions) on an "AS IS" BASIS,
+           WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+           implied, including, without limitation, any warranties or conditions
+           of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+           PARTICULAR PURPOSE. You are solely responsible for determining the
+           appropriateness of using or redistributing the Work and assume any
+           risks associated with Your exercise of permissions under this License.
+        
+        8. Limitation of Liability. In no event and under no legal theory,
+           whether in tort (including negligence), contract, or otherwise,
+           unless required by applicable law (such as deliberate and grossly
+           negligent acts) or agreed to in writing, shall any Contributor be
+           liable to You for damages, including any direct, indirect, special,
+           incidental, or consequential damages of any character arising as a
+           result of this License or out of the use or inability to use the
+           Work (including but not limited to damages for loss of goodwill,
+           work stoppage, computer failure or malfunction, or any and all
+           other commercial damages or losses), even if such Contributor
+           has been advised of the possibility of such damages.
+        
+        9. Accepting Warranty or Additional Liability. While redistributing
+           the Work or Derivative Works thereof, You may choose to offer,
+           and charge a fee for, acceptance of support, warranty, indemnity,
+           or other liability obligations and/or rights consistent with this
+           License. However, in accepting such obligations, You may act only
+           on Your own behalf and on Your sole responsibility, not on behalf
+           of any other Contributor, and only if You agree to indemnify,
+           defend, and hold each Contributor harmless for any liability
+           incurred by, or claims asserted against, such Contributor by reason
+           of your accepting any such warranty or additional liability.
+        
+        END OF TERMS AND CONDITIONS
+        
+        APPENDIX: How to apply the Apache License to your work.
+        
+           To apply the Apache License to your work, attach the following
+           boilerplate notice, with the fields enclosed by brackets "[]"
+           replaced with your own identifying information. (Don't include
+           the brackets!)  The text should be enclosed in the appropriate
+           comment syntax for the file format. We also recommend that a
+           file or class name and description of purpose be included on the
+           same "printed page" as the copyright notice for easier
+           identification within third-party archives.
+        
+        
+        Copyright [yyyy] [name of copyright owner]
+        
+        Licensed under the Apache License, Version 2.0 (the "License");
+        you may not use this file except in compliance with the License.
+        You may obtain a copy of the License at
+        
+            http://www.apache.org/licenses/LICENSE-2.0
+        
+        Unless required by applicable law or agreed to in writing, software
+        distributed under the License is distributed on an "AS IS" BASIS,
+        WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+        See the License for the specific language governing permissions and
+        limitations under the License.
+        
+        ...............................................................................
+        
+        The BSD 3-Clause License
+        
+        Redistribution and use in source and binary forms, with or without
+        modification, are permitted provided that the following conditions are
+        met:
+        
+            * Redistributions of source code must retain the above copyright
+        notice, this list of conditions and the following disclaimer.
+            * Redistributions in binary form must reproduce the above
+        copyright notice, this list of conditions and the following disclaimer
+        in the documentation and/or other materials provided with the
+        distribution.
+            * Neither the name of Google Inc. nor the names of its
+        contributors may be used to endorse or promote products derived from
+        this software without specific prior written permission.
+        
+        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+        "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+        LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+        A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+        OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+        SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+        LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+        DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+        THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+        (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+        
+        .............................................................................
+        
+Requires-Python: >=3.10.0
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: circle-schema
+Requires-Dist: packaging
+Requires-Dist: cffi
+Requires-Dist: torch
+Requires-Dist: pyyaml
+Requires-Dist: tqdm
+Dynamic: license-file
+
+# TICO
+
+_TICO_ (Torch IR to Circle [ONE](https://github.com/Samsung/ONE)) is a python library for converting 
+ Pytorch modules into a circle model that is a lightweight and efficient representation in ONE 
+designed for optimized on-device neural network inference.
+
+## Table of Contents
+
+### For Users
+
+- [Installation](#installation)
+- [Getting Started](#getting-started)
+  - [From torch module](#from-torch-module)
+  - [From .pt2](#from-pt2)
+  - [Running circle models directly in Python](#running-circle-models-directly-in-python)
+  - [Quantization](#quantization)
+
+### For Developers
+
+- [Testing & Code Formatting](#testing--code-formatting)
+- [Testing](#testing)
+- [Code Formatting](#code-formatting)
+
+## For Users
+
+### Installation
+
+0. Prerequisites
+
+- Python 3.10
+- (Optional) [one-compiler 1.30.0](https://github.com/Samsung/ONE/releases/tag/1.30.0)
+  - It is only required if you intend to run inference with the converted Circle model. If you are only converting models without running them, this dependency is not needed.
+
+We highly recommend to use a virtual env, e.g., conda.
+
+1. Clone this repo
+
+2. Build python package
+
+```bash
+./ccex build
+```
+
+This will generate `build` and `dist` directories in the root directory.
+
+3. Install generated package
+
+```bash
+./ccex install
+```
+
+**Available options**
+- `--dist` To install the package from .whl (without this option, _TICO_ is installed in an editable mode)
+- `--torch_ver <torch version>` To install a specific torch version (default: 2.6).
+  - Available <torch version>: 2.5, 2.6, 2.7, 2.8, nightly
+
+4. Now you can convert a torch module to a `.circle`.
+
+### Getting started
+
+This tutorial explains how you can use _TICO_ to generate a circle model from a torch module. 
+
+Let's assume we have a torch module.
+
+```python
+import tico
+import torch
+
+class AddModule(torch.nn.Module):
+    def __init__(self):
+        super().__init__()
+
+    def forward(self, x, y):
+        return x + y
+```
+
+**NOTE**
+_TICO_ internally uses [torch.export](https://pytorch.org/docs/stable/export.html#torch-export).
+Therefore, the torch module must be 'export'able. Please see 
+[this document](https://pytorch.org/docs/stable/export.html#limitations-of-torch-export) 
+if you have any trouble to export.
+
+#### From torch module
+
+You can convert a torch module to a circle model with these steps.
+
+```python
+torch_module = AddModule()
+example_inputs = (torch.ones(4), torch.ones(4))
+
+circle_model = tico.convert(torch_module.eval(), example_inputs)
+circle_model.save('add.circle')
+```
+
+**NOTE**
+Please make sure to call `eval()` on the PyTorch module before passing it to our API.
+This ensures the model runs in inference mode, disabling layers like dropout and
+batch normalization updates.
+
+**Compile with configuration**
+
+```python
+from test.modules.op.add import AddWithCausalMaskFolded
+
+torch_module = AddWithCausalMaskFolded()
+example_inputs = torch_module.get_example_inputs()
+
+config = tico.CompileConfigV1()
+config.legalize_causal_mask_value = True
+circle_model = tico.convert(torch_module, example_inputs, config = config)
+circle_model.save('add_causal_mask_m120.circle')
+```
+
+With `legalize_causal_mask_value` option on, causal mask value is converted from 
+ -inf to -120, creating a more quantization-friendly circle model with the cost of 
+slight accuracy drop.
+
+#### From .pt2
+
+The torch module can be exported and saved as `.pt2` file (from PyTorch 2.1).
+
+```python
+module = AddModule()
+example_inputs = (torch.ones(4), torch.ones(4))
+
+exported_program = torch.export.export(module, example_inputs)
+torch.export.save(exported_program, 'add.pt2')
+```
+
+There are two ways to convert `.pt2` file: python api, command line tool.
+
+- Python API
+
+```python
+circle_model = tico.convert_from_pt2('add.pt2')
+circle_model.save('add.circle')
+```
+
+- Command Line Tool
+
+```bash
+pt2-to-circle -i add.pt2 -o add.circle
+```
+
+- Command Line Tool with configuration
+
+```bash
+pt2-to-circle -i add.pt2 -o add.circle -c config.yaml
+```
+
+```yaml
+# config.yaml
+
+version: '1.0' # You must specify the config version. 
+legalize_causal_mask_value: True
+```
+
+#### Running circle models directly in Python
+
+After circle export, you can run the model directly in Python.
+
+Note that you should install one-compiler package first.
+
+The output types are numpy.ndarray.
+
+```python
+torch_module = AddModule()
+example_inputs = (torch.ones(4), torch.ones(4))
+
+circle_model = tico.convert(torch_module, example_inputs)
+circle_model(*example_inputs)
+# numpy.ndarray([2., 2., 2., 2.], dtype=float32)
+```
+
+### Quantization
+
+The `tico.quantization` module provides a unified and modular interface for quantizing
+ large language models (LLMs) and other neural networks.
+ 
+It introduces a simple two-step workflow — **prepare** and **convert** — that
+ abstracts the details of different quantization algorithms.
+
+#### Basic Usage
+
+```python
+from tico.quantization import prepare, convert
+from tico.quantization.config.gptq import GPTQConfig
+import torch
+import torch.nn as nn
+
+class LinearModel(nn.Module):
+    def __init__(self):
+        super().__init__()
+        self.linear = nn.Linear(8, 8)
+
+    def forward(self, x):
+        return self.linear(x)
+
+model = LinearModel().eval()
+
+# 1. Prepare for quantization
+quant_config = GPTQConfig()
+prepared_model = prepare(model, quant_config)
+
+# 2. Calibration
+for d in dataset:
+    prepared_model(d)
+
+# 3. Apply GPTQ
+quantized_model = convert(prepared_model, quant_config)
+```
+
+For detailed documentation, design notes, and contributing guidelines, 
+see [tico/quantization/README.md](./tico/quantization/README.md).
+
+
+## For Developers
+
+### Testing & Code Formatting
+
+Run below commands to configure testing or formatting environment.
+
+Refer to the dedicated section to have more fine-grained control.
+
+```bash
+$ ./ccex configure                          # to set up testing & formatting environment
+$ ./ccex configure format                   # to set up only formatting environment
+$ ./ccex configure test                     # to set up only testing environment
+```
+
+**Available options**
+- `--torch_ver <torch version>` To install a specific torch family package(ex. torchvision) version (default: 2.6)
+  - Available <torch version>: '2.5', '2.6', 'nightly'
+
+```bash
+$ ./ccex configure                          # to set up testing & formatting environment with stable2.6.x version
+$ ./ccex configure test                     # to set up only testing environment with stable 2.6.x version
+$ ./ccex configure test --torch_ver 2.5     # to set up only testing environment with stable 2.5.x version
+$ ./ccex configure test --torch_ver nightly     # to set up only testing environment with nightly version
+```
+
+### Testing
+
+#### Test congifure
+
+Run below commands to install requirements for testing.
+
+**NOTE** `TICO` will be installed in an editable mode.
+
+```bash
+./ccex configure test
+
+# without editable install
+./ccex configure test --dist
+```
+
+#### Test All
+
+Run below commands to run the all unit tests.
+
+**NOTE** Unit tests don't include model test.
+
+```bash
+./ccex test
+# OR
+./ccex test run-all-tests
+```
+
+#### Test Subset
+
+To run subset of `test.modules.*`,
+Run `./ccex test -k <keyword>`
+
+
+For example, to run tests in specific sub-directory (op, net, ..)
+```bash
+# To run tests in specific sub-directory (op/, net/ ..)
+./ccex test -k op
+./ccex test -k net
+
+# To run tests in one file (single/op/add, single/op/sub, ...)
+./ccex test -k add
+./ccex test -k sub
+
+# To run SimpleAdd test in test/modules/single/op/add.py
+./ccex test -k SimpleAdd
+```
+
+To see the full debug log, add `-v` or `TICO_LOG=4`.
+
+```bash
+TICO_LOG=4 ./ccex test -k add
+# OR
+./ccex test -v -k add
+```
+
+#### Test Model
+
+If you want to test them locally, you can do so by navigating to each model directory, 
+ installing the dependencies listed in its `requirements.txt`, and running the tests one by one.
+```bash
+$ pip install -r test/modules/model/<model_name>/requirements.txt
+# Run test for a single model
+$ ./ccex test -m <model_name>
+# Run models whose names contain "Llama" (e.g., Llama, LlamaDecoderLayer, LlamaWithGQA, etc.)
+# Note that you should use quotes for the wildcard(*) pattern
+$ ./ccex test -m "Llama*"
+```
+
+For example, to run a single model
+```
+./ccex test -m InceptionV3
+```
+
+#### Runtime Options
+
+By default, `./ccex test` runs all modules with the `circle-interpreter` engine.
+ You can override this and run tests using the `onert` runtime instead.
+
+
+##### 0. Install ONERT
+
+```bash
+pip install onert
+```
+
+##### 1. Command-Line Flag
+
+Use the `--runtime` (or `-r`) flag to select a runtime:
+
+```bash
+# Run with the default circle-interpreter
+./ccex test
+
+# Run all tests with onert
+./ccex test --runtime onert
+# or
+./ccex test -r onert
+```
+
+##### 2. Environment Variable
+
+You can also set the `CCEX_RUNTIME` environment variable:
+
+```bash
+# Temporarily override for one command
+CCEX_RUNTIME=onert ./ccex test
+
+# Persist in your shell session
+export CCEX_RUNTIME=onert
+./ccex test
+```
+
+##### Supported Runtimes
+
+- circle-interpreter (default): uses the Circle interpreter for inference.
+- onert: uses the ONERT package for inference, useful when the Circle interpreter
+ cannot run a given module.
+
+### Code Formatting
+
+#### Format configure
+
+Run below commands to install requirements for formatting.
+
+```bash
+./ccex configure format
+```
+
+#### Format run
+
+```bash
+./ccex format
+```