tico 0.2.0.dev260701__tar.gz → 0.2.0.dev260702__tar.gz

{tico-0.2.0.dev260701 → tico-0.2.0.dev260702}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: tico
-Version: 0.2.0.dev260701
+Version: 0.2.0.dev260702
 Summary: Convert Exported Torch Module To Circle
 License: This file provides full text of licenses used in this project
         
@@ -257,91 +257,62 @@ Dynamic: license-file
 
 # TICO
 
-_TICO_ (Torch IR to Circle [ONE](https://github.com/Samsung/ONE)) is a python library for converting 
- Pytorch modules into a circle model that is a lightweight and efficient representation in ONE 
-designed for optimized on-device neural network inference.
+_TICO_ (**T**orch **I**R to **C**ircle [ONE](https://github.com/Samsung/ONE)) is a Python
+library that converts PyTorch modules into Circle models — a lightweight and efficient
+representation in ONE designed for optimized on-device neural network inference.
 
-## Table of Contents
+## Highlights
 
-### For Users
+- **One-call conversion** — `tico.convert()` turns an `nn.Module` into a ready-to-deploy `.circle` binary.
+- **`.pt2` support** — convert saved `torch.export` programs via the Python API or the `pt2-to-circle` command-line tool.
+- **Run Circle models in Python** — execute converted models directly for quick parity checks against PyTorch.
+- **Quantization toolkit** — a unified `prepare`/`convert` API with GPTQ, PTQ (WrapQ), SmoothQuant, SpinQuant, and CLE, plus config-driven CLI recipes for LLMs and VLMs.
 
-- [Installation](#installation)
-- [Getting Started](#getting-started)
-  - [From torch module](#from-torch-module)
-  - [From .pt2](#from-pt2)
-  - [Running circle models directly in Python](#running-circle-models-directly-in-python)
-  - [Quantization](#quantization)
+## Installation
 
-### For Developers
+### Prerequisites
 
-- [Testing & Code Formatting](#testing--code-formatting)
-- [Testing](#testing)
-- [Code Formatting](#code-formatting)
+- Python **3.10+**
+- (Optional) [one-compiler](https://github.com/Samsung/ONE/releases) — only required to run
+  inference with converted Circle models. Conversion itself does not need it.
 
-## For Users
+We highly recommend using a virtual environment (e.g., conda, venv).
 
-### Installation
-
-0. Prerequisites
-
-- Python 3.10
-- (Optional) [one-compiler 1.30.0](https://github.com/Samsung/ONE/releases/tag/1.30.0)
-  - It is only required if you intend to run inference with the converted Circle model. If you are only converting models without running them, this dependency is not needed.
-
-We highly recommend to use a virtual env, e.g., conda.
-
-1. Clone this repo
-
-2. Build python package
+### From PyPI
 
 ```bash
-./ccex build
+pip install tico
 ```
 
-This will generate `build` and `dist` directories in the root directory.
-
-3. Install generated package
+### From source
 
 ```bash
-./ccex install
-```
-
-**Available options**
-- `--dist` To install the package from .whl (without this option, _TICO_ is installed in an editable mode)
-- `--torch_ver <torch version>` To install a specific torch version (default: 2.6).
-  - Available <torch version>: 2.5, 2.6, 2.7, 2.8, nightly
+git clone https://github.com/Samsung/TICO.git
+cd TICO
 
-4. Now you can convert a torch module to a `.circle`.
+./ccex build     # generates build/ and dist/
+./ccex install   # installs the package
+```
 
-### Getting started
+**`./ccex install` options**
 
-This tutorial explains how you can use _TICO_ to generate a circle model from a torch module. 
+| Option | Description |
+|---|---|
+| `--dist` | Install from the built wheel (default is editable mode) |
+| `--torch_ver <ver>` | Torch version to install: a family (`2.5` ~ `2.10`), an exact version (e.g. `2.7.0+cu118`), or `nightly`. Default: `2.7` |
+| `--cuda_ver <maj.min>` | Override the detected CUDA version (e.g. `12.1`) |
+| `--cpu_only` | Force a CPU-only Torch installation |
 
-Let's assume we have a torch module.
+## Quick start
 
 ```python
 import tico
 import torch
 
 class AddModule(torch.nn.Module):
-    def __init__(self):
-        super().__init__()
-
     def forward(self, x, y):
         return x + y
-```
-
-**NOTE**
-_TICO_ internally uses [torch.export](https://pytorch.org/docs/stable/export.html#torch-export).
-Therefore, the torch module must be 'export'able. Please see 
-[this document](https://pytorch.org/docs/stable/export.html#limitations-of-torch-export) 
-if you have any trouble to export.
 
-#### From torch module
-
-You can convert a torch module to a circle model with these steps.
-
-```python
 torch_module = AddModule()
 example_inputs = (torch.ones(4), torch.ones(4))
 
@@ -349,290 +320,69 @@ circle_model = tico.convert(torch_module.eval(), example_inputs)
 circle_model.save('add.circle')
 ```
 
-**NOTE**
-Please make sure to call `eval()` on the PyTorch module before passing it to our API.
-This ensures the model runs in inference mode, disabling layers like dropout and
-batch normalization updates.
-
-**Compile with configuration**
-
-```python
-from test.modules.op.add import AddWithCausalMaskFolded
-
-torch_module = AddWithCausalMaskFolded()
-example_inputs = torch_module.get_example_inputs()
-
-config = tico.CompileConfigV1()
-config.legalize_causal_mask_value = True
-circle_model = tico.convert(torch_module, example_inputs, config = config)
-circle_model.save('add_causal_mask_m120.circle')
-```
-
-With `legalize_causal_mask_value` option on, causal mask value is converted from 
- -inf to -120, creating a more quantization-friendly circle model with the cost of 
-slight accuracy drop.
-
-#### From .pt2
-
-The torch module can be exported and saved as `.pt2` file (from PyTorch 2.1).
-
-```python
-module = AddModule()
-example_inputs = (torch.ones(4), torch.ones(4))
-
-exported_program = torch.export.export(module, example_inputs)
-torch.export.save(exported_program, 'add.pt2')
-```
-
-There are two ways to convert `.pt2` file: python api, command line tool.
-
-- Python API
-
-```python
-circle_model = tico.convert_from_pt2('add.pt2')
-circle_model.save('add.circle')
-```
+> [!NOTE]
+> Call `eval()` on the module before conversion. _TICO_ internally uses
+> [`torch.export`](https://pytorch.org/docs/stable/export.html#torch-export), so the module
+> must be export-able.
 
-- Command Line Tool
+Converting a saved `.pt2` file from the command line:
 
 ```bash
 pt2-to-circle -i add.pt2 -o add.circle
 ```
 
-- Command Line Tool with configuration
-
-```bash
-pt2-to-circle -i add.pt2 -o add.circle -c config.yaml
-```
-
-```yaml
-# config.yaml
-
-version: '1.0' # You must specify the config version. 
-legalize_causal_mask_value: True
-```
-
-#### Running circle models directly in Python
-
-After circle export, you can run the model directly in Python.
-
-Note that you should install one-compiler package first.
-
-The output types are numpy.ndarray.
-
-```python
-torch_module = AddModule()
-example_inputs = (torch.ones(4), torch.ones(4))
-
-circle_model = tico.convert(torch_module, example_inputs)
-circle_model(*example_inputs)
-# numpy.ndarray([2., 2., 2., 2.], dtype=float32)
-```
-
-### Quantization
+See the [Getting Started guide](./docs/getting_started.md) for compile configurations,
+`.pt2` conversion, and running Circle models directly in Python.
 
-The `tico.quantization` module provides a unified and modular interface for quantizing
- large language models (LLMs) and other neural networks.
- 
-It introduces a simple two-step workflow — **prepare** and **convert** — that
- abstracts the details of different quantization algorithms.
+## Quantization
 
-#### Basic Usage
+The [`tico.quantization`](./tico/quantization/README.md) module provides a unified,
+modular interface for quantizing neural networks — including large language models —
+through a simple two-step **prepare** → **convert** workflow:
 
 ```python
 from tico.quantization import prepare, convert
 from tico.quantization.config.gptq import GPTQConfig
-import torch
-import torch.nn as nn
-
-class LinearModel(nn.Module):
-    def __init__(self):
-        super().__init__()
-        self.linear = nn.Linear(8, 8)
-
-    def forward(self, x):
-        return self.linear(x)
-
-model = LinearModel().eval()
 
-# 1. Prepare for quantization
-quant_config = GPTQConfig()
-prepared_model = prepare(model, quant_config)
+prepared_model = prepare(model.eval(), GPTQConfig())
 
-# 2. Calibration
-for d in dataset:
+for d in dataset:          # calibration
     prepared_model(d)
 
-# 3. Apply GPTQ
-quantized_model = convert(prepared_model, quant_config)
+quantized_model = convert(prepared_model, GPTQConfig())
 ```
 
-For detailed documentation, design notes, and contributing guidelines, 
-see [tico/quantization/README.md](./tico/quantization/README.md).
+- [Quantization overview](./tico/quantization/README.md) — API, architecture, and how to add a new algorithm
+- [Quantization algorithms](./tico/quantization/algorithm/README.md) — GPTQ, SmoothQuant, SpinQuant, CLE, …
+- [Config-driven CLI examples](./tico/quantization/examples/README.md) — quantize, evaluate, and inspect LLM/VLM recipes from the command line
 
+## Documentation
 
-## For Developers
+### For users
 
-### Testing & Code Formatting
+| Document | Description |
+|---|---|
+| [Getting Started](./docs/getting_started.md) | Converting modules and `.pt2` files, compile configuration, running Circle models in Python |
+| [Quantization](./tico/quantization/README.md) | The `prepare`/`convert` quantization API and toolkit |
+| [Quantization examples](./tico/quantization/examples/README.md) | Command-line quantization, evaluation, and debugging workflows |
 
-Run below commands to configure testing or formatting environment.
-
-Refer to the dedicated section to have more fine-grained control.
-
-```bash
-$ ./ccex configure                          # to set up testing & formatting environment
-$ ./ccex configure format                   # to set up only formatting environment
-$ ./ccex configure test                     # to set up only testing environment
-```
-
-**Available options**
-- `--torch_ver <torch version>` To install a specific torch family package(ex. torchvision) version (default: 2.6)
-  - Available <torch version>: '2.5', '2.6', 'nightly'
-
-```bash
-$ ./ccex configure                          # to set up testing & formatting environment with stable2.6.x version
-$ ./ccex configure test                     # to set up only testing environment with stable 2.6.x version
-$ ./ccex configure test --torch_ver 2.5     # to set up only testing environment with stable 2.5.x version
-$ ./ccex configure test --torch_ver nightly     # to set up only testing environment with nightly version
-```
+### For developers
 
-### Testing
+| Document | Description |
+|---|---|
+| [Development guide](./docs/development.md) | Environment setup, testing, and code formatting with `./ccex` |
+| [System design](./docs/design.md) | Architecture, pass pipeline, invariants, and behavior design |
+| [Requirements](./docs/requirements.md) | Functional and non-functional requirements |
+| [System tests](./docs/system_test.md) | System-level test coverage |
 
-#### Test congifure
+## Contributing
 
-Run below commands to install requirements for testing.
+Contributions are welcome! For quantization algorithms, start with the
+[quantization contribution guide](./tico/quantization/README.md#contributing-adding-a-new-algorithm)
+and the [recipes developer guide](./tico/quantization/recipes/README.md).
+Before submitting a PR, set up the development environment and run the tests and
+formatter as described in the [development guide](./docs/development.md).
 
-**NOTE** `TICO` will be installed in an editable mode.
+## License
 
-```bash
-./ccex configure test
-
-# without editable install
-./ccex configure test --dist
-```
-
-#### Test All
-
-Run below commands to run the all unit tests.
-
-**NOTE** Unit tests don't include model test.
-
-```bash
-./ccex test
-# OR
-./ccex test run-all-tests
-```
-
-#### Test Subset
-
-To run subset of `test.modules.*`,
-Run `./ccex test -k <keyword>`
-
-
-For example, to run tests in specific sub-directory (op, net, ..)
-```bash
-# To run tests in specific sub-directory (op/, net/ ..)
-./ccex test -k op
-./ccex test -k net
-
-# To run tests in one file (single/op/add, single/op/sub, ...)
-./ccex test -k add
-./ccex test -k sub
-
-# To run SimpleAdd test in test/modules/single/op/add.py
-./ccex test -k SimpleAdd
-```
-
-To see the full debug log, add `-v` or `TICO_LOG=4`.
-
-```bash
-TICO_LOG=4 ./ccex test -k add
-# OR
-./ccex test -v -k add
-```
-
-To dump intermediate torch graph ir as `.png`, add `TICO_GRAPH_DUMP=1`
-
-```bash
-TICO_GRAPH_DUMP=1 ./ccex test -k add
-# Images are dumped into `$(pwd)/.tico_temp`
-```
-
-#### Test Model
-
-If you want to test them locally, you can do so by navigating to each model directory, 
- installing the dependencies listed in its `requirements.txt`, and running the tests one by one.
-```bash
-$ pip install -r test/modules/model/<model_name>/requirements.txt
-# Run test for a single model
-$ ./ccex test -m <model_name>
-# Run models whose names contain "Llama" (e.g., Llama, LlamaDecoderLayer, LlamaWithGQA, etc.)
-# Note that you should use quotes for the wildcard(*) pattern
-$ ./ccex test -m "Llama*"
-```
-
-For example, to run a single model
-```
-./ccex test -m InceptionV3
-```
-
-#### Runtime Options
-
-By default, `./ccex test` runs all modules with the `circle-interpreter` engine.
- You can override this and run tests using the `onert` runtime instead.
-
-
-##### 0. Install ONERT
-
-```bash
-pip install onert
-```
-
-##### 1. Command-Line Flag
-
-Use the `--runtime` (or `-r`) flag to select a runtime:
-
-```bash
-# Run with the default circle-interpreter
-./ccex test
-
-# Run all tests with onert
-./ccex test --runtime onert
-# or
-./ccex test -r onert
-```
-
-##### 2. Environment Variable
-
-You can also set the `CCEX_RUNTIME` environment variable:
-
-```bash
-# Temporarily override for one command
-CCEX_RUNTIME=onert ./ccex test
-
-# Persist in your shell session
-export CCEX_RUNTIME=onert
-./ccex test
-```
-
-##### Supported Runtimes
-
-- circle-interpreter (default): uses the Circle interpreter for inference.
-- onert: uses the ONERT package for inference, useful when the Circle interpreter
- cannot run a given module.
-
-### Code Formatting
-
-#### Format configure
-
-Run below commands to install requirements for formatting.
-
-```bash
-./ccex configure format
-```
-
-#### Format run
-
-```bash
-./ccex format
-```
+Licensed under the Apache License 2.0 — see [LICENSE](./LICENSE).

tico-0.2.0.dev260702/README.md

@@ -0,0 +1,131 @@
+# TICO
+
+_TICO_ (**T**orch **I**R to **C**ircle [ONE](https://github.com/Samsung/ONE)) is a Python
+library that converts PyTorch modules into Circle models — a lightweight and efficient
+representation in ONE designed for optimized on-device neural network inference.
+
+## Highlights
+
+- **One-call conversion** — `tico.convert()` turns an `nn.Module` into a ready-to-deploy `.circle` binary.
+- **`.pt2` support** — convert saved `torch.export` programs via the Python API or the `pt2-to-circle` command-line tool.
+- **Run Circle models in Python** — execute converted models directly for quick parity checks against PyTorch.
+- **Quantization toolkit** — a unified `prepare`/`convert` API with GPTQ, PTQ (WrapQ), SmoothQuant, SpinQuant, and CLE, plus config-driven CLI recipes for LLMs and VLMs.
+
+## Installation
+
+### Prerequisites
+
+- Python **3.10+**
+- (Optional) [one-compiler](https://github.com/Samsung/ONE/releases) — only required to run
+  inference with converted Circle models. Conversion itself does not need it.
+
+We highly recommend using a virtual environment (e.g., conda, venv).
+
+### From PyPI
+
+```bash
+pip install tico
+```
+
+### From source
+
+```bash
+git clone https://github.com/Samsung/TICO.git
+cd TICO
+
+./ccex build     # generates build/ and dist/
+./ccex install   # installs the package
+```
+
+**`./ccex install` options**
+
+| Option | Description |
+|---|---|
+| `--dist` | Install from the built wheel (default is editable mode) |
+| `--torch_ver <ver>` | Torch version to install: a family (`2.5` ~ `2.10`), an exact version (e.g. `2.7.0+cu118`), or `nightly`. Default: `2.7` |
+| `--cuda_ver <maj.min>` | Override the detected CUDA version (e.g. `12.1`) |
+| `--cpu_only` | Force a CPU-only Torch installation |
+
+## Quick start
+
+```python
+import tico
+import torch
+
+class AddModule(torch.nn.Module):
+    def forward(self, x, y):
+        return x + y
+
+torch_module = AddModule()
+example_inputs = (torch.ones(4), torch.ones(4))
+
+circle_model = tico.convert(torch_module.eval(), example_inputs)
+circle_model.save('add.circle')
+```
+
+> [!NOTE]
+> Call `eval()` on the module before conversion. _TICO_ internally uses
+> [`torch.export`](https://pytorch.org/docs/stable/export.html#torch-export), so the module
+> must be export-able.
+
+Converting a saved `.pt2` file from the command line:
+
+```bash
+pt2-to-circle -i add.pt2 -o add.circle
+```
+
+See the [Getting Started guide](./docs/getting_started.md) for compile configurations,
+`.pt2` conversion, and running Circle models directly in Python.
+
+## Quantization
+
+The [`tico.quantization`](./tico/quantization/README.md) module provides a unified,
+modular interface for quantizing neural networks — including large language models —
+through a simple two-step **prepare** → **convert** workflow:
+
+```python
+from tico.quantization import prepare, convert
+from tico.quantization.config.gptq import GPTQConfig
+
+prepared_model = prepare(model.eval(), GPTQConfig())
+
+for d in dataset:          # calibration
+    prepared_model(d)
+
+quantized_model = convert(prepared_model, GPTQConfig())
+```
+
+- [Quantization overview](./tico/quantization/README.md) — API, architecture, and how to add a new algorithm
+- [Quantization algorithms](./tico/quantization/algorithm/README.md) — GPTQ, SmoothQuant, SpinQuant, CLE, …
+- [Config-driven CLI examples](./tico/quantization/examples/README.md) — quantize, evaluate, and inspect LLM/VLM recipes from the command line
+
+## Documentation
+
+### For users
+
+| Document | Description |
+|---|---|
+| [Getting Started](./docs/getting_started.md) | Converting modules and `.pt2` files, compile configuration, running Circle models in Python |
+| [Quantization](./tico/quantization/README.md) | The `prepare`/`convert` quantization API and toolkit |
+| [Quantization examples](./tico/quantization/examples/README.md) | Command-line quantization, evaluation, and debugging workflows |
+
+### For developers
+
+| Document | Description |
+|---|---|
+| [Development guide](./docs/development.md) | Environment setup, testing, and code formatting with `./ccex` |
+| [System design](./docs/design.md) | Architecture, pass pipeline, invariants, and behavior design |
+| [Requirements](./docs/requirements.md) | Functional and non-functional requirements |
+| [System tests](./docs/system_test.md) | System-level test coverage |
+
+## Contributing
+
+Contributions are welcome! For quantization algorithms, start with the
+[quantization contribution guide](./tico/quantization/README.md#contributing-adding-a-new-algorithm)
+and the [recipes developer guide](./tico/quantization/recipes/README.md).
+Before submitting a PR, set up the development environment and run the tests and
+formatter as described in the [development guide](./docs/development.md).
+
+## License
+
+Licensed under the Apache License 2.0 — see [LICENSE](./LICENSE).

tico-0.2.0.dev260702/tico/_version.py

@@ -0,0 +1 @@
+__version__ = "0.2.0.dev260702"

{tico-0.2.0.dev260701 → tico-0.2.0.dev260702}/tico/quantization/config/gemma4_builders.py

@@ -85,9 +85,9 @@ def build_gemma4_e2b_ptq_config(
     weight = weight or _default_weight()
     linear_weight = linear_weight or weight
 
-    overrides: Dict[str, Any] = {"model": {"model": {}}}
+    overrides: Dict[str, Any] = {"model": {}}
 
-    model_overrides = overrides["model"]["model"]
+    model_overrides = overrides["model"]
     model_overrides["language_model"] = {
         "embed_tokens": _weight_override(embedding_weight),
         "layers": {},
@@ -126,7 +126,7 @@ def build_gemma4_e2b_ptq_config(
     model_overrides["embed_vision"] = {
         "embedding_projection": _weight_override(linear_weight),
     }
-    overrides["model"]["lm_head"] = _weight_override(lm_head_weight or linear_weight)
+    overrides["lm_head"] = _weight_override(lm_head_weight or linear_weight)
 
     return PTQConfig(
         activation=activation,

{tico-0.2.0.dev260701 → tico-0.2.0.dev260702}/tico/quantization/recipes/debug/wrapper_smoke/cases/qwen3_vl.py

@@ -292,6 +292,11 @@ def _create_image_input(
     return example
 
 
+def _causal_mask(seq_len: int, fill_value: float = -120.0) -> torch.Tensor:
+    mask = torch.full((1, 1, seq_len, seq_len), fill_value)
+    return torch.triu(mask, diagonal=1)
+
+
 class QwenBaseCase(WrapperSmokeCase):
     """Base class for Qwen3-VL wrapper smoke cases."""
 
@@ -341,7 +346,7 @@ class QwenTextAttentionCase(QwenBaseCase):
     ) -> Any:
         """Run the original text attention signature with an explicit mask."""
         hidden, rope = sample.args
-        mask = torch.zeros(1, 1, hidden.shape[1], hidden.shape[1])
+        mask = _causal_mask(hidden.shape[1])
         return reference(hidden, position_embeddings=rope, attention_mask=mask)[0]
 
     def export_input(