gaia-ml 0.6.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- gaia_ml-0.6.0/LICENSE +21 -0
- gaia_ml-0.6.0/MANIFEST.in +1 -0
- gaia_ml-0.6.0/PKG-INFO +324 -0
- gaia_ml-0.6.0/README.md +291 -0
- gaia_ml-0.6.0/pyproject.toml +71 -0
- gaia_ml-0.6.0/python/gaia_ml/__init__.py +17 -0
- gaia_ml-0.6.0/python/gaia_ml/_version.py +3 -0
- gaia_ml-0.6.0/python/gaia_ml/cli.py +98 -0
- gaia_ml-0.6.0/python/gaia_ml/limits.py +18 -0
- gaia_ml-0.6.0/python/gaia_ml/logging.py +36 -0
- gaia_ml-0.6.0/python/gaia_ml/model_ir.py +127 -0
- gaia_ml-0.6.0/python/gaia_ml/onnx_frontend.py +348 -0
- gaia_ml-0.6.0/python/gaia_ml/optimizations.py +49 -0
- gaia_ml-0.6.0/python/gaia_ml/project.py +1389 -0
- gaia_ml-0.6.0/python/gaia_ml/templates/aie/CMakeLists.txt.j2 +110 -0
- gaia_ml-0.6.0/python/gaia_ml/templates/aie/Graph/modelGraph.cpp.j2 +13 -0
- gaia_ml-0.6.0/python/gaia_ml/templates/aie/Graph/modelGraph.hpp.j2 +40 -0
- gaia_ml-0.6.0/python/gaia_ml/templates/aie/Kernel/include/activation.hpp +240 -0
- gaia_ml-0.6.0/python/gaia_ml/templates/aie/README.md.j2 +54 -0
- gaia_ml-0.6.0/python/gaia_ml.egg-info/PKG-INFO +324 -0
- gaia_ml-0.6.0/python/gaia_ml.egg-info/SOURCES.txt +24 -0
- gaia_ml-0.6.0/python/gaia_ml.egg-info/dependency_links.txt +1 -0
- gaia_ml-0.6.0/python/gaia_ml.egg-info/entry_points.txt +2 -0
- gaia_ml-0.6.0/python/gaia_ml.egg-info/requires.txt +3 -0
- gaia_ml-0.6.0/python/gaia_ml.egg-info/top_level.txt +1 -0
- gaia_ml-0.6.0/setup.cfg +4 -0
gaia_ml-0.6.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 I. Xiotidis and the NGT WP2.1 Group
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
include LICENSE
|
gaia_ml-0.6.0/PKG-INFO
ADDED
|
@@ -0,0 +1,324 @@
|
|
|
1
|
+
Metadata-Version: 2.2
|
|
2
|
+
Name: gaia-ml
|
|
3
|
+
Version: 0.6.0
|
|
4
|
+
Summary: GAIA-ML ONNX-to-AIE project generator
|
|
5
|
+
Author: I. Xiotidis
|
|
6
|
+
Maintainer: NGT WP2.1 Group
|
|
7
|
+
License: MIT
|
|
8
|
+
Project-URL: Homepage, https://gitlab.cern.ch/atlas-nextgen-wp21/aie/gaia-ml
|
|
9
|
+
Project-URL: Documentation, https://gitlab.cern.ch/atlas-nextgen-wp21/aie/gaia-ml/-/blob/main/README.md
|
|
10
|
+
Project-URL: Repository, https://gitlab.cern.ch/atlas-nextgen-wp21/aie/gaia-ml
|
|
11
|
+
Project-URL: Issues, https://gitlab.cern.ch/atlas-nextgen-wp21/aie/gaia-ml/-/issues
|
|
12
|
+
Keywords: aie,amd,cmake,machine-learning,onnx,vitis
|
|
13
|
+
Classifier: Development Status :: 3 - Alpha
|
|
14
|
+
Classifier: Environment :: Console
|
|
15
|
+
Classifier: Intended Audience :: Developers
|
|
16
|
+
Classifier: Intended Audience :: Science/Research
|
|
17
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
18
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
19
|
+
Classifier: Programming Language :: Python :: 3
|
|
20
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
22
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
23
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
24
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
25
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
26
|
+
Classifier: Topic :: Software Development :: Code Generators
|
|
27
|
+
Requires-Python: >=3.9
|
|
28
|
+
Description-Content-Type: text/markdown
|
|
29
|
+
License-File: LICENSE
|
|
30
|
+
Requires-Dist: jinja2>=3.1
|
|
31
|
+
Requires-Dist: numpy>=1.22
|
|
32
|
+
Requires-Dist: onnx>=1.14
|
|
33
|
+
|
|
34
|
+
# GAIA-ML
|
|
35
|
+
|
|
36
|
+
Global AI-Engine Architecture - ML (GAIA-ML) is a software tool hoping to automatically transform ML algorithms into AMD AI-Engine (AIE) optimal implementations. GAIA-ML is developed within the ATLAS Global Trigger group, as the ML layers implemented are designed to provide optimal computation for low latency and high-throughput environments
|
|
37
|
+
|
|
38
|
+
Main developer: I. Xiotidis.
|
|
39
|
+
|
|
40
|
+
GAIA-ML is developed by the NGT WP2.1 Group.
|
|
41
|
+
|
|
42
|
+
License: MIT.
|
|
43
|
+
|
|
44
|
+
## Framework infrastructure
|
|
45
|
+
|
|
46
|
+
GAIA-ML follows a layers approach where each layer of the ML architecture is coded in a single AIE kernel. Having each layer coded in a single layer doesn't mean that the final result will be deployed in different AIE tiles (parametric graph implementation).
|
|
47
|
+
|
|
48
|
+
<!-- GAIA_STATUS_START -->
|
|
49
|
+
## Project Status
|
|
50
|
+
|
|
51
|
+
This section is generated from the repository contents. Refresh it with:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
cmake -S . -B build
|
|
55
|
+
cmake --build build --target readme-status
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
or directly:
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
python3 tools/update_readme_status.py
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
### Layers
|
|
65
|
+
|
|
66
|
+
| Layer | Submodule | Branch | SHA | CMake | Stream I/O | Kernel headers | Local graphs |
|
|
67
|
+
| --- | --- | --- | --- | --- | --- | --- | --- |
|
|
68
|
+
| activation | yes | master | bc7efd9 | yes | no | activation.hpp | - |
|
|
69
|
+
| conv | yes | cnn_v2 | 794c567 | yes | yes | activation_policy.hpp, conv2d.hpp, conv2d_depthwise.hpp | convGraph.cpp, depthwiseGraph.cpp |
|
|
70
|
+
| dense | yes | gaia_int | c2ae42f | yes | yes | activation_policy.hpp, dense.hpp, dense_aux.hpp | denseAuxGraph.cpp, denseGraph.cpp |
|
|
71
|
+
| tensor_op | yes | master | 39e64de | yes | yes | flatten_chw.hpp | - |
|
|
72
|
+
|
|
73
|
+
### CMake Targets
|
|
74
|
+
|
|
75
|
+
```text
|
|
76
|
+
aie
|
|
77
|
+
aie-conv2d
|
|
78
|
+
aie-dense
|
|
79
|
+
aie-dense-aux
|
|
80
|
+
aie-depthwise
|
|
81
|
+
aiesim
|
|
82
|
+
aiesim-conv2d
|
|
83
|
+
aiesim-dense
|
|
84
|
+
aiesim-dense-aux
|
|
85
|
+
aiesim-depthwise
|
|
86
|
+
data
|
|
87
|
+
data-conv2d
|
|
88
|
+
data-dense
|
|
89
|
+
data-depthwise
|
|
90
|
+
gaia_activation_kernel
|
|
91
|
+
gaia_conv_kernel
|
|
92
|
+
gaia_dense_kernel
|
|
93
|
+
gaia_kernels
|
|
94
|
+
gaia_tensor_op_kernel
|
|
95
|
+
layer_aie
|
|
96
|
+
layer_aiesim
|
|
97
|
+
layer_data
|
|
98
|
+
layer_x86
|
|
99
|
+
layer_x86sim
|
|
100
|
+
readme-status
|
|
101
|
+
x86
|
|
102
|
+
x86-conv2d
|
|
103
|
+
x86-dense
|
|
104
|
+
x86-dense-aux
|
|
105
|
+
x86-depthwise
|
|
106
|
+
x86sim
|
|
107
|
+
x86sim-conv2d
|
|
108
|
+
x86sim-dense
|
|
109
|
+
x86sim-dense-aux
|
|
110
|
+
x86sim-depthwise
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
### Repository Layout
|
|
114
|
+
|
|
115
|
+
```text
|
|
116
|
+
.
|
|
117
|
+
|-- examples
|
|
118
|
+
| |-- conv_flatten_dense_onnx_generate.py
|
|
119
|
+
| `-- dense_onnx_generate.py
|
|
120
|
+
|-- layers
|
|
121
|
+
| |-- activation
|
|
122
|
+
| | |-- Kernel
|
|
123
|
+
| | | `-- include
|
|
124
|
+
| | |-- .gitignore
|
|
125
|
+
| | |-- CMakeLists.txt
|
|
126
|
+
| | `-- README.md
|
|
127
|
+
| |-- conv
|
|
128
|
+
| | |-- Graph
|
|
129
|
+
| | | |-- convGraph.cpp
|
|
130
|
+
| | | |-- convGraph.hpp
|
|
131
|
+
| | | |-- depthwiseGraph.cpp
|
|
132
|
+
| | | `-- depthwiseGraph.hpp
|
|
133
|
+
| | |-- Kernel
|
|
134
|
+
| | | |-- include
|
|
135
|
+
| | | `-- src
|
|
136
|
+
| | |-- testVectors
|
|
137
|
+
| | | `-- makeDummyData.py
|
|
138
|
+
| | |-- .gitignore
|
|
139
|
+
| | |-- CMakeLists.txt
|
|
140
|
+
| | `-- README.md
|
|
141
|
+
| |-- dense
|
|
142
|
+
| | |-- Graph
|
|
143
|
+
| | | |-- denseAuxGraph.cpp
|
|
144
|
+
| | | |-- denseAuxGraph.hpp
|
|
145
|
+
| | | |-- denseGraph.cpp
|
|
146
|
+
| | | `-- denseGraph.hpp
|
|
147
|
+
| | |-- Kernel
|
|
148
|
+
| | | |-- include
|
|
149
|
+
| | | `-- src
|
|
150
|
+
| | |-- testVectors
|
|
151
|
+
| | | `-- makeDummyData.py
|
|
152
|
+
| | |-- .gitignore
|
|
153
|
+
| | |-- CMakeLists.txt
|
|
154
|
+
| | |-- notes.md
|
|
155
|
+
| | `-- README.md
|
|
156
|
+
| `-- tensor_op
|
|
157
|
+
| |-- Kernel
|
|
158
|
+
| | |-- include
|
|
159
|
+
| | `-- src
|
|
160
|
+
| |-- CMakeLists.txt
|
|
161
|
+
| `-- README.md
|
|
162
|
+
|-- python
|
|
163
|
+
| |-- gaia_ml
|
|
164
|
+
| | |-- __init__.py
|
|
165
|
+
| | |-- _version.py
|
|
166
|
+
| | |-- cli.py
|
|
167
|
+
| | |-- limits.py
|
|
168
|
+
| | |-- model_ir.py
|
|
169
|
+
| | |-- onnx_frontend.py
|
|
170
|
+
| | `-- project.py
|
|
171
|
+
| `-- gaia_ml.egg-info
|
|
172
|
+
| |-- dependency_links.txt
|
|
173
|
+
| |-- entry_points.txt
|
|
174
|
+
| |-- PKG-INFO
|
|
175
|
+
| |-- requires.txt
|
|
176
|
+
| |-- SOURCES.txt
|
|
177
|
+
| `-- top_level.txt
|
|
178
|
+
|-- tools
|
|
179
|
+
| `-- update_readme_status.py
|
|
180
|
+
|-- .gitignore
|
|
181
|
+
|-- .gitmodules
|
|
182
|
+
|-- CMakeLists.txt
|
|
183
|
+
|-- pyproject.toml
|
|
184
|
+
|-- README.md
|
|
185
|
+
`-- setup.cfg
|
|
186
|
+
```
|
|
187
|
+
<!-- GAIA_STATUS_END -->
|
|
188
|
+
|
|
189
|
+
## CMake hierarchy
|
|
190
|
+
|
|
191
|
+
Each layer repository can be configured and built on its own. In that mode, the
|
|
192
|
+
layer-local graph is used for `x86`, `aie`, `x86sim`, and `aiesim` targets.
|
|
193
|
+
|
|
194
|
+
From the top-level package, layer repositories are added as kernel providers.
|
|
195
|
+
The exported targets are:
|
|
196
|
+
|
|
197
|
+
```text
|
|
198
|
+
gaia_activation_kernel
|
|
199
|
+
gaia_conv_kernel
|
|
200
|
+
gaia_dense_kernel
|
|
201
|
+
gaia_kernels
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
Enable `GAIA_BUILD_LAYER_GRAPHS=ON` only when you also want the top-level build
|
|
205
|
+
to expose the layer-local smoke-test graph targets.
|
|
206
|
+
|
|
207
|
+
## Notes
|
|
208
|
+
|
|
209
|
+
GAIA-ML v1.0 supports stream I/O only. Layer kernels use
|
|
210
|
+
`input_stream<int32>` and `output_stream<int32>` ports so the
|
|
211
|
+
top-level graph can be built around a single low-latency dataflow contract.
|
|
212
|
+
|
|
213
|
+
The current convolution stream kernels keep the existing frame-based compute
|
|
214
|
+
path internally by reading one streamed frame into local AIE memory before
|
|
215
|
+
computing. This preserves the stream interface for v1.0 while leaving room for
|
|
216
|
+
a later line-buffer implementation that can reduce first-output latency.
|
|
217
|
+
|
|
218
|
+
Activations are header-only compile-time policies in v1.0. Layers default to
|
|
219
|
+
`gaia::activation::identity` when the shared activation header is not on the
|
|
220
|
+
include path, and the top-level package exposes `activation.hpp` so graphs can
|
|
221
|
+
fuse policies such as `linear`, `relu`, `sigmoid`, `tanh`, or `softmax` into the
|
|
222
|
+
producer kernel instead of adding a separate activation tile.
|
|
223
|
+
|
|
224
|
+
## Python Automation
|
|
225
|
+
|
|
226
|
+
The first automation scaffold is available as a pip-installable package:
|
|
227
|
+
|
|
228
|
+
```bash
|
|
229
|
+
pip install -e .
|
|
230
|
+
gaia_ml version
|
|
231
|
+
gaia_ml inspect model.onnx
|
|
232
|
+
gaia_ml generate model.onnx --output build/generated_model --strategy throughput
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
As of `gaia_ml` v0.5.0, generated project structure is emitted from Jinja2
|
|
236
|
+
templates packaged under `gaia_ml/templates/aie`. Backend Python code now builds
|
|
237
|
+
the graph/layer context and renders the project files from templates, so changes
|
|
238
|
+
to generated `Graph/`, `CMakeLists.txt`, and generated README layout no longer
|
|
239
|
+
need to be hard-coded directly into `project.py`.
|
|
240
|
+
|
|
241
|
+
As of `gaia_ml` v0.6.0, generated kernels keep the same stream interface but
|
|
242
|
+
avoid full input-frame caching where possible. Depthwise Conv uses a K-row line
|
|
243
|
+
buffer per channel; generic Conv and 1x1 pointwise Conv stream input rows and
|
|
244
|
+
accumulate output partials; Dense and Dense-CHW stream inputs directly into
|
|
245
|
+
output accumulators. This preserves the existing CHW padded stream contract
|
|
246
|
+
while reducing local input storage and preparing the backend for more aggressive
|
|
247
|
+
overlap.
|
|
248
|
+
|
|
249
|
+
Project generation supports two lowering strategies:
|
|
250
|
+
|
|
251
|
+
```text
|
|
252
|
+
throughput Use more AIE tiles and preserve layer-level graph pipelining.
|
|
253
|
+
compact Fuse supported patterns to minimize tile count.
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
`throughput` is the default because AIE latency often benefits from multiple
|
|
257
|
+
tiles running concurrently. `compact` is useful when fitting into the smallest
|
|
258
|
+
number of tiles matters more than first-output latency.
|
|
259
|
+
|
|
260
|
+
The current generator parses ONNX Conv, depthwise Conv, 1x1 pointwise Conv,
|
|
261
|
+
Flatten, and Dense/Gemm-style blocks, runs ONNX shape inference, checks what
|
|
262
|
+
can be emitted, and creates a self-contained AIE project with `Graph/`,
|
|
263
|
+
`Kernel/`, `weights/`, `testVectors/`, and `CMakeLists.txt`. Generation emits
|
|
264
|
+
one packed-output tile per supported compute block and chains them in model
|
|
265
|
+
order, unless a low-latency fused pattern is available:
|
|
266
|
+
|
|
267
|
+
```text
|
|
268
|
+
input stream -> layer0 -> layer1 -> ... -> output stream
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
`gaia_ml inspect` always reports optimization suggestions, including patterns
|
|
272
|
+
that are not generated yet. The CLI uses colored log levels for info, warning,
|
|
273
|
+
critical warning, and error messages so compatibility problems and latency
|
|
274
|
+
recommendations are easier to scan.
|
|
275
|
+
|
|
276
|
+
The current `int32` backend pads dense vectors and convolution rows to 8-lane
|
|
277
|
+
chunks. Dense blocks are rejected if either side exceeds 128 padded lanes.
|
|
278
|
+
Convolution blocks are currently single-tile only: input spatial dimensions are
|
|
279
|
+
limited to 32x32, kernels to 8x8, padded input/output rows to 128 lanes, and
|
|
280
|
+
padded frame buffers to the current local-memory guardrails. Larger layers need
|
|
281
|
+
an explicit tiling strategy before generation.
|
|
282
|
+
|
|
283
|
+
`Flatten` is represented in the Python IR as a tensor operation. When possible,
|
|
284
|
+
it is fused away: `Conv -> Flatten -> Dense` lowers to a `packed_dense_chw`
|
|
285
|
+
consumer that reads the padded CHW stream directly and skips row padding without
|
|
286
|
+
an extra AIE tile. The `tensor_op` layer also provides an explicit
|
|
287
|
+
`flatten_chw` kernel for future graph cases where a standalone reshuffle tile is
|
|
288
|
+
actually required. ONNX depthwise Conv lowers to `packed_depthwise_conv2d`, and
|
|
289
|
+
ONNX 1x1 Conv lowers to a specialized `packed_pointwise_conv2d` kernel. Weights
|
|
290
|
+
are emitted as headers under `weights/`, and the generated
|
|
291
|
+
`testVectors/reference_output.csv` contains the expected end-to-end inference
|
|
292
|
+
stream for the generated test input.
|
|
293
|
+
The first low-latency fused lowering is
|
|
294
|
+
`DepthwiseConv -> 1x1 Conv -> Flatten -> Dense(1)`, which emits
|
|
295
|
+
`packed_dw_pw_dense_scalar`. The generic kernels are still emitted in generated
|
|
296
|
+
projects so each kernel can continue to be developed and tested independently.
|
|
297
|
+
Generated graphs write simulator output to `testVectors/model_output.csv`; after
|
|
298
|
+
hardware AIE simulation this appears under
|
|
299
|
+
`aiesimulator_output/testVectors/model_output.csv`, while the expected output
|
|
300
|
+
remains in `testVectors/reference_output.csv`.
|
|
301
|
+
The generated project can be built with:
|
|
302
|
+
|
|
303
|
+
```bash
|
|
304
|
+
cmake -S build/generated_model -B build/generated_model/build
|
|
305
|
+
cmake --build build/generated_model/build --target x86
|
|
306
|
+
cmake --build build/generated_model/build --target x86sim
|
|
307
|
+
```
|
|
308
|
+
|
|
309
|
+
For now, project emission supports sequential Conv, Flatten, and Dense/Gemm
|
|
310
|
+
models with optional supported activations and zero/no bias. Branching, general
|
|
311
|
+
transpose/concat/split reshuffles, softmax fusion, and larger-than-single-tile
|
|
312
|
+
blocks still need explicit graph lowering rules before generation.
|
|
313
|
+
|
|
314
|
+
The `examples/conv_flatten_dense_onnx_generate.py` script builds a compact
|
|
315
|
+
performance-check model:
|
|
316
|
+
|
|
317
|
+
```text
|
|
318
|
+
Depthwise 3x3 over 6 channels -> ReLU -> 1x1 pointwise Conv ->
|
|
319
|
+
Flatten -> Dense -> ReLU -> Dense logits
|
|
320
|
+
```
|
|
321
|
+
|
|
322
|
+
It can also append ONNX Softmax with `--include-softmax` for semantic
|
|
323
|
+
inspection. GAIA-ML recognizes that Softmax but rejects project generation until
|
|
324
|
+
the backend has a whole-vector normalization kernel.
|
gaia_ml-0.6.0/README.md
ADDED
|
@@ -0,0 +1,291 @@
|
|
|
1
|
+
# GAIA-ML
|
|
2
|
+
|
|
3
|
+
Global AI-Engine Architecture - ML (GAIA-ML) is a software tool hoping to automatically transform ML algorithms into AMD AI-Engine (AIE) optimal implementations. GAIA-ML is developed within the ATLAS Global Trigger group, as the ML layers implemented are designed to provide optimal computation for low latency and high-throughput environments
|
|
4
|
+
|
|
5
|
+
Main developer: I. Xiotidis.
|
|
6
|
+
|
|
7
|
+
GAIA-ML is developed by the NGT WP2.1 Group.
|
|
8
|
+
|
|
9
|
+
License: MIT.
|
|
10
|
+
|
|
11
|
+
## Framework infrastructure
|
|
12
|
+
|
|
13
|
+
GAIA-ML follows a layers approach where each layer of the ML architecture is coded in a single AIE kernel. Having each layer coded in a single layer doesn't mean that the final result will be deployed in different AIE tiles (parametric graph implementation).
|
|
14
|
+
|
|
15
|
+
<!-- GAIA_STATUS_START -->
|
|
16
|
+
## Project Status
|
|
17
|
+
|
|
18
|
+
This section is generated from the repository contents. Refresh it with:
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
cmake -S . -B build
|
|
22
|
+
cmake --build build --target readme-status
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
or directly:
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
python3 tools/update_readme_status.py
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
### Layers
|
|
32
|
+
|
|
33
|
+
| Layer | Submodule | Branch | SHA | CMake | Stream I/O | Kernel headers | Local graphs |
|
|
34
|
+
| --- | --- | --- | --- | --- | --- | --- | --- |
|
|
35
|
+
| activation | yes | master | bc7efd9 | yes | no | activation.hpp | - |
|
|
36
|
+
| conv | yes | cnn_v2 | 794c567 | yes | yes | activation_policy.hpp, conv2d.hpp, conv2d_depthwise.hpp | convGraph.cpp, depthwiseGraph.cpp |
|
|
37
|
+
| dense | yes | gaia_int | c2ae42f | yes | yes | activation_policy.hpp, dense.hpp, dense_aux.hpp | denseAuxGraph.cpp, denseGraph.cpp |
|
|
38
|
+
| tensor_op | yes | master | 39e64de | yes | yes | flatten_chw.hpp | - |
|
|
39
|
+
|
|
40
|
+
### CMake Targets
|
|
41
|
+
|
|
42
|
+
```text
|
|
43
|
+
aie
|
|
44
|
+
aie-conv2d
|
|
45
|
+
aie-dense
|
|
46
|
+
aie-dense-aux
|
|
47
|
+
aie-depthwise
|
|
48
|
+
aiesim
|
|
49
|
+
aiesim-conv2d
|
|
50
|
+
aiesim-dense
|
|
51
|
+
aiesim-dense-aux
|
|
52
|
+
aiesim-depthwise
|
|
53
|
+
data
|
|
54
|
+
data-conv2d
|
|
55
|
+
data-dense
|
|
56
|
+
data-depthwise
|
|
57
|
+
gaia_activation_kernel
|
|
58
|
+
gaia_conv_kernel
|
|
59
|
+
gaia_dense_kernel
|
|
60
|
+
gaia_kernels
|
|
61
|
+
gaia_tensor_op_kernel
|
|
62
|
+
layer_aie
|
|
63
|
+
layer_aiesim
|
|
64
|
+
layer_data
|
|
65
|
+
layer_x86
|
|
66
|
+
layer_x86sim
|
|
67
|
+
readme-status
|
|
68
|
+
x86
|
|
69
|
+
x86-conv2d
|
|
70
|
+
x86-dense
|
|
71
|
+
x86-dense-aux
|
|
72
|
+
x86-depthwise
|
|
73
|
+
x86sim
|
|
74
|
+
x86sim-conv2d
|
|
75
|
+
x86sim-dense
|
|
76
|
+
x86sim-dense-aux
|
|
77
|
+
x86sim-depthwise
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
### Repository Layout
|
|
81
|
+
|
|
82
|
+
```text
|
|
83
|
+
.
|
|
84
|
+
|-- examples
|
|
85
|
+
| |-- conv_flatten_dense_onnx_generate.py
|
|
86
|
+
| `-- dense_onnx_generate.py
|
|
87
|
+
|-- layers
|
|
88
|
+
| |-- activation
|
|
89
|
+
| | |-- Kernel
|
|
90
|
+
| | | `-- include
|
|
91
|
+
| | |-- .gitignore
|
|
92
|
+
| | |-- CMakeLists.txt
|
|
93
|
+
| | `-- README.md
|
|
94
|
+
| |-- conv
|
|
95
|
+
| | |-- Graph
|
|
96
|
+
| | | |-- convGraph.cpp
|
|
97
|
+
| | | |-- convGraph.hpp
|
|
98
|
+
| | | |-- depthwiseGraph.cpp
|
|
99
|
+
| | | `-- depthwiseGraph.hpp
|
|
100
|
+
| | |-- Kernel
|
|
101
|
+
| | | |-- include
|
|
102
|
+
| | | `-- src
|
|
103
|
+
| | |-- testVectors
|
|
104
|
+
| | | `-- makeDummyData.py
|
|
105
|
+
| | |-- .gitignore
|
|
106
|
+
| | |-- CMakeLists.txt
|
|
107
|
+
| | `-- README.md
|
|
108
|
+
| |-- dense
|
|
109
|
+
| | |-- Graph
|
|
110
|
+
| | | |-- denseAuxGraph.cpp
|
|
111
|
+
| | | |-- denseAuxGraph.hpp
|
|
112
|
+
| | | |-- denseGraph.cpp
|
|
113
|
+
| | | `-- denseGraph.hpp
|
|
114
|
+
| | |-- Kernel
|
|
115
|
+
| | | |-- include
|
|
116
|
+
| | | `-- src
|
|
117
|
+
| | |-- testVectors
|
|
118
|
+
| | | `-- makeDummyData.py
|
|
119
|
+
| | |-- .gitignore
|
|
120
|
+
| | |-- CMakeLists.txt
|
|
121
|
+
| | |-- notes.md
|
|
122
|
+
| | `-- README.md
|
|
123
|
+
| `-- tensor_op
|
|
124
|
+
| |-- Kernel
|
|
125
|
+
| | |-- include
|
|
126
|
+
| | `-- src
|
|
127
|
+
| |-- CMakeLists.txt
|
|
128
|
+
| `-- README.md
|
|
129
|
+
|-- python
|
|
130
|
+
| |-- gaia_ml
|
|
131
|
+
| | |-- __init__.py
|
|
132
|
+
| | |-- _version.py
|
|
133
|
+
| | |-- cli.py
|
|
134
|
+
| | |-- limits.py
|
|
135
|
+
| | |-- model_ir.py
|
|
136
|
+
| | |-- onnx_frontend.py
|
|
137
|
+
| | `-- project.py
|
|
138
|
+
| `-- gaia_ml.egg-info
|
|
139
|
+
| |-- dependency_links.txt
|
|
140
|
+
| |-- entry_points.txt
|
|
141
|
+
| |-- PKG-INFO
|
|
142
|
+
| |-- requires.txt
|
|
143
|
+
| |-- SOURCES.txt
|
|
144
|
+
| `-- top_level.txt
|
|
145
|
+
|-- tools
|
|
146
|
+
| `-- update_readme_status.py
|
|
147
|
+
|-- .gitignore
|
|
148
|
+
|-- .gitmodules
|
|
149
|
+
|-- CMakeLists.txt
|
|
150
|
+
|-- pyproject.toml
|
|
151
|
+
|-- README.md
|
|
152
|
+
`-- setup.cfg
|
|
153
|
+
```
|
|
154
|
+
<!-- GAIA_STATUS_END -->
|
|
155
|
+
|
|
156
|
+
## CMake hierarchy
|
|
157
|
+
|
|
158
|
+
Each layer repository can be configured and built on its own. In that mode, the
|
|
159
|
+
layer-local graph is used for `x86`, `aie`, `x86sim`, and `aiesim` targets.
|
|
160
|
+
|
|
161
|
+
From the top-level package, layer repositories are added as kernel providers.
|
|
162
|
+
The exported targets are:
|
|
163
|
+
|
|
164
|
+
```text
|
|
165
|
+
gaia_activation_kernel
|
|
166
|
+
gaia_conv_kernel
|
|
167
|
+
gaia_dense_kernel
|
|
168
|
+
gaia_kernels
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
Enable `GAIA_BUILD_LAYER_GRAPHS=ON` only when you also want the top-level build
|
|
172
|
+
to expose the layer-local smoke-test graph targets.
|
|
173
|
+
|
|
174
|
+
## Notes
|
|
175
|
+
|
|
176
|
+
GAIA-ML v1.0 supports stream I/O only. Layer kernels use
|
|
177
|
+
`input_stream<int32>` and `output_stream<int32>` ports so the
|
|
178
|
+
top-level graph can be built around a single low-latency dataflow contract.
|
|
179
|
+
|
|
180
|
+
The current convolution stream kernels keep the existing frame-based compute
|
|
181
|
+
path internally by reading one streamed frame into local AIE memory before
|
|
182
|
+
computing. This preserves the stream interface for v1.0 while leaving room for
|
|
183
|
+
a later line-buffer implementation that can reduce first-output latency.
|
|
184
|
+
|
|
185
|
+
Activations are header-only compile-time policies in v1.0. Layers default to
|
|
186
|
+
`gaia::activation::identity` when the shared activation header is not on the
|
|
187
|
+
include path, and the top-level package exposes `activation.hpp` so graphs can
|
|
188
|
+
fuse policies such as `linear`, `relu`, `sigmoid`, `tanh`, or `softmax` into the
|
|
189
|
+
producer kernel instead of adding a separate activation tile.
|
|
190
|
+
|
|
191
|
+
## Python Automation
|
|
192
|
+
|
|
193
|
+
The first automation scaffold is available as a pip-installable package:
|
|
194
|
+
|
|
195
|
+
```bash
|
|
196
|
+
pip install -e .
|
|
197
|
+
gaia_ml version
|
|
198
|
+
gaia_ml inspect model.onnx
|
|
199
|
+
gaia_ml generate model.onnx --output build/generated_model --strategy throughput
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
As of `gaia_ml` v0.5.0, generated project structure is emitted from Jinja2
|
|
203
|
+
templates packaged under `gaia_ml/templates/aie`. Backend Python code now builds
|
|
204
|
+
the graph/layer context and renders the project files from templates, so changes
|
|
205
|
+
to generated `Graph/`, `CMakeLists.txt`, and generated README layout no longer
|
|
206
|
+
need to be hard-coded directly into `project.py`.
|
|
207
|
+
|
|
208
|
+
As of `gaia_ml` v0.6.0, generated kernels keep the same stream interface but
|
|
209
|
+
avoid full input-frame caching where possible. Depthwise Conv uses a K-row line
|
|
210
|
+
buffer per channel; generic Conv and 1x1 pointwise Conv stream input rows and
|
|
211
|
+
accumulate output partials; Dense and Dense-CHW stream inputs directly into
|
|
212
|
+
output accumulators. This preserves the existing CHW padded stream contract
|
|
213
|
+
while reducing local input storage and preparing the backend for more aggressive
|
|
214
|
+
overlap.
|
|
215
|
+
|
|
216
|
+
Project generation supports two lowering strategies:
|
|
217
|
+
|
|
218
|
+
```text
|
|
219
|
+
throughput Use more AIE tiles and preserve layer-level graph pipelining.
|
|
220
|
+
compact Fuse supported patterns to minimize tile count.
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
`throughput` is the default because AIE latency often benefits from multiple
|
|
224
|
+
tiles running concurrently. `compact` is useful when fitting into the smallest
|
|
225
|
+
number of tiles matters more than first-output latency.
|
|
226
|
+
|
|
227
|
+
The current generator parses ONNX Conv, depthwise Conv, 1x1 pointwise Conv,
|
|
228
|
+
Flatten, and Dense/Gemm-style blocks, runs ONNX shape inference, checks what
|
|
229
|
+
can be emitted, and creates a self-contained AIE project with `Graph/`,
|
|
230
|
+
`Kernel/`, `weights/`, `testVectors/`, and `CMakeLists.txt`. Generation emits
|
|
231
|
+
one packed-output tile per supported compute block and chains them in model
|
|
232
|
+
order, unless a low-latency fused pattern is available:
|
|
233
|
+
|
|
234
|
+
```text
|
|
235
|
+
input stream -> layer0 -> layer1 -> ... -> output stream
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
`gaia_ml inspect` always reports optimization suggestions, including patterns
|
|
239
|
+
that are not generated yet. The CLI uses colored log levels for info, warning,
|
|
240
|
+
critical warning, and error messages so compatibility problems and latency
|
|
241
|
+
recommendations are easier to scan.
|
|
242
|
+
|
|
243
|
+
The current `int32` backend pads dense vectors and convolution rows to 8-lane
|
|
244
|
+
chunks. Dense blocks are rejected if either side exceeds 128 padded lanes.
|
|
245
|
+
Convolution blocks are currently single-tile only: input spatial dimensions are
|
|
246
|
+
limited to 32x32, kernels to 8x8, padded input/output rows to 128 lanes, and
|
|
247
|
+
padded frame buffers to the current local-memory guardrails. Larger layers need
|
|
248
|
+
an explicit tiling strategy before generation.
|
|
249
|
+
|
|
250
|
+
`Flatten` is represented in the Python IR as a tensor operation. When possible,
|
|
251
|
+
it is fused away: `Conv -> Flatten -> Dense` lowers to a `packed_dense_chw`
|
|
252
|
+
consumer that reads the padded CHW stream directly and skips row padding without
|
|
253
|
+
an extra AIE tile. The `tensor_op` layer also provides an explicit
|
|
254
|
+
`flatten_chw` kernel for future graph cases where a standalone reshuffle tile is
|
|
255
|
+
actually required. ONNX depthwise Conv lowers to `packed_depthwise_conv2d`, and
|
|
256
|
+
ONNX 1x1 Conv lowers to a specialized `packed_pointwise_conv2d` kernel. Weights
|
|
257
|
+
are emitted as headers under `weights/`, and the generated
|
|
258
|
+
`testVectors/reference_output.csv` contains the expected end-to-end inference
|
|
259
|
+
stream for the generated test input.
|
|
260
|
+
The first low-latency fused lowering is
|
|
261
|
+
`DepthwiseConv -> 1x1 Conv -> Flatten -> Dense(1)`, which emits
|
|
262
|
+
`packed_dw_pw_dense_scalar`. The generic kernels are still emitted in generated
|
|
263
|
+
projects so each kernel can continue to be developed and tested independently.
|
|
264
|
+
Generated graphs write simulator output to `testVectors/model_output.csv`; after
|
|
265
|
+
hardware AIE simulation this appears under
|
|
266
|
+
`aiesimulator_output/testVectors/model_output.csv`, while the expected output
|
|
267
|
+
remains in `testVectors/reference_output.csv`.
|
|
268
|
+
The generated project can be built with:
|
|
269
|
+
|
|
270
|
+
```bash
|
|
271
|
+
cmake -S build/generated_model -B build/generated_model/build
|
|
272
|
+
cmake --build build/generated_model/build --target x86
|
|
273
|
+
cmake --build build/generated_model/build --target x86sim
|
|
274
|
+
```
|
|
275
|
+
|
|
276
|
+
For now, project emission supports sequential Conv, Flatten, and Dense/Gemm
|
|
277
|
+
models with optional supported activations and zero/no bias. Branching, general
|
|
278
|
+
transpose/concat/split reshuffles, softmax fusion, and larger-than-single-tile
|
|
279
|
+
blocks still need explicit graph lowering rules before generation.
|
|
280
|
+
|
|
281
|
+
The `examples/conv_flatten_dense_onnx_generate.py` script builds a compact
|
|
282
|
+
performance-check model:
|
|
283
|
+
|
|
284
|
+
```text
|
|
285
|
+
Depthwise 3x3 over 6 channels -> ReLU -> 1x1 pointwise Conv ->
|
|
286
|
+
Flatten -> Dense -> ReLU -> Dense logits
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
It can also append ONNX Softmax with `--include-softmax` for semantic
|
|
290
|
+
inspection. GAIA-ML recognizes that Softmax but rejects project generation until
|
|
291
|
+
the backend has a whole-vector normalization kernel.
|