gaia-ml 0.6.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
gaia_ml-0.6.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 I. Xiotidis and the NGT WP2.1 Group
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1 @@
1
+ include LICENSE
gaia_ml-0.6.0/PKG-INFO ADDED
@@ -0,0 +1,324 @@
1
+ Metadata-Version: 2.2
2
+ Name: gaia-ml
3
+ Version: 0.6.0
4
+ Summary: GAIA-ML ONNX-to-AIE project generator
5
+ Author: I. Xiotidis
6
+ Maintainer: NGT WP2.1 Group
7
+ License: MIT
8
+ Project-URL: Homepage, https://gitlab.cern.ch/atlas-nextgen-wp21/aie/gaia-ml
9
+ Project-URL: Documentation, https://gitlab.cern.ch/atlas-nextgen-wp21/aie/gaia-ml/-/blob/main/README.md
10
+ Project-URL: Repository, https://gitlab.cern.ch/atlas-nextgen-wp21/aie/gaia-ml
11
+ Project-URL: Issues, https://gitlab.cern.ch/atlas-nextgen-wp21/aie/gaia-ml/-/issues
12
+ Keywords: aie,amd,cmake,machine-learning,onnx,vitis
13
+ Classifier: Development Status :: 3 - Alpha
14
+ Classifier: Environment :: Console
15
+ Classifier: Intended Audience :: Developers
16
+ Classifier: Intended Audience :: Science/Research
17
+ Classifier: License :: OSI Approved :: MIT License
18
+ Classifier: Operating System :: POSIX :: Linux
19
+ Classifier: Programming Language :: Python :: 3
20
+ Classifier: Programming Language :: Python :: 3 :: Only
21
+ Classifier: Programming Language :: Python :: 3.9
22
+ Classifier: Programming Language :: Python :: 3.10
23
+ Classifier: Programming Language :: Python :: 3.11
24
+ Classifier: Programming Language :: Python :: 3.12
25
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
26
+ Classifier: Topic :: Software Development :: Code Generators
27
+ Requires-Python: >=3.9
28
+ Description-Content-Type: text/markdown
29
+ License-File: LICENSE
30
+ Requires-Dist: jinja2>=3.1
31
+ Requires-Dist: numpy>=1.22
32
+ Requires-Dist: onnx>=1.14
33
+
34
+ # GAIA-ML
35
+
36
+ Global AI-Engine Architecture - ML (GAIA-ML) is a software tool hoping to automatically transform ML algorithms into AMD AI-Engine (AIE) optimal implementations. GAIA-ML is developed within the ATLAS Global Trigger group, as the ML layers implemented are designed to provide optimal computation for low latency and high-throughput environments
37
+
38
+ Main developer: I. Xiotidis.
39
+
40
+ GAIA-ML is developed by the NGT WP2.1 Group.
41
+
42
+ License: MIT.
43
+
44
+ ## Framework infrastructure
45
+
46
+ GAIA-ML follows a layers approach where each layer of the ML architecture is coded in a single AIE kernel. Having each layer coded in a single layer doesn't mean that the final result will be deployed in different AIE tiles (parametric graph implementation).
47
+
48
+ <!-- GAIA_STATUS_START -->
49
+ ## Project Status
50
+
51
+ This section is generated from the repository contents. Refresh it with:
52
+
53
+ ```bash
54
+ cmake -S . -B build
55
+ cmake --build build --target readme-status
56
+ ```
57
+
58
+ or directly:
59
+
60
+ ```bash
61
+ python3 tools/update_readme_status.py
62
+ ```
63
+
64
+ ### Layers
65
+
66
+ | Layer | Submodule | Branch | SHA | CMake | Stream I/O | Kernel headers | Local graphs |
67
+ | --- | --- | --- | --- | --- | --- | --- | --- |
68
+ | activation | yes | master | bc7efd9 | yes | no | activation.hpp | - |
69
+ | conv | yes | cnn_v2 | 794c567 | yes | yes | activation_policy.hpp, conv2d.hpp, conv2d_depthwise.hpp | convGraph.cpp, depthwiseGraph.cpp |
70
+ | dense | yes | gaia_int | c2ae42f | yes | yes | activation_policy.hpp, dense.hpp, dense_aux.hpp | denseAuxGraph.cpp, denseGraph.cpp |
71
+ | tensor_op | yes | master | 39e64de | yes | yes | flatten_chw.hpp | - |
72
+
73
+ ### CMake Targets
74
+
75
+ ```text
76
+ aie
77
+ aie-conv2d
78
+ aie-dense
79
+ aie-dense-aux
80
+ aie-depthwise
81
+ aiesim
82
+ aiesim-conv2d
83
+ aiesim-dense
84
+ aiesim-dense-aux
85
+ aiesim-depthwise
86
+ data
87
+ data-conv2d
88
+ data-dense
89
+ data-depthwise
90
+ gaia_activation_kernel
91
+ gaia_conv_kernel
92
+ gaia_dense_kernel
93
+ gaia_kernels
94
+ gaia_tensor_op_kernel
95
+ layer_aie
96
+ layer_aiesim
97
+ layer_data
98
+ layer_x86
99
+ layer_x86sim
100
+ readme-status
101
+ x86
102
+ x86-conv2d
103
+ x86-dense
104
+ x86-dense-aux
105
+ x86-depthwise
106
+ x86sim
107
+ x86sim-conv2d
108
+ x86sim-dense
109
+ x86sim-dense-aux
110
+ x86sim-depthwise
111
+ ```
112
+
113
+ ### Repository Layout
114
+
115
+ ```text
116
+ .
117
+ |-- examples
118
+ | |-- conv_flatten_dense_onnx_generate.py
119
+ | `-- dense_onnx_generate.py
120
+ |-- layers
121
+ | |-- activation
122
+ | | |-- Kernel
123
+ | | | `-- include
124
+ | | |-- .gitignore
125
+ | | |-- CMakeLists.txt
126
+ | | `-- README.md
127
+ | |-- conv
128
+ | | |-- Graph
129
+ | | | |-- convGraph.cpp
130
+ | | | |-- convGraph.hpp
131
+ | | | |-- depthwiseGraph.cpp
132
+ | | | `-- depthwiseGraph.hpp
133
+ | | |-- Kernel
134
+ | | | |-- include
135
+ | | | `-- src
136
+ | | |-- testVectors
137
+ | | | `-- makeDummyData.py
138
+ | | |-- .gitignore
139
+ | | |-- CMakeLists.txt
140
+ | | `-- README.md
141
+ | |-- dense
142
+ | | |-- Graph
143
+ | | | |-- denseAuxGraph.cpp
144
+ | | | |-- denseAuxGraph.hpp
145
+ | | | |-- denseGraph.cpp
146
+ | | | `-- denseGraph.hpp
147
+ | | |-- Kernel
148
+ | | | |-- include
149
+ | | | `-- src
150
+ | | |-- testVectors
151
+ | | | `-- makeDummyData.py
152
+ | | |-- .gitignore
153
+ | | |-- CMakeLists.txt
154
+ | | |-- notes.md
155
+ | | `-- README.md
156
+ | `-- tensor_op
157
+ | |-- Kernel
158
+ | | |-- include
159
+ | | `-- src
160
+ | |-- CMakeLists.txt
161
+ | `-- README.md
162
+ |-- python
163
+ | |-- gaia_ml
164
+ | | |-- __init__.py
165
+ | | |-- _version.py
166
+ | | |-- cli.py
167
+ | | |-- limits.py
168
+ | | |-- model_ir.py
169
+ | | |-- onnx_frontend.py
170
+ | | `-- project.py
171
+ | `-- gaia_ml.egg-info
172
+ | |-- dependency_links.txt
173
+ | |-- entry_points.txt
174
+ | |-- PKG-INFO
175
+ | |-- requires.txt
176
+ | |-- SOURCES.txt
177
+ | `-- top_level.txt
178
+ |-- tools
179
+ | `-- update_readme_status.py
180
+ |-- .gitignore
181
+ |-- .gitmodules
182
+ |-- CMakeLists.txt
183
+ |-- pyproject.toml
184
+ |-- README.md
185
+ `-- setup.cfg
186
+ ```
187
+ <!-- GAIA_STATUS_END -->
188
+
189
+ ## CMake hierarchy
190
+
191
+ Each layer repository can be configured and built on its own. In that mode, the
192
+ layer-local graph is used for `x86`, `aie`, `x86sim`, and `aiesim` targets.
193
+
194
+ From the top-level package, layer repositories are added as kernel providers.
195
+ The exported targets are:
196
+
197
+ ```text
198
+ gaia_activation_kernel
199
+ gaia_conv_kernel
200
+ gaia_dense_kernel
201
+ gaia_kernels
202
+ ```
203
+
204
+ Enable `GAIA_BUILD_LAYER_GRAPHS=ON` only when you also want the top-level build
205
+ to expose the layer-local smoke-test graph targets.
206
+
207
+ ## Notes
208
+
209
+ GAIA-ML v1.0 supports stream I/O only. Layer kernels use
210
+ `input_stream<int32>` and `output_stream<int32>` ports so the
211
+ top-level graph can be built around a single low-latency dataflow contract.
212
+
213
+ The current convolution stream kernels keep the existing frame-based compute
214
+ path internally by reading one streamed frame into local AIE memory before
215
+ computing. This preserves the stream interface for v1.0 while leaving room for
216
+ a later line-buffer implementation that can reduce first-output latency.
217
+
218
+ Activations are header-only compile-time policies in v1.0. Layers default to
219
+ `gaia::activation::identity` when the shared activation header is not on the
220
+ include path, and the top-level package exposes `activation.hpp` so graphs can
221
+ fuse policies such as `linear`, `relu`, `sigmoid`, `tanh`, or `softmax` into the
222
+ producer kernel instead of adding a separate activation tile.
223
+
224
+ ## Python Automation
225
+
226
+ The first automation scaffold is available as a pip-installable package:
227
+
228
+ ```bash
229
+ pip install -e .
230
+ gaia_ml version
231
+ gaia_ml inspect model.onnx
232
+ gaia_ml generate model.onnx --output build/generated_model --strategy throughput
233
+ ```
234
+
235
+ As of `gaia_ml` v0.5.0, generated project structure is emitted from Jinja2
236
+ templates packaged under `gaia_ml/templates/aie`. Backend Python code now builds
237
+ the graph/layer context and renders the project files from templates, so changes
238
+ to generated `Graph/`, `CMakeLists.txt`, and generated README layout no longer
239
+ need to be hard-coded directly into `project.py`.
240
+
241
+ As of `gaia_ml` v0.6.0, generated kernels keep the same stream interface but
242
+ avoid full input-frame caching where possible. Depthwise Conv uses a K-row line
243
+ buffer per channel; generic Conv and 1x1 pointwise Conv stream input rows and
244
+ accumulate output partials; Dense and Dense-CHW stream inputs directly into
245
+ output accumulators. This preserves the existing CHW padded stream contract
246
+ while reducing local input storage and preparing the backend for more aggressive
247
+ overlap.
248
+
249
+ Project generation supports two lowering strategies:
250
+
251
+ ```text
252
+ throughput Use more AIE tiles and preserve layer-level graph pipelining.
253
+ compact Fuse supported patterns to minimize tile count.
254
+ ```
255
+
256
+ `throughput` is the default because AIE latency often benefits from multiple
257
+ tiles running concurrently. `compact` is useful when fitting into the smallest
258
+ number of tiles matters more than first-output latency.
259
+
260
+ The current generator parses ONNX Conv, depthwise Conv, 1x1 pointwise Conv,
261
+ Flatten, and Dense/Gemm-style blocks, runs ONNX shape inference, checks what
262
+ can be emitted, and creates a self-contained AIE project with `Graph/`,
263
+ `Kernel/`, `weights/`, `testVectors/`, and `CMakeLists.txt`. Generation emits
264
+ one packed-output tile per supported compute block and chains them in model
265
+ order, unless a low-latency fused pattern is available:
266
+
267
+ ```text
268
+ input stream -> layer0 -> layer1 -> ... -> output stream
269
+ ```
270
+
271
+ `gaia_ml inspect` always reports optimization suggestions, including patterns
272
+ that are not generated yet. The CLI uses colored log levels for info, warning,
273
+ critical warning, and error messages so compatibility problems and latency
274
+ recommendations are easier to scan.
275
+
276
+ The current `int32` backend pads dense vectors and convolution rows to 8-lane
277
+ chunks. Dense blocks are rejected if either side exceeds 128 padded lanes.
278
+ Convolution blocks are currently single-tile only: input spatial dimensions are
279
+ limited to 32x32, kernels to 8x8, padded input/output rows to 128 lanes, and
280
+ padded frame buffers to the current local-memory guardrails. Larger layers need
281
+ an explicit tiling strategy before generation.
282
+
283
+ `Flatten` is represented in the Python IR as a tensor operation. When possible,
284
+ it is fused away: `Conv -> Flatten -> Dense` lowers to a `packed_dense_chw`
285
+ consumer that reads the padded CHW stream directly and skips row padding without
286
+ an extra AIE tile. The `tensor_op` layer also provides an explicit
287
+ `flatten_chw` kernel for future graph cases where a standalone reshuffle tile is
288
+ actually required. ONNX depthwise Conv lowers to `packed_depthwise_conv2d`, and
289
+ ONNX 1x1 Conv lowers to a specialized `packed_pointwise_conv2d` kernel. Weights
290
+ are emitted as headers under `weights/`, and the generated
291
+ `testVectors/reference_output.csv` contains the expected end-to-end inference
292
+ stream for the generated test input.
293
+ The first low-latency fused lowering is
294
+ `DepthwiseConv -> 1x1 Conv -> Flatten -> Dense(1)`, which emits
295
+ `packed_dw_pw_dense_scalar`. The generic kernels are still emitted in generated
296
+ projects so each kernel can continue to be developed and tested independently.
297
+ Generated graphs write simulator output to `testVectors/model_output.csv`; after
298
+ hardware AIE simulation this appears under
299
+ `aiesimulator_output/testVectors/model_output.csv`, while the expected output
300
+ remains in `testVectors/reference_output.csv`.
301
+ The generated project can be built with:
302
+
303
+ ```bash
304
+ cmake -S build/generated_model -B build/generated_model/build
305
+ cmake --build build/generated_model/build --target x86
306
+ cmake --build build/generated_model/build --target x86sim
307
+ ```
308
+
309
+ For now, project emission supports sequential Conv, Flatten, and Dense/Gemm
310
+ models with optional supported activations and zero/no bias. Branching, general
311
+ transpose/concat/split reshuffles, softmax fusion, and larger-than-single-tile
312
+ blocks still need explicit graph lowering rules before generation.
313
+
314
+ The `examples/conv_flatten_dense_onnx_generate.py` script builds a compact
315
+ performance-check model:
316
+
317
+ ```text
318
+ Depthwise 3x3 over 6 channels -> ReLU -> 1x1 pointwise Conv ->
319
+ Flatten -> Dense -> ReLU -> Dense logits
320
+ ```
321
+
322
+ It can also append ONNX Softmax with `--include-softmax` for semantic
323
+ inspection. GAIA-ML recognizes that Softmax but rejects project generation until
324
+ the backend has a whole-vector normalization kernel.
@@ -0,0 +1,291 @@
1
+ # GAIA-ML
2
+
3
+ Global AI-Engine Architecture - ML (GAIA-ML) is a software tool hoping to automatically transform ML algorithms into AMD AI-Engine (AIE) optimal implementations. GAIA-ML is developed within the ATLAS Global Trigger group, as the ML layers implemented are designed to provide optimal computation for low latency and high-throughput environments
4
+
5
+ Main developer: I. Xiotidis.
6
+
7
+ GAIA-ML is developed by the NGT WP2.1 Group.
8
+
9
+ License: MIT.
10
+
11
+ ## Framework infrastructure
12
+
13
+ GAIA-ML follows a layers approach where each layer of the ML architecture is coded in a single AIE kernel. Having each layer coded in a single layer doesn't mean that the final result will be deployed in different AIE tiles (parametric graph implementation).
14
+
15
+ <!-- GAIA_STATUS_START -->
16
+ ## Project Status
17
+
18
+ This section is generated from the repository contents. Refresh it with:
19
+
20
+ ```bash
21
+ cmake -S . -B build
22
+ cmake --build build --target readme-status
23
+ ```
24
+
25
+ or directly:
26
+
27
+ ```bash
28
+ python3 tools/update_readme_status.py
29
+ ```
30
+
31
+ ### Layers
32
+
33
+ | Layer | Submodule | Branch | SHA | CMake | Stream I/O | Kernel headers | Local graphs |
34
+ | --- | --- | --- | --- | --- | --- | --- | --- |
35
+ | activation | yes | master | bc7efd9 | yes | no | activation.hpp | - |
36
+ | conv | yes | cnn_v2 | 794c567 | yes | yes | activation_policy.hpp, conv2d.hpp, conv2d_depthwise.hpp | convGraph.cpp, depthwiseGraph.cpp |
37
+ | dense | yes | gaia_int | c2ae42f | yes | yes | activation_policy.hpp, dense.hpp, dense_aux.hpp | denseAuxGraph.cpp, denseGraph.cpp |
38
+ | tensor_op | yes | master | 39e64de | yes | yes | flatten_chw.hpp | - |
39
+
40
+ ### CMake Targets
41
+
42
+ ```text
43
+ aie
44
+ aie-conv2d
45
+ aie-dense
46
+ aie-dense-aux
47
+ aie-depthwise
48
+ aiesim
49
+ aiesim-conv2d
50
+ aiesim-dense
51
+ aiesim-dense-aux
52
+ aiesim-depthwise
53
+ data
54
+ data-conv2d
55
+ data-dense
56
+ data-depthwise
57
+ gaia_activation_kernel
58
+ gaia_conv_kernel
59
+ gaia_dense_kernel
60
+ gaia_kernels
61
+ gaia_tensor_op_kernel
62
+ layer_aie
63
+ layer_aiesim
64
+ layer_data
65
+ layer_x86
66
+ layer_x86sim
67
+ readme-status
68
+ x86
69
+ x86-conv2d
70
+ x86-dense
71
+ x86-dense-aux
72
+ x86-depthwise
73
+ x86sim
74
+ x86sim-conv2d
75
+ x86sim-dense
76
+ x86sim-dense-aux
77
+ x86sim-depthwise
78
+ ```
79
+
80
+ ### Repository Layout
81
+
82
+ ```text
83
+ .
84
+ |-- examples
85
+ | |-- conv_flatten_dense_onnx_generate.py
86
+ | `-- dense_onnx_generate.py
87
+ |-- layers
88
+ | |-- activation
89
+ | | |-- Kernel
90
+ | | | `-- include
91
+ | | |-- .gitignore
92
+ | | |-- CMakeLists.txt
93
+ | | `-- README.md
94
+ | |-- conv
95
+ | | |-- Graph
96
+ | | | |-- convGraph.cpp
97
+ | | | |-- convGraph.hpp
98
+ | | | |-- depthwiseGraph.cpp
99
+ | | | `-- depthwiseGraph.hpp
100
+ | | |-- Kernel
101
+ | | | |-- include
102
+ | | | `-- src
103
+ | | |-- testVectors
104
+ | | | `-- makeDummyData.py
105
+ | | |-- .gitignore
106
+ | | |-- CMakeLists.txt
107
+ | | `-- README.md
108
+ | |-- dense
109
+ | | |-- Graph
110
+ | | | |-- denseAuxGraph.cpp
111
+ | | | |-- denseAuxGraph.hpp
112
+ | | | |-- denseGraph.cpp
113
+ | | | `-- denseGraph.hpp
114
+ | | |-- Kernel
115
+ | | | |-- include
116
+ | | | `-- src
117
+ | | |-- testVectors
118
+ | | | `-- makeDummyData.py
119
+ | | |-- .gitignore
120
+ | | |-- CMakeLists.txt
121
+ | | |-- notes.md
122
+ | | `-- README.md
123
+ | `-- tensor_op
124
+ | |-- Kernel
125
+ | | |-- include
126
+ | | `-- src
127
+ | |-- CMakeLists.txt
128
+ | `-- README.md
129
+ |-- python
130
+ | |-- gaia_ml
131
+ | | |-- __init__.py
132
+ | | |-- _version.py
133
+ | | |-- cli.py
134
+ | | |-- limits.py
135
+ | | |-- model_ir.py
136
+ | | |-- onnx_frontend.py
137
+ | | `-- project.py
138
+ | `-- gaia_ml.egg-info
139
+ | |-- dependency_links.txt
140
+ | |-- entry_points.txt
141
+ | |-- PKG-INFO
142
+ | |-- requires.txt
143
+ | |-- SOURCES.txt
144
+ | `-- top_level.txt
145
+ |-- tools
146
+ | `-- update_readme_status.py
147
+ |-- .gitignore
148
+ |-- .gitmodules
149
+ |-- CMakeLists.txt
150
+ |-- pyproject.toml
151
+ |-- README.md
152
+ `-- setup.cfg
153
+ ```
154
+ <!-- GAIA_STATUS_END -->
155
+
156
+ ## CMake hierarchy
157
+
158
+ Each layer repository can be configured and built on its own. In that mode, the
159
+ layer-local graph is used for `x86`, `aie`, `x86sim`, and `aiesim` targets.
160
+
161
+ From the top-level package, layer repositories are added as kernel providers.
162
+ The exported targets are:
163
+
164
+ ```text
165
+ gaia_activation_kernel
166
+ gaia_conv_kernel
167
+ gaia_dense_kernel
168
+ gaia_kernels
169
+ ```
170
+
171
+ Enable `GAIA_BUILD_LAYER_GRAPHS=ON` only when you also want the top-level build
172
+ to expose the layer-local smoke-test graph targets.
173
+
174
+ ## Notes
175
+
176
+ GAIA-ML v1.0 supports stream I/O only. Layer kernels use
177
+ `input_stream<int32>` and `output_stream<int32>` ports so the
178
+ top-level graph can be built around a single low-latency dataflow contract.
179
+
180
+ The current convolution stream kernels keep the existing frame-based compute
181
+ path internally by reading one streamed frame into local AIE memory before
182
+ computing. This preserves the stream interface for v1.0 while leaving room for
183
+ a later line-buffer implementation that can reduce first-output latency.
184
+
185
+ Activations are header-only compile-time policies in v1.0. Layers default to
186
+ `gaia::activation::identity` when the shared activation header is not on the
187
+ include path, and the top-level package exposes `activation.hpp` so graphs can
188
+ fuse policies such as `linear`, `relu`, `sigmoid`, `tanh`, or `softmax` into the
189
+ producer kernel instead of adding a separate activation tile.
190
+
191
+ ## Python Automation
192
+
193
+ The first automation scaffold is available as a pip-installable package:
194
+
195
+ ```bash
196
+ pip install -e .
197
+ gaia_ml version
198
+ gaia_ml inspect model.onnx
199
+ gaia_ml generate model.onnx --output build/generated_model --strategy throughput
200
+ ```
201
+
202
+ As of `gaia_ml` v0.5.0, generated project structure is emitted from Jinja2
203
+ templates packaged under `gaia_ml/templates/aie`. Backend Python code now builds
204
+ the graph/layer context and renders the project files from templates, so changes
205
+ to generated `Graph/`, `CMakeLists.txt`, and generated README layout no longer
206
+ need to be hard-coded directly into `project.py`.
207
+
208
+ As of `gaia_ml` v0.6.0, generated kernels keep the same stream interface but
209
+ avoid full input-frame caching where possible. Depthwise Conv uses a K-row line
210
+ buffer per channel; generic Conv and 1x1 pointwise Conv stream input rows and
211
+ accumulate output partials; Dense and Dense-CHW stream inputs directly into
212
+ output accumulators. This preserves the existing CHW padded stream contract
213
+ while reducing local input storage and preparing the backend for more aggressive
214
+ overlap.
215
+
216
+ Project generation supports two lowering strategies:
217
+
218
+ ```text
219
+ throughput Use more AIE tiles and preserve layer-level graph pipelining.
220
+ compact Fuse supported patterns to minimize tile count.
221
+ ```
222
+
223
+ `throughput` is the default because AIE latency often benefits from multiple
224
+ tiles running concurrently. `compact` is useful when fitting into the smallest
225
+ number of tiles matters more than first-output latency.
226
+
227
+ The current generator parses ONNX Conv, depthwise Conv, 1x1 pointwise Conv,
228
+ Flatten, and Dense/Gemm-style blocks, runs ONNX shape inference, checks what
229
+ can be emitted, and creates a self-contained AIE project with `Graph/`,
230
+ `Kernel/`, `weights/`, `testVectors/`, and `CMakeLists.txt`. Generation emits
231
+ one packed-output tile per supported compute block and chains them in model
232
+ order, unless a low-latency fused pattern is available:
233
+
234
+ ```text
235
+ input stream -> layer0 -> layer1 -> ... -> output stream
236
+ ```
237
+
238
+ `gaia_ml inspect` always reports optimization suggestions, including patterns
239
+ that are not generated yet. The CLI uses colored log levels for info, warning,
240
+ critical warning, and error messages so compatibility problems and latency
241
+ recommendations are easier to scan.
242
+
243
+ The current `int32` backend pads dense vectors and convolution rows to 8-lane
244
+ chunks. Dense blocks are rejected if either side exceeds 128 padded lanes.
245
+ Convolution blocks are currently single-tile only: input spatial dimensions are
246
+ limited to 32x32, kernels to 8x8, padded input/output rows to 128 lanes, and
247
+ padded frame buffers to the current local-memory guardrails. Larger layers need
248
+ an explicit tiling strategy before generation.
249
+
250
+ `Flatten` is represented in the Python IR as a tensor operation. When possible,
251
+ it is fused away: `Conv -> Flatten -> Dense` lowers to a `packed_dense_chw`
252
+ consumer that reads the padded CHW stream directly and skips row padding without
253
+ an extra AIE tile. The `tensor_op` layer also provides an explicit
254
+ `flatten_chw` kernel for future graph cases where a standalone reshuffle tile is
255
+ actually required. ONNX depthwise Conv lowers to `packed_depthwise_conv2d`, and
256
+ ONNX 1x1 Conv lowers to a specialized `packed_pointwise_conv2d` kernel. Weights
257
+ are emitted as headers under `weights/`, and the generated
258
+ `testVectors/reference_output.csv` contains the expected end-to-end inference
259
+ stream for the generated test input.
260
+ The first low-latency fused lowering is
261
+ `DepthwiseConv -> 1x1 Conv -> Flatten -> Dense(1)`, which emits
262
+ `packed_dw_pw_dense_scalar`. The generic kernels are still emitted in generated
263
+ projects so each kernel can continue to be developed and tested independently.
264
+ Generated graphs write simulator output to `testVectors/model_output.csv`; after
265
+ hardware AIE simulation this appears under
266
+ `aiesimulator_output/testVectors/model_output.csv`, while the expected output
267
+ remains in `testVectors/reference_output.csv`.
268
+ The generated project can be built with:
269
+
270
+ ```bash
271
+ cmake -S build/generated_model -B build/generated_model/build
272
+ cmake --build build/generated_model/build --target x86
273
+ cmake --build build/generated_model/build --target x86sim
274
+ ```
275
+
276
+ For now, project emission supports sequential Conv, Flatten, and Dense/Gemm
277
+ models with optional supported activations and zero/no bias. Branching, general
278
+ transpose/concat/split reshuffles, softmax fusion, and larger-than-single-tile
279
+ blocks still need explicit graph lowering rules before generation.
280
+
281
+ The `examples/conv_flatten_dense_onnx_generate.py` script builds a compact
282
+ performance-check model:
283
+
284
+ ```text
285
+ Depthwise 3x3 over 6 channels -> ReLU -> 1x1 pointwise Conv ->
286
+ Flatten -> Dense -> ReLU -> Dense logits
287
+ ```
288
+
289
+ It can also append ONNX Softmax with `--include-softmax` for semantic
290
+ inspection. GAIA-ML recognizes that Softmax but rejects project generation until
291
+ the backend has a whole-vector normalization kernel.