embedl-deploy-tensorrt 0.0.0.dev0__tar.gz → 0.4.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- embedl_deploy_tensorrt-0.4.1/LICENSE +361 -0
- embedl_deploy_tensorrt-0.4.1/MANIFEST.in +13 -0
- embedl_deploy_tensorrt-0.4.1/NOTICE +1 -0
- embedl_deploy_tensorrt-0.4.1/PKG-INFO +201 -0
- embedl_deploy_tensorrt-0.4.1/README.md +182 -0
- embedl_deploy_tensorrt-0.4.1/pyproject.toml +45 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/__init__.py +23 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/__init__.py +3 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/__init__.py +3 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/backend.py +18 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/modules/__init__.py +3 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/modules/attention.py +309 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/modules/conv.py +222 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/modules/linear.py +159 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/modules/pointwise.py +38 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/modules/pool.py +25 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/modules/swin_attention.py +460 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/patterns/__init__.py +3 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/patterns/conversions/__init__.py +15 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/patterns/conversions/attention.py +954 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/patterns/conversions/general.py +345 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/patterns/fusions.py +217 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/patterns/quantizations.py +343 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/patterns/recompositions.py +1475 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/patterns/smoothings.py +143 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/patterns/utils.py +89 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/_internal/tensorrt/plan.py +127 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/tensorrt/__init__.py +59 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/tensorrt/modules/__init__.py +40 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/tensorrt/patterns/__init__.py +52 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/version/__init__.py +3 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy/version/public.py +5 -0
- embedl_deploy_tensorrt-0.4.1/src/embedl_deploy_tensorrt.egg-info/SOURCES.txt +31 -0
- embedl_deploy_tensorrt-0.0.0.dev0/PKG-INFO +0 -6
- embedl_deploy_tensorrt-0.0.0.dev0/embedl_deploy_tensorrt.egg-info/PKG-INFO +0 -6
- embedl_deploy_tensorrt-0.0.0.dev0/embedl_deploy_tensorrt.egg-info/SOURCES.txt +0 -5
- embedl_deploy_tensorrt-0.0.0.dev0/embedl_deploy_tensorrt.egg-info/dependency_links.txt +0 -1
- embedl_deploy_tensorrt-0.0.0.dev0/embedl_deploy_tensorrt.egg-info/top_level.txt +0 -1
- embedl_deploy_tensorrt-0.0.0.dev0/pyproject.toml +0 -10
- {embedl_deploy_tensorrt-0.0.0.dev0 → embedl_deploy_tensorrt-0.4.1}/setup.cfg +0 -0
|
@@ -0,0 +1,361 @@
|
|
|
1
|
+
Copyright (C) 2026 Embedl AB
|
|
2
|
+
|
|
3
|
+
EMBEDL COMMUNITY LICENSE AGREEMENT v.1.0
|
|
4
|
+
|
|
5
|
+
Embedl is pleased to make available Embedl Deploy and source code files under
|
|
6
|
+
this Community License Agreement (the "License" or the "Terms"). Our goal is to
|
|
7
|
+
foster a vibrant community of developers, researchers, and organizations who
|
|
8
|
+
can freely use, modify, and build upon the tools and optimized AI models. To
|
|
9
|
+
that end, this license permits unlimited use, distribution and
|
|
10
|
+
commercialization for most individuals and organizations, subject only to the
|
|
11
|
+
limited restrictions described below.
|
|
12
|
+
|
|
13
|
+
To sustain our ability to continue developing and releasing tools for
|
|
14
|
+
optimizing AI models, we have established three limitations on use. First,
|
|
15
|
+
competitors to Embedl may not use the licensed materials, as permitting such
|
|
16
|
+
use would undermine our ability to invest in future development. Second, larger
|
|
17
|
+
organizations—those with more than 250 employees or annual revenue exceeding
|
|
18
|
+
€10 million—can use the licensed materials for internal evaluation purposes,
|
|
19
|
+
but must obtain a separate Commercial Use license before using the licensed
|
|
20
|
+
materials for commercial purposes; this ensures that entities with greater
|
|
21
|
+
resources contribute appropriately to the ecosystem. Third, no licensee may
|
|
22
|
+
host the licensed materials as a service, which protects both Embedl's
|
|
23
|
+
interests and the integrity of the community by preventing commoditization of
|
|
24
|
+
these models (although specific use cases are possible with prior approval
|
|
25
|
+
from us).
|
|
26
|
+
|
|
27
|
+
We recognize that individual circumstances vary, and we welcome inquiries from
|
|
28
|
+
those who wish to discuss alternative licensing arrangements that fall outside
|
|
29
|
+
these terms. Please contact Embedl at sales@embedl.com to discuss your
|
|
30
|
+
specific use case. We would be happy to work with you.
|
|
31
|
+
|
|
32
|
+
1.0 DEFINED TERMS
|
|
33
|
+
|
|
34
|
+
"Commercial Use": (a) any provision of the functionality of Embedl Deploy, in
|
|
35
|
+
whole or in part, in any product, offering, consultancy, or as a service; (b)
|
|
36
|
+
any Distribution; (c) any use of Embedl Deploy in whole or in part for
|
|
37
|
+
direct or indirect strategic or commercial gain or advantage; or (d) any use of
|
|
38
|
+
Embedl Deploy or any of its outputs or results, in whole or in part, for
|
|
39
|
+
purposes of training or improving AI models.
|
|
40
|
+
|
|
41
|
+
"Competitor": any person or entity in the business of selling access to AI
|
|
42
|
+
development and optimization tools.
|
|
43
|
+
|
|
44
|
+
"Contribution": any contribution to Embedl Deploy that is intentionally
|
|
45
|
+
submitted for inclusion in Embedl Deploy by the contribution owner (copyright
|
|
46
|
+
and otherwise), or by a person or entity authorized to submit the contribution
|
|
47
|
+
on behalf of such owner. For the purposes of this definition, "submitted" means
|
|
48
|
+
any form of electronic, verbal, or written communication sent to us, including
|
|
49
|
+
but not limited to communication on electronic mailing lists, source code
|
|
50
|
+
control systems, and issue tracking systems, but excluding communication that
|
|
51
|
+
is conspicuously marked or otherwise designated in writing by the contribution
|
|
52
|
+
owner as "Not a Contribution."
|
|
53
|
+
|
|
54
|
+
"Contributor": any person or entity submitting a Contribution that has been
|
|
55
|
+
accepted by us and incorporated within Embedl Deploy.
|
|
56
|
+
|
|
57
|
+
"Distribution" or "Distribute": means any transmission, distribution,
|
|
58
|
+
deployment, publication, or other sharing of Embedl Deploy in whole or in part,
|
|
59
|
+
whether standalone or as incorporated or bundled with other works, services,
|
|
60
|
+
or products, to any third party, including by providing or making Embedl Deploy
|
|
61
|
+
or its functionality available as a hosted service via API, web access, or any
|
|
62
|
+
other electronic or remote means.
|
|
63
|
+
|
|
64
|
+
"Embedl Deploy": the software, algorithms, code, generated models and weights,
|
|
65
|
+
licensed by us under these Terms, as any of the foregoing may be updated or
|
|
66
|
+
modified by us on one or more occasions.
|
|
67
|
+
|
|
68
|
+
"We," "us," "our," or "Embedl": Embedl AB, organized under the laws of Sweden.
|
|
69
|
+
|
|
70
|
+
"You" or "your": the person, and any entity such person works for or on behalf
|
|
71
|
+
of, exercising the rights of this license, excluding any Competitor.
|
|
72
|
+
|
|
73
|
+
2.0 LICENSE RIGHTS
|
|
74
|
+
|
|
75
|
+
2.1 License Grant. Subject to your compliance with section 3.0, including the
|
|
76
|
+
Commercial Use and Competitor limitations below, and the Third Party Terms, we
|
|
77
|
+
hereby grant to you a worldwide, royalty-free, sublicensable, non-transferable,
|
|
78
|
+
and non-exclusive copyright license to use, reproduce, distribute, display,
|
|
79
|
+
perform, modify, and create derivative works of Embedl Deploy in whole or in
|
|
80
|
+
part.
|
|
81
|
+
|
|
82
|
+
2.2 Contributor License Grant. Subject to your compliance with these Terms,
|
|
83
|
+
including the Commercial Use and Competitor limitations below, and the Third
|
|
84
|
+
Party Terms, each Contributor hereby grants to you a worldwide, non-exclusive,
|
|
85
|
+
royalty-free, sublicensable, and non-transferable
|
|
86
|
+
(a) copyright license to use, reproduce, distribute, display, perform, modify,
|
|
87
|
+
and create derivative works of Embedl Deploy in whole or in part; and
|
|
88
|
+
(b) patent license to make, have made, use, offer to sell, sell, import and
|
|
89
|
+
otherwise transfer Embedl Deploy in whole or in part, but only to the extent
|
|
90
|
+
of patent claims licensable by such Contributor that are necessarily infringed
|
|
91
|
+
by the exercise of such rights, including for standalone Contributions and
|
|
92
|
+
Contributions in combination with Embedl Deploy.
|
|
93
|
+
|
|
94
|
+
2.3 Reservation of Rights. All rights not expressly granted in this section
|
|
95
|
+
are reserved to us. You will have no right or license to Embedl Deploy other
|
|
96
|
+
than the rights set forth in this section 2.0, and no other right or license
|
|
97
|
+
will be implied, by estoppel, conduct, or otherwise. If you are an entity, you
|
|
98
|
+
may exercise the rights granted in this section solely via your own employees
|
|
99
|
+
and contractors. We and our licensors retain all right, title and interest in
|
|
100
|
+
Embedl Deploy and associated intellectual property rights, and all copies of
|
|
101
|
+
Embedl Deploy in whole or in part, and we grant no rights to our patents.
|
|
102
|
+
|
|
103
|
+
3.0 LICENSE CONDITIONS
|
|
104
|
+
|
|
105
|
+
3.1 No Use by Competitors. If you are, or your employer or client for whom
|
|
106
|
+
you work is, a Competitor, you are not licensed to download, license, install
|
|
107
|
+
or use Embedl Deploy. You represent and warrant that you are not a Competitor
|
|
108
|
+
and are not working for a Competitor. If you are in breach of this warranty,
|
|
109
|
+
now or in the future, you are not licensed to use Embedl Deploy; doing so
|
|
110
|
+
anyway constitutes breach of these Terms and infringement of our intellectual
|
|
111
|
+
property rights.
|
|
112
|
+
|
|
113
|
+
3.2 Commercial Use Limitation.
|
|
114
|
+
|
|
115
|
+
a) The licensed rights set forth in section 2.0 exclude Commercial Use if you
|
|
116
|
+
are an entity, or an affiliate or member of a company group, that has more than
|
|
117
|
+
250 employees or annual revenue of €10,000,000 or more ("Large Entity"). If you
|
|
118
|
+
are or become a Large Entity or part of a group of one, Commercial Use is not
|
|
119
|
+
or is no longer licensed to you under these Terms; you must contact us for a
|
|
120
|
+
Commercial Use license. You represent and warrant that you are not a Large
|
|
121
|
+
Entity and are not working for a Large Entity. If you are in breach of this
|
|
122
|
+
warranty, now or in the future, you are not licensed to use Embedl Deploy for
|
|
123
|
+
Commercial Use; doing so anyway constitutes breach of these Terms and
|
|
124
|
+
infringement of our intellectual property rights.
|
|
125
|
+
|
|
126
|
+
b) In addition to any other remedies, if you violate section 3.2(a), you hereby
|
|
127
|
+
grant to Embedl on behalf of yourself and each of your affiliates, employees
|
|
128
|
+
and contractors, a non-exclusive, worldwide, perpetual, irrevocable,
|
|
129
|
+
non-terminable, sublicensable (through multiple tiers), fully paid-up,
|
|
130
|
+
royalty-free, assignable and transferable intellectual property rights license
|
|
131
|
+
as to Distributions, to make, use, reproduce, modify, adapt, create derivative
|
|
132
|
+
works of, improve, extend, enhance, translate, distribute (directly and
|
|
133
|
+
indirectly, in any medium, under terms of choice), combine, compile, transmit,
|
|
134
|
+
display and perform publicly, license, rent, lease, and manufacture, sell,
|
|
135
|
+
offer to sell, and import, your Distributions, for any commercial or
|
|
136
|
+
non-commercial purpose.
|
|
137
|
+
|
|
138
|
+
3.3 No Use as a Hosted Service. You may not, without our explicit permission,
|
|
139
|
+
use the Embedl
|
|
140
|
+
Deploy, any derivative works thereof, or any models created or trained via the
|
|
141
|
+
use or incorporation of Embedl Deploy, to provide a hosted service,
|
|
142
|
+
software-as-a-service offering, API, or any other mechanism that allows third
|
|
143
|
+
parties to access or use the functionality of such materials, or to submit
|
|
144
|
+
inputs to, receive outputs from, Embedl Deploy, whether or not for a fee.
|
|
145
|
+
Notwithstanding the foregoing, you may use Embedl Deploy on an internal
|
|
146
|
+
network accessible solely to your employees and contractors performing work
|
|
147
|
+
exclusively for you, solely for your internal business
|
|
148
|
+
purposes.
|
|
149
|
+
|
|
150
|
+
In addition, we are open to hosted use in support of a specific function or
|
|
151
|
+
feature in a separate application, depending on the context, on a case-by-case
|
|
152
|
+
basis. Please contact us, and we would be happy to explore that.
|
|
153
|
+
|
|
154
|
+
3.4 Required Notices and Pass-Throughs.
|
|
155
|
+
|
|
156
|
+
a) You will preserve and not suppress proprietary notices, markings, and
|
|
157
|
+
branding visible in or associated with or displayed via the use of Embedl
|
|
158
|
+
Deploy, and in any Distributions.
|
|
159
|
+
|
|
160
|
+
b) You will give each recipient of your Distribution a copy of these Terms and
|
|
161
|
+
the following notice: "Embedl Deploy is licensed under the Embedl Community
|
|
162
|
+
Source License v.1, and the terms of Embedl's licensors.
|
|
163
|
+
Copyright © Embedl AB."
|
|
164
|
+
|
|
165
|
+
c) You must cause any modified files in your Distributions to carry prominent
|
|
166
|
+
notices explaining the changes that were made.
|
|
167
|
+
|
|
168
|
+
d) You must require each recipient to a Distribution to agree in writing (or
|
|
169
|
+
through another mechanism establishing enforceable assent) to be bound by the
|
|
170
|
+
terms and restrictions of this section, section 3.0 and section 5.2, and to
|
|
171
|
+
impose the same requirement on any subsequent recipients, so that each
|
|
172
|
+
recipient in the chain of distribution and/or use is bound by such terms and
|
|
173
|
+
restrictions.
|
|
174
|
+
|
|
175
|
+
3.5 Contributions. Each Contributor hereby grants to us a worldwide,
|
|
176
|
+
non-exclusive, royalty-free, sublicensable, and non-transferable
|
|
177
|
+
(a) copyright license to use, reproduce, distribute, display,
|
|
178
|
+
perform, modify, and create derivative works of each Contribution; and
|
|
179
|
+
(b) patent license to make, have made, use, offer to sell, sell, import and
|
|
180
|
+
otherwise transfer Embedl Deploy, but only to the extent of patent claims
|
|
181
|
+
licensable by such Contributor that are necessarily infringed by the exercise
|
|
182
|
+
of such rights, for standalone Contributions or in Contributions in combination
|
|
183
|
+
with Embedl Deploy. By submitting a Contribution, you represent and warrant
|
|
184
|
+
that you have sufficient rights to grant the licenses under these Terms, and
|
|
185
|
+
that doing so does not violate any obligation to a third party.
|
|
186
|
+
|
|
187
|
+
3.6 Our Marks. You agree that any use of our marks, branding and logos,
|
|
188
|
+
including Embedl Deploy ("Marks"), whether permitted or otherwise, will inure
|
|
189
|
+
to our sole benefit. You will not directly or indirectly: (a) file or prepare
|
|
190
|
+
any application for registration of any Marks; (b) assert any right, title,
|
|
191
|
+
license to, or interest in the Marks; or (c) adopt, use, file for registration,
|
|
192
|
+
or register, in whole or in part, any trademark, service mark, trade name,
|
|
193
|
+
logo, or domain name which may be confusingly similar to or an infringement of
|
|
194
|
+
the Marks or any of our domain names.
|
|
195
|
+
|
|
196
|
+
3.7 Feedback. You agree that we may freely use and exploit in perpetuity any
|
|
197
|
+
feedback, requirements, recommendations, ideas, bug fixes, reviews, ratings,
|
|
198
|
+
comments, suggestions, or improvements, that you, or any employee or agent
|
|
199
|
+
thereof, may at any time disclose or submit to us relating to Embedl Deploy
|
|
200
|
+
for our business purposes, including for product, services or solution
|
|
201
|
+
sales, licensing, support and development, without any obligation or payment
|
|
202
|
+
to you.
|
|
203
|
+
|
|
204
|
+
3.8 Unauthorized Use. You will not use Embedl Deploy or participate in any
|
|
205
|
+
activities via Embedl Deploy in a manner that is likely to be prohibited by
|
|
206
|
+
law or these Terms or violative of third party rights in any applicable
|
|
207
|
+
jurisdiction, including intellectual property and data privacy rights. Your
|
|
208
|
+
use of Embedl Deploy must be in full compliance with applicable law and the
|
|
209
|
+
Third Party Terms. You acknowledge that use of Embedl Deploy may be subject
|
|
210
|
+
to the EU AI Act and other AI-specific laws and regulations. You are solely
|
|
211
|
+
responsible for determining whether your use or Distribution constitutes a
|
|
212
|
+
high-risk AI system or is otherwise subject to regulatory obligations,
|
|
213
|
+
and for compliance with all applicable requirements.
|
|
214
|
+
|
|
215
|
+
3.9 Notifications to Us. If you believe that you are entitled or obligated to
|
|
216
|
+
act contrary to these Terms under any mandatory or applicable law, you agree to
|
|
217
|
+
provide us with detailed and substantiated explanation of your reasons in
|
|
218
|
+
writing at least thirty days before you act, to allow us to assess whether we
|
|
219
|
+
may, at our sole discretion, provide an alternative remedy for the situation,
|
|
220
|
+
though we are under no obligation to do so. To be effective, notices to us must
|
|
221
|
+
be sent to legal@embedl.com.
|
|
222
|
+
|
|
223
|
+
4.0 DISCLAIMER OF WARRANTIES; LIMITATION OF LIABILITY
|
|
224
|
+
|
|
225
|
+
4.1 Disclaimer. TO THE MAXIMUM EXTENT POSSIBLE UNDER APPLICABLE LAW, we
|
|
226
|
+
provide Embedl Deploy on an as-is, as-available basis with all faults, and WE
|
|
227
|
+
DISCLAIM ALL WARRANTIES OF ANY KIND WITH RESPECT TO EMBEDL DEPLOY, WHETHER
|
|
228
|
+
EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF
|
|
229
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT.
|
|
230
|
+
Specifically, we make no warranty that
|
|
231
|
+
|
|
232
|
+
(a) Embedl Deploy will meet your requirements, goals or needs, or
|
|
233
|
+
(b) any errors or deficiencies in Embedl Deploy will be corrected.
|
|
234
|
+
|
|
235
|
+
4.2 Exclusion. Notwithstanding any other provision of these Terms, we will
|
|
236
|
+
not be liable for any losses or damages, whether direct, indirect, punitive,
|
|
237
|
+
special, incidental or consequential damages, or liable for interruption of
|
|
238
|
+
business, diminution of value, cost of replacement, downtime, or loss of
|
|
239
|
+
profits, revenue, use, data, or other economic advantage, in connection with,
|
|
240
|
+
related to or arising out of Embedl Deploy or these Terms, regardless of the
|
|
241
|
+
theory of liability, whether for breach of these Terms, including breach of
|
|
242
|
+
warranty, or in tort or otherwise, even if we have been previously advised of
|
|
243
|
+
the possibility of such damages and even if such damages are reasonably
|
|
244
|
+
foreseeable. Liability for damages will be so limited and excluded, regardless
|
|
245
|
+
of the validity or efficacy of any remedy provided herein and even if any
|
|
246
|
+
remedy fails of its essential purpose. Nothing in these Terms excludes or
|
|
247
|
+
limits liability for death or personal injury caused by negligence, fraud,
|
|
248
|
+
intentional misconduct, or gross negligence, or any other liability that cannot
|
|
249
|
+
be excluded under applicable law.
|
|
250
|
+
|
|
251
|
+
4.3 Indemnification. You hereby agree to defend, indemnify, release and hold
|
|
252
|
+
harmless Embedl, and its officers, directors, employees, consultants, agents,
|
|
253
|
+
and representatives, in full from and against any and all claims (including
|
|
254
|
+
third party claims), losses, liability, damages, and/or costs of every kind and
|
|
255
|
+
nature, arising out of or in any way connected with: (a) your access to or use
|
|
256
|
+
of, Embedl Deploy or conduct engendered thereby, (b) your breach of these Terms
|
|
257
|
+
or the Third Party Terms, (c) your infringement or misappropriation of any
|
|
258
|
+
intellectual property right or other right of any person or entity, or
|
|
259
|
+
(d) your violations of applicable law.
|
|
260
|
+
|
|
261
|
+
5.0 TERMINATION
|
|
262
|
+
|
|
263
|
+
5.1 Upon Breach. The licenses granted by these Terms will terminate
|
|
264
|
+
automatically if you are in breach of these Terms, including if you exceed the
|
|
265
|
+
scope of the licenses granted in section 2.0. Breach of these Terms may result
|
|
266
|
+
in pursuit of all available remedies for intellectual property rights
|
|
267
|
+
(including intellectual property rights infringement), the availability of
|
|
268
|
+
which you hereby acknowledge.
|
|
269
|
+
|
|
270
|
+
5.2 Upon Litigation. The license rights set forth in section 2.0 will
|
|
271
|
+
terminate automatically if you institute litigation (including a cross-claim
|
|
272
|
+
or counterclaim in a lawsuit) against any entity alleging that the use,
|
|
273
|
+
distribution, modification, sale, offer to sell, transfer or import of Embedl
|
|
274
|
+
Deploy, any Contribution (alone or in combination), or the outputs or results
|
|
275
|
+
of Embedl Deploy, constitutes direct, indirect, vicarious, or contributory
|
|
276
|
+
infringement of intellectual property rights.
|
|
277
|
+
|
|
278
|
+
5.3 By Us. We may on written notice terminate rights granted under these
|
|
279
|
+
Terms in the event the Embedl Deploy become subject to third party claims of
|
|
280
|
+
infringement or violation of intellectual property or other rights or
|
|
281
|
+
applicable law.
|
|
282
|
+
|
|
283
|
+
5.4 Effect. Upon termination of rights for any reason, all licensed rights
|
|
284
|
+
granted under these Terms will terminate, and you will immediately cease all
|
|
285
|
+
use, and destroy all and not retain any copies, of Embedl Deploy. Our rights
|
|
286
|
+
under this section 5.0 are in addition to any other rights and remedies
|
|
287
|
+
permitted by law or under these Terms.
|
|
288
|
+
|
|
289
|
+
6.0 GENERAL PROVISIONS
|
|
290
|
+
|
|
291
|
+
6.1 Governing Law. These Terms and any disputes between us and related to or
|
|
292
|
+
concerning these Terms or Embedl Deploy (including tort as well as contract
|
|
293
|
+
claims, and whether pre-contractual or extra-contractual), notwithstanding the
|
|
294
|
+
choice of laws rules of any jurisdiction to the contrary, will be governed by
|
|
295
|
+
the procedural and substantive laws of Sweden. The United Nations Convention on
|
|
296
|
+
Contracts for the International Sale of Goods will not apply and is hereby
|
|
297
|
+
excluded.
|
|
298
|
+
|
|
299
|
+
6.2 Arbitration. Subject to section 6.3, any disputes between or claims
|
|
300
|
+
brought by either party arising out of or related to Embedl Deploy or these
|
|
301
|
+
Terms shall be referred to and finally settled exclusively by binding
|
|
302
|
+
arbitration before the Arbitration Institute of the Stockholm Chamber of
|
|
303
|
+
Commerce (the "SCC"), and the venue for arbitration will be Stockholm, Sweden.
|
|
304
|
+
The Rules for Expedited Arbitrations will apply, unless the SCC in its
|
|
305
|
+
discretion determines, taking into account the complexity of the case, the
|
|
306
|
+
amount in dispute and other circumstances, that the Arbitration Rules will
|
|
307
|
+
apply. In the latter case, the SCC will also decide whether the Arbitral
|
|
308
|
+
Tribunal will be composed of one or three arbitrators. The arbitration will be
|
|
309
|
+
conducted by telephone, on-line and/or based solely upon written submissions
|
|
310
|
+
where no in-person appearance is required. If in-person appearance is required,
|
|
311
|
+
such hearings will be held in Stockholm, Sweden. The arbitrator will apply
|
|
312
|
+
the law specified in section 6.1. All awards may if necessary be enforced by
|
|
313
|
+
any court having jurisdiction. The existence of any dispute, the existence or
|
|
314
|
+
details of the arbitration proceeding, and all related documents, materials,
|
|
315
|
+
evidence, judgments and awards therein, shall be kept confidential. Except as
|
|
316
|
+
required by law, no party will make any public announcements with respect to
|
|
317
|
+
the other party or the proceeding or the award, except as required to enforce
|
|
318
|
+
same. All disputes will be arbitrated only on an individual basis and not in a
|
|
319
|
+
class, consolidated or representative action. The arbitrator does not have the
|
|
320
|
+
power to vary these provisions. Subject to section 6.3, all claims between the
|
|
321
|
+
parties must be resolved via arbitration in accordance with this section.
|
|
322
|
+
Should either party file an action contrary to this section, the other party
|
|
323
|
+
may recover lawyers' fees and costs associated with enforcing this section,
|
|
324
|
+
provided that the party seeking the award has notified the other party in
|
|
325
|
+
writing of the improperly filed claim, and the other party has failed to
|
|
326
|
+
withdraw the claim in a timely fashion.
|
|
327
|
+
|
|
328
|
+
6.3 Injunctive Relief; Prevailing Party. You acknowledge and agree that
|
|
329
|
+
breach of these Terms, or any unauthorized use or distribution of Embedl
|
|
330
|
+
Deploy, may cause irreparable harm to us, the extent of which would be
|
|
331
|
+
difficult to ascertain, and that we will be entitled to seek immediate
|
|
332
|
+
injunctive relief and specific performance (in addition to any other available
|
|
333
|
+
remedies), and to enforce and/or seek redress for infringement of intellectual
|
|
334
|
+
property rights, in any court of competent jurisdiction under the applicable
|
|
335
|
+
laws thereto (and such proceeding, and our right to prosecute such a claim,
|
|
336
|
+
is not subject to arbitration). You hereby consent to the jurisdiction of
|
|
337
|
+
such courts for any such actions. A party prevailing in any litigation or
|
|
338
|
+
arbitration related to these Terms or Embedl Deploy will be entitled, in
|
|
339
|
+
addition to such other relief as may be granted, to an award of reasonable
|
|
340
|
+
attorneys' fees. This section is in addition and without prejudice to the
|
|
341
|
+
availability of attorneys' fees as a remedy under applicable law, including
|
|
342
|
+
laws pertaining to intellectual property rights.
|
|
343
|
+
|
|
344
|
+
6.4 Important Final Terms. The Terms constitute the entire agreement between
|
|
345
|
+
you and us and govern your use of Embedl Deploy, superseding any prior
|
|
346
|
+
agreements, understandings, communications or proposals; except that these
|
|
347
|
+
Terms do not override, supersede, or modify the terms of any written license
|
|
348
|
+
agreement signed by you and us. If any provision of the Terms is found by a
|
|
349
|
+
court of competent jurisdiction to be invalid, the parties nevertheless agree
|
|
350
|
+
that the court should endeavor to give effect to the parties' intentions as
|
|
351
|
+
reflected in the provision, and the other provisions of the Terms will remain
|
|
352
|
+
in full force and effect. No waiver of any provision of these Terms will be
|
|
353
|
+
deemed a further waiver or continuing waiver or such provision or any other
|
|
354
|
+
provision, and our failure to assert any right or provision under these Terms
|
|
355
|
+
will not constitute a waiver of such right or provision. Nothing herein will
|
|
356
|
+
be deemed to create an agency, partnership, joint venture, or
|
|
357
|
+
franchisor-franchisee relationship of any kind between us and any user or other
|
|
358
|
+
person or entity, nor do these terms extend rights to any third party. These
|
|
359
|
+
Terms are in the English language only, which language will be controlling in
|
|
360
|
+
all respects, and all versions of these Terms in any other language will be
|
|
361
|
+
for accommodation only and will not be binding on you or us.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
prune *
|
|
2
|
+
include src/embedl_deploy/__init__.py
|
|
3
|
+
include src/embedl_deploy/_internal/__init__.py
|
|
4
|
+
include src/embedl_deploy/version/__init__.py
|
|
5
|
+
include src/embedl_deploy/version/public.py
|
|
6
|
+
include LICENSE
|
|
7
|
+
include NOTICE
|
|
8
|
+
include README.md
|
|
9
|
+
graft src/embedl_deploy/tensorrt
|
|
10
|
+
graft src/embedl_deploy/_internal/tensorrt
|
|
11
|
+
global-exclude CLAUDE.md
|
|
12
|
+
global-exclude *.pyc
|
|
13
|
+
global-exclude __pycache__
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
Copyright (C) 2026 Embedl AB
|
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: embedl-deploy-tensorrt
|
|
3
|
+
Version: 0.4.1
|
|
4
|
+
Summary: TensorRT backend for embedl-deploy.
|
|
5
|
+
Author-email: Embedl AB <support@embedl.com>
|
|
6
|
+
Project-URL: Homepage, https://www.embedl.com/
|
|
7
|
+
Classifier: Programming Language :: Python :: 3
|
|
8
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
9
|
+
Classifier: License :: Other/Proprietary License
|
|
10
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
11
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
12
|
+
Requires-Python: >=3.10
|
|
13
|
+
Description-Content-Type: text/markdown
|
|
14
|
+
License-File: LICENSE
|
|
15
|
+
License-File: NOTICE
|
|
16
|
+
Provides-Extra: core
|
|
17
|
+
Requires-Dist: embedl-deploy; extra == "core"
|
|
18
|
+
Dynamic: license-file
|
|
19
|
+
|
|
20
|
+
# embedl-deploy
|
|
21
|
+
|
|
22
|
+
Python package to make AI models deployment-ready for any hardware.
|
|
23
|
+
|
|
24
|
+
## Why embedl-deploy
|
|
25
|
+
|
|
26
|
+
PyTorch models are flexible, but edge hardware is not. Hardware toolchains may
|
|
27
|
+
fail due to unsupported operators, apply implicit transformations and fusions
|
|
28
|
+
during compilation and quantization leading to deployment issues.
|
|
29
|
+
|
|
30
|
+
`embedl-deploy` eliminates these surprises by enforcing hardware and compiler
|
|
31
|
+
constraints directly in PyTorch, so what you build, train, and debug is what
|
|
32
|
+
actually runs on the device. It converts your models to be compatible for the
|
|
33
|
+
hardware target ensuring correct quantization and compilation.
|
|
34
|
+
|
|
35
|
+
## Features
|
|
36
|
+
|
|
37
|
+
- **Hardware-accurate PyTorch Intermediate Representation (IR):**
|
|
38
|
+
Build models using a hardware-aware PyTorch intermediate representation that
|
|
39
|
+
mirrors the behavior of the compiled artifact, e.g., fused convolutions.
|
|
40
|
+
Unsupported operators and compatibility issues are surfaced early and
|
|
41
|
+
resolved explicitly before compilation, within PyTorch.
|
|
42
|
+
|
|
43
|
+
- **Quantization:**
|
|
44
|
+
Supports post-training quantization (PTQ) and quantization-aware-training
|
|
45
|
+
(QAT). Fake quantization is applied in PyTorch with explicit quantization
|
|
46
|
+
operator placements. PTQ methods are included and QAT can be applied directly
|
|
47
|
+
to the transformed and quantized models with no additional dependencies
|
|
48
|
+
required.
|
|
49
|
+
|
|
50
|
+
- **Guaranteed deployable artifacts:**
|
|
51
|
+
Produce optimized compilation artifacts ready for deployment on the target
|
|
52
|
+
device with predictable performance and accuracy.
|
|
53
|
+
|
|
54
|
+
|
|
55
|
+
## Supported Backends
|
|
56
|
+
|
|
57
|
+
| Backend | Status |
|
|
58
|
+
|-------------------------|-------------|
|
|
59
|
+
| NVIDIA TensorRT (v10.3) | Supported |
|
|
60
|
+
|
|
61
|
+
Contact Embedl for other backends.
|
|
62
|
+
|
|
63
|
+
## Installation
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
pip install "embedl-deploy[tensorrt]"
|
|
67
|
+
```
|
|
68
|
+
Note that you may need to also install `onnx` and `onnx-simplifier` to export
|
|
69
|
+
and get the exported model compiled with TensorRT if using ONNX as an
|
|
70
|
+
intermediate.
|
|
71
|
+
|
|
72
|
+
---
|
|
73
|
+
|
|
74
|
+
## Quick Start
|
|
75
|
+
|
|
76
|
+
```python
|
|
77
|
+
import torch
|
|
78
|
+
from embedl_deploy import transform
|
|
79
|
+
from embedl_deploy.quantize import quantize
|
|
80
|
+
from embedl_deploy.tensorrt import TENSORRT_PATTERNS
|
|
81
|
+
from torchvision.models import resnet18 as Model
|
|
82
|
+
|
|
83
|
+
# 1. Load a standard PyTorch model
|
|
84
|
+
model = Model().eval()
|
|
85
|
+
example_input = torch.randn(1, 3, 224, 224)
|
|
86
|
+
|
|
87
|
+
# 2. Transform — fuse and optimize for TensorRT in one call
|
|
88
|
+
# For more compatibilty you can trace your model with torch.export.export
|
|
89
|
+
# as follows:
|
|
90
|
+
# model = torch.export.export(model, (example_input)).module()
|
|
91
|
+
res = transform(model, patterns=TENSORRT_PATTERNS)
|
|
92
|
+
print("Model\n", res.model.print_readable())
|
|
93
|
+
print("Matches", "\n".join([str(match) for match in res.matches]))
|
|
94
|
+
|
|
95
|
+
|
|
96
|
+
# 3. Quantize (PTQ)
|
|
97
|
+
def calibration_loop(model: torch.fx.GraphModule):
|
|
98
|
+
model.eval()
|
|
99
|
+
for _ in range(100):
|
|
100
|
+
model(torch.randn(1, 3, 224, 224))
|
|
101
|
+
|
|
102
|
+
|
|
103
|
+
quantized_model = quantize(
|
|
104
|
+
res.model, (example_input,), forward_loop=calibration_loop
|
|
105
|
+
)
|
|
106
|
+
quantized_model.eval()
|
|
107
|
+
|
|
108
|
+
# 4. Export as usual (dynamo exported models may have compilation issues)
|
|
109
|
+
torch.onnx.export(
|
|
110
|
+
quantized_model, (example_input,), "model.onnx", dynamo=False
|
|
111
|
+
)
|
|
112
|
+
|
|
113
|
+
# 5. Quantization-aware training with a training loop
|
|
114
|
+
qat_model = quantized_model.train()
|
|
115
|
+
# Freeze BatchNorm, or apply other QAT utilities as needed
|
|
116
|
+
# train(qat_model)
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
### Compile
|
|
120
|
+
|
|
121
|
+
Compilation can be done with TensorRT's trtexec tool, which can take the ONNX
|
|
122
|
+
model and compile it for inference. The exported layer info and profile can
|
|
123
|
+
be used for debugging, optimization and visualization.
|
|
124
|
+
|
|
125
|
+
Note: that the ONNX model might need to be simplified with onnx-simplifier to
|
|
126
|
+
make trtexec compile it. Dynamo exported models may have compilation issues,
|
|
127
|
+
so it's recommended to export with dynamo=False.
|
|
128
|
+
|
|
129
|
+
```bash
|
|
130
|
+
onnxsim model.onnx model.onnx
|
|
131
|
+
/usr/src/tensorrt/bin/trtexec --onnx=model.onnx --fp16 --int8 --useCudaGraph
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
Optionally you can get the layer profile with the following flags:
|
|
135
|
+
```
|
|
136
|
+
--exportLayerInfo=layer_info.json
|
|
137
|
+
--exportProfile=profile.json
|
|
138
|
+
--profilingVerbosity=detailed
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
## Mixed Precision
|
|
142
|
+
|
|
143
|
+
To keep a specific layer in higher precision while quantizing the rest to INT8,
|
|
144
|
+
pass its `nn.Conv2d` instance to `ModulesToSkip` after `transform`. Note that
|
|
145
|
+
`torch.fx.GraphModule` deep-copies submodules during tracing, so you must take
|
|
146
|
+
the reference **from the fused graph**, not from the original model:
|
|
147
|
+
|
|
148
|
+
```python
|
|
149
|
+
from embedl_deploy.quantize import quantize, QuantConfig, ModulesToSkip
|
|
150
|
+
|
|
151
|
+
res = transform(model, patterns=TENSORRT_PATTERNS)
|
|
152
|
+
|
|
153
|
+
# Grab the conv instance from the fused graph (not from the original model)
|
|
154
|
+
first_conv = res.model.FusedConvBNActMaxPool_0.conv
|
|
155
|
+
|
|
156
|
+
config = QuantConfig(
|
|
157
|
+
skip=ModulesToSkip(
|
|
158
|
+
stub={first_conv}, # disables input activation quantization
|
|
159
|
+
weight={first_conv}, # disables weight fake-quantization
|
|
160
|
+
)
|
|
161
|
+
)
|
|
162
|
+
quantized_model = quantize(
|
|
163
|
+
res.model, (example_input,), config=config, forward_loop=calibration_loop
|
|
164
|
+
)
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
## Design Principles
|
|
168
|
+
|
|
169
|
+
1. **Patterns are the only abstraction.**
|
|
170
|
+
Every graph transformation — fusion, conversion, quantization — is a
|
|
171
|
+
`Pattern` subclass. Adding a new backend (TIDL, QNN, …) means defining a
|
|
172
|
+
new set of `Pattern` subclasses and fused modules with quantization
|
|
173
|
+
information. The core plan/apply machinery stays the same.
|
|
174
|
+
|
|
175
|
+
2. **Plans are editable.**
|
|
176
|
+
`get_transformation_plan()` returns a plan the user can inspect and edit
|
|
177
|
+
before applying. Toggle `match.apply = False` to skip specific matches.
|
|
178
|
+
`transform()` is a convenience for the common case where you want
|
|
179
|
+
everything applied.
|
|
180
|
+
|
|
181
|
+
3. **Graph-based models (torch.export.export and symbolic traced).**
|
|
182
|
+
All graph analysis and surgery uses traced graphs. Models are traced once
|
|
183
|
+
and manipulated as `fx.GraphModule` objects with suport for tracing via both
|
|
184
|
+
`torch.fx` (symbolic) as well as `torch.export.export` (Aten). Support for
|
|
185
|
+
Aten graphs is automatically enabled using Aten recomposition
|
|
186
|
+
patterns that compose Aten operations into equivalent `torch.nn` modules
|
|
187
|
+
automatically before conversions and fusions.
|
|
188
|
+
|
|
189
|
+
## Support
|
|
190
|
+
|
|
191
|
+
- [GitHub Issues](https://github.com/embedl/embedl-deploy/issues)
|
|
192
|
+
- Maintainers: The Embedl Team
|
|
193
|
+
- Contact: Shahnawaz Ahmed | shahnawaz@embedl.com | @quantshah
|
|
194
|
+
|
|
195
|
+
## License
|
|
196
|
+
|
|
197
|
+
Free for non-commercial use within the Embedl Community License (v.1.0).
|
|
198
|
+
|
|
199
|
+
Please [Contact us](https://embedl.com/contact) for commercial licensing.
|
|
200
|
+
|
|
201
|
+
Copyright (C) 2026 Embedl AB
|