embedl-deploy-tensorrt 0.0.0.dev0__tar.gz → 0.4.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (40) hide show
  1. embedl_deploy_tensorrt-0.4.0/LICENSE +361 -0
  2. embedl_deploy_tensorrt-0.4.0/MANIFEST.in +13 -0
  3. embedl_deploy_tensorrt-0.4.0/NOTICE +1 -0
  4. embedl_deploy_tensorrt-0.4.0/PKG-INFO +170 -0
  5. embedl_deploy_tensorrt-0.4.0/README.md +150 -0
  6. embedl_deploy_tensorrt-0.4.0/pyproject.toml +45 -0
  7. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/__init__.py +23 -0
  8. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/__init__.py +3 -0
  9. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/__init__.py +3 -0
  10. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/backend.py +18 -0
  11. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/modules/__init__.py +3 -0
  12. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/modules/attention.py +309 -0
  13. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/modules/conv.py +222 -0
  14. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/modules/linear.py +159 -0
  15. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/modules/pointwise.py +38 -0
  16. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/modules/pool.py +25 -0
  17. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/modules/swin_attention.py +460 -0
  18. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/patterns/__init__.py +3 -0
  19. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/patterns/conversions/__init__.py +15 -0
  20. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/patterns/conversions/attention.py +954 -0
  21. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/patterns/conversions/general.py +330 -0
  22. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/patterns/fusions.py +217 -0
  23. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/patterns/quantizations.py +343 -0
  24. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/patterns/recompositions.py +1475 -0
  25. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/patterns/smoothings.py +143 -0
  26. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/patterns/utils.py +89 -0
  27. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/_internal/tensorrt/plan.py +127 -0
  28. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/tensorrt/__init__.py +59 -0
  29. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/tensorrt/modules/__init__.py +40 -0
  30. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/tensorrt/patterns/__init__.py +52 -0
  31. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/version/__init__.py +3 -0
  32. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy/version/public.py +5 -0
  33. embedl_deploy_tensorrt-0.4.0/src/embedl_deploy_tensorrt.egg-info/SOURCES.txt +31 -0
  34. embedl_deploy_tensorrt-0.0.0.dev0/PKG-INFO +0 -6
  35. embedl_deploy_tensorrt-0.0.0.dev0/embedl_deploy_tensorrt.egg-info/PKG-INFO +0 -6
  36. embedl_deploy_tensorrt-0.0.0.dev0/embedl_deploy_tensorrt.egg-info/SOURCES.txt +0 -5
  37. embedl_deploy_tensorrt-0.0.0.dev0/embedl_deploy_tensorrt.egg-info/dependency_links.txt +0 -1
  38. embedl_deploy_tensorrt-0.0.0.dev0/embedl_deploy_tensorrt.egg-info/top_level.txt +0 -1
  39. embedl_deploy_tensorrt-0.0.0.dev0/pyproject.toml +0 -10
  40. {embedl_deploy_tensorrt-0.0.0.dev0 → embedl_deploy_tensorrt-0.4.0}/setup.cfg +0 -0
@@ -0,0 +1,361 @@
1
+ Copyright (C) 2026 Embedl AB
2
+
3
+ EMBEDL COMMUNITY LICENSE AGREEMENT v.1.0
4
+
5
+ Embedl is pleased to make available Embedl Deploy and source code files under
6
+ this Community License Agreement (the "License" or the "Terms"). Our goal is to
7
+ foster a vibrant community of developers, researchers, and organizations who
8
+ can freely use, modify, and build upon the tools and optimized AI models. To
9
+ that end, this license permits unlimited use, distribution and
10
+ commercialization for most individuals and organizations, subject only to the
11
+ limited restrictions described below.
12
+
13
+ To sustain our ability to continue developing and releasing tools for
14
+ optimizing AI models, we have established three limitations on use. First,
15
+ competitors to Embedl may not use the licensed materials, as permitting such
16
+ use would undermine our ability to invest in future development. Second, larger
17
+ organizations—those with more than 250 employees or annual revenue exceeding
18
+ €10 million—can use the licensed materials for internal evaluation purposes,
19
+ but must obtain a separate Commercial Use license before using the licensed
20
+ materials for commercial purposes; this ensures that entities with greater
21
+ resources contribute appropriately to the ecosystem. Third, no licensee may
22
+ host the licensed materials as a service, which protects both Embedl's
23
+ interests and the integrity of the community by preventing commoditization of
24
+ these models (although specific use cases are possible with prior approval
25
+ from us).
26
+
27
+ We recognize that individual circumstances vary, and we welcome inquiries from
28
+ those who wish to discuss alternative licensing arrangements that fall outside
29
+ these terms. Please contact Embedl at sales@embedl.com to discuss your
30
+ specific use case. We would be happy to work with you.
31
+
32
+ 1.0 DEFINED TERMS
33
+
34
+ "Commercial Use": (a) any provision of the functionality of Embedl Deploy, in
35
+ whole or in part, in any product, offering, consultancy, or as a service; (b)
36
+ any Distribution; (c) any use of Embedl Deploy in whole or in part for
37
+ direct or indirect strategic or commercial gain or advantage; or (d) any use of
38
+ Embedl Deploy or any of its outputs or results, in whole or in part, for
39
+ purposes of training or improving AI models.
40
+
41
+ "Competitor": any person or entity in the business of selling access to AI
42
+ development and optimization tools.
43
+
44
+ "Contribution": any contribution to Embedl Deploy that is intentionally
45
+ submitted for inclusion in Embedl Deploy by the contribution owner (copyright
46
+ and otherwise), or by a person or entity authorized to submit the contribution
47
+ on behalf of such owner. For the purposes of this definition, "submitted" means
48
+ any form of electronic, verbal, or written communication sent to us, including
49
+ but not limited to communication on electronic mailing lists, source code
50
+ control systems, and issue tracking systems, but excluding communication that
51
+ is conspicuously marked or otherwise designated in writing by the contribution
52
+ owner as "Not a Contribution."
53
+
54
+ "Contributor": any person or entity submitting a Contribution that has been
55
+ accepted by us and incorporated within Embedl Deploy.
56
+
57
+ "Distribution" or "Distribute": means any transmission, distribution,
58
+ deployment, publication, or other sharing of Embedl Deploy in whole or in part,
59
+ whether standalone or as incorporated or bundled with other works, services,
60
+ or products, to any third party, including by providing or making Embedl Deploy
61
+ or its functionality available as a hosted service via API, web access, or any
62
+ other electronic or remote means.
63
+
64
+ "Embedl Deploy": the software, algorithms, code, generated models and weights,
65
+ licensed by us under these Terms, as any of the foregoing may be updated or
66
+ modified by us on one or more occasions.
67
+
68
+ "We," "us," "our," or "Embedl": Embedl AB, organized under the laws of Sweden.
69
+
70
+ "You" or "your": the person, and any entity such person works for or on behalf
71
+ of, exercising the rights of this license, excluding any Competitor.
72
+
73
+ 2.0 LICENSE RIGHTS
74
+
75
+ 2.1 License Grant. Subject to your compliance with section 3.0, including the
76
+ Commercial Use and Competitor limitations below, and the Third Party Terms, we
77
+ hereby grant to you a worldwide, royalty-free, sublicensable, non-transferable,
78
+ and non-exclusive copyright license to use, reproduce, distribute, display,
79
+ perform, modify, and create derivative works of Embedl Deploy in whole or in
80
+ part.
81
+
82
+ 2.2 Contributor License Grant. Subject to your compliance with these Terms,
83
+ including the Commercial Use and Competitor limitations below, and the Third
84
+ Party Terms, each Contributor hereby grants to you a worldwide, non-exclusive,
85
+ royalty-free, sublicensable, and non-transferable
86
+ (a) copyright license to use, reproduce, distribute, display, perform, modify,
87
+ and create derivative works of Embedl Deploy in whole or in part; and
88
+ (b) patent license to make, have made, use, offer to sell, sell, import and
89
+ otherwise transfer Embedl Deploy in whole or in part, but only to the extent
90
+ of patent claims licensable by such Contributor that are necessarily infringed
91
+ by the exercise of such rights, including for standalone Contributions and
92
+ Contributions in combination with Embedl Deploy.
93
+
94
+ 2.3 Reservation of Rights. All rights not expressly granted in this section
95
+ are reserved to us. You will have no right or license to Embedl Deploy other
96
+ than the rights set forth in this section 2.0, and no other right or license
97
+ will be implied, by estoppel, conduct, or otherwise. If you are an entity, you
98
+ may exercise the rights granted in this section solely via your own employees
99
+ and contractors. We and our licensors retain all right, title and interest in
100
+ Embedl Deploy and associated intellectual property rights, and all copies of
101
+ Embedl Deploy in whole or in part, and we grant no rights to our patents.
102
+
103
+ 3.0 LICENSE CONDITIONS
104
+
105
+ 3.1 No Use by Competitors. If you are, or your employer or client for whom
106
+ you work is, a Competitor, you are not licensed to download, license, install
107
+ or use Embedl Deploy. You represent and warrant that you are not a Competitor
108
+ and are not working for a Competitor. If you are in breach of this warranty,
109
+ now or in the future, you are not licensed to use Embedl Deploy; doing so
110
+ anyway constitutes breach of these Terms and infringement of our intellectual
111
+ property rights.
112
+
113
+ 3.2 Commercial Use Limitation.
114
+
115
+ a) The licensed rights set forth in section 2.0 exclude Commercial Use if you
116
+ are an entity, or an affiliate or member of a company group, that has more than
117
+ 250 employees or annual revenue of €10,000,000 or more ("Large Entity"). If you
118
+ are or become a Large Entity or part of a group of one, Commercial Use is not
119
+ or is no longer licensed to you under these Terms; you must contact us for a
120
+ Commercial Use license. You represent and warrant that you are not a Large
121
+ Entity and are not working for a Large Entity. If you are in breach of this
122
+ warranty, now or in the future, you are not licensed to use Embedl Deploy for
123
+ Commercial Use; doing so anyway constitutes breach of these Terms and
124
+ infringement of our intellectual property rights.
125
+
126
+ b) In addition to any other remedies, if you violate section 3.2(a), you hereby
127
+ grant to Embedl on behalf of yourself and each of your affiliates, employees
128
+ and contractors, a non-exclusive, worldwide, perpetual, irrevocable,
129
+ non-terminable, sublicensable (through multiple tiers), fully paid-up,
130
+ royalty-free, assignable and transferable intellectual property rights license
131
+ as to Distributions, to make, use, reproduce, modify, adapt, create derivative
132
+ works of, improve, extend, enhance, translate, distribute (directly and
133
+ indirectly, in any medium, under terms of choice), combine, compile, transmit,
134
+ display and perform publicly, license, rent, lease, and manufacture, sell,
135
+ offer to sell, and import, your Distributions, for any commercial or
136
+ non-commercial purpose.
137
+
138
+ 3.3 No Use as a Hosted Service. You may not, without our explicit permission,
139
+ use the Embedl
140
+ Deploy, any derivative works thereof, or any models created or trained via the
141
+ use or incorporation of Embedl Deploy, to provide a hosted service,
142
+ software-as-a-service offering, API, or any other mechanism that allows third
143
+ parties to access or use the functionality of such materials, or to submit
144
+ inputs to, receive outputs from, Embedl Deploy, whether or not for a fee.
145
+ Notwithstanding the foregoing, you may use Embedl Deploy on an internal
146
+ network accessible solely to your employees and contractors performing work
147
+ exclusively for you, solely for your internal business
148
+ purposes.
149
+
150
+ In addition, we are open to hosted use in support of a specific function or
151
+ feature in a separate application, depending on the context, on a case-by-case
152
+ basis. Please contact us, and we would be happy to explore that.
153
+
154
+ 3.4 Required Notices and Pass-Throughs.
155
+
156
+ a) You will preserve and not suppress proprietary notices, markings, and
157
+ branding visible in or associated with or displayed via the use of Embedl
158
+ Deploy, and in any Distributions.
159
+
160
+ b) You will give each recipient of your Distribution a copy of these Terms and
161
+ the following notice: "Embedl Deploy is licensed under the Embedl Community
162
+ Source License v.1, and the terms of Embedl's licensors.
163
+ Copyright © Embedl AB."
164
+
165
+ c) You must cause any modified files in your Distributions to carry prominent
166
+ notices explaining the changes that were made.
167
+
168
+ d) You must require each recipient to a Distribution to agree in writing (or
169
+ through another mechanism establishing enforceable assent) to be bound by the
170
+ terms and restrictions of this section, section 3.0 and section 5.2, and to
171
+ impose the same requirement on any subsequent recipients, so that each
172
+ recipient in the chain of distribution and/or use is bound by such terms and
173
+ restrictions.
174
+
175
+ 3.5 Contributions. Each Contributor hereby grants to us a worldwide,
176
+ non-exclusive, royalty-free, sublicensable, and non-transferable
177
+ (a) copyright license to use, reproduce, distribute, display,
178
+ perform, modify, and create derivative works of each Contribution; and
179
+ (b) patent license to make, have made, use, offer to sell, sell, import and
180
+ otherwise transfer Embedl Deploy, but only to the extent of patent claims
181
+ licensable by such Contributor that are necessarily infringed by the exercise
182
+ of such rights, for standalone Contributions or in Contributions in combination
183
+ with Embedl Deploy. By submitting a Contribution, you represent and warrant
184
+ that you have sufficient rights to grant the licenses under these Terms, and
185
+ that doing so does not violate any obligation to a third party.
186
+
187
+ 3.6 Our Marks. You agree that any use of our marks, branding and logos,
188
+ including Embedl Deploy ("Marks"), whether permitted or otherwise, will inure
189
+ to our sole benefit. You will not directly or indirectly: (a) file or prepare
190
+ any application for registration of any Marks; (b) assert any right, title,
191
+ license to, or interest in the Marks; or (c) adopt, use, file for registration,
192
+ or register, in whole or in part, any trademark, service mark, trade name,
193
+ logo, or domain name which may be confusingly similar to or an infringement of
194
+ the Marks or any of our domain names.
195
+
196
+ 3.7 Feedback. You agree that we may freely use and exploit in perpetuity any
197
+ feedback, requirements, recommendations, ideas, bug fixes, reviews, ratings,
198
+ comments, suggestions, or improvements, that you, or any employee or agent
199
+ thereof, may at any time disclose or submit to us relating to Embedl Deploy
200
+ for our business purposes, including for product, services or solution
201
+ sales, licensing, support and development, without any obligation or payment
202
+ to you.
203
+
204
+ 3.8 Unauthorized Use. You will not use Embedl Deploy or participate in any
205
+ activities via Embedl Deploy in a manner that is likely to be prohibited by
206
+ law or these Terms or violative of third party rights in any applicable
207
+ jurisdiction, including intellectual property and data privacy rights. Your
208
+ use of Embedl Deploy must be in full compliance with applicable law and the
209
+ Third Party Terms. You acknowledge that use of Embedl Deploy may be subject
210
+ to the EU AI Act and other AI-specific laws and regulations. You are solely
211
+ responsible for determining whether your use or Distribution constitutes a
212
+ high-risk AI system or is otherwise subject to regulatory obligations,
213
+ and for compliance with all applicable requirements.
214
+
215
+ 3.9 Notifications to Us. If you believe that you are entitled or obligated to
216
+ act contrary to these Terms under any mandatory or applicable law, you agree to
217
+ provide us with detailed and substantiated explanation of your reasons in
218
+ writing at least thirty days before you act, to allow us to assess whether we
219
+ may, at our sole discretion, provide an alternative remedy for the situation,
220
+ though we are under no obligation to do so. To be effective, notices to us must
221
+ be sent to legal@embedl.com.
222
+
223
+ 4.0 DISCLAIMER OF WARRANTIES; LIMITATION OF LIABILITY
224
+
225
+ 4.1 Disclaimer. TO THE MAXIMUM EXTENT POSSIBLE UNDER APPLICABLE LAW, we
226
+ provide Embedl Deploy on an as-is, as-available basis with all faults, and WE
227
+ DISCLAIM ALL WARRANTIES OF ANY KIND WITH RESPECT TO EMBEDL DEPLOY, WHETHER
228
+ EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF
229
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT.
230
+ Specifically, we make no warranty that
231
+
232
+ (a) Embedl Deploy will meet your requirements, goals or needs, or
233
+ (b) any errors or deficiencies in Embedl Deploy will be corrected.
234
+
235
+ 4.2 Exclusion. Notwithstanding any other provision of these Terms, we will
236
+ not be liable for any losses or damages, whether direct, indirect, punitive,
237
+ special, incidental or consequential damages, or liable for interruption of
238
+ business, diminution of value, cost of replacement, downtime, or loss of
239
+ profits, revenue, use, data, or other economic advantage, in connection with,
240
+ related to or arising out of Embedl Deploy or these Terms, regardless of the
241
+ theory of liability, whether for breach of these Terms, including breach of
242
+ warranty, or in tort or otherwise, even if we have been previously advised of
243
+ the possibility of such damages and even if such damages are reasonably
244
+ foreseeable. Liability for damages will be so limited and excluded, regardless
245
+ of the validity or efficacy of any remedy provided herein and even if any
246
+ remedy fails of its essential purpose. Nothing in these Terms excludes or
247
+ limits liability for death or personal injury caused by negligence, fraud,
248
+ intentional misconduct, or gross negligence, or any other liability that cannot
249
+ be excluded under applicable law.
250
+
251
+ 4.3 Indemnification. You hereby agree to defend, indemnify, release and hold
252
+ harmless Embedl, and its officers, directors, employees, consultants, agents,
253
+ and representatives, in full from and against any and all claims (including
254
+ third party claims), losses, liability, damages, and/or costs of every kind and
255
+ nature, arising out of or in any way connected with: (a) your access to or use
256
+ of, Embedl Deploy or conduct engendered thereby, (b) your breach of these Terms
257
+ or the Third Party Terms, (c) your infringement or misappropriation of any
258
+ intellectual property right or other right of any person or entity, or
259
+ (d) your violations of applicable law.
260
+
261
+ 5.0 TERMINATION
262
+
263
+ 5.1 Upon Breach. The licenses granted by these Terms will terminate
264
+ automatically if you are in breach of these Terms, including if you exceed the
265
+ scope of the licenses granted in section 2.0. Breach of these Terms may result
266
+ in pursuit of all available remedies for intellectual property rights
267
+ (including intellectual property rights infringement), the availability of
268
+ which you hereby acknowledge.
269
+
270
+ 5.2 Upon Litigation. The license rights set forth in section 2.0 will
271
+ terminate automatically if you institute litigation (including a cross-claim
272
+ or counterclaim in a lawsuit) against any entity alleging that the use,
273
+ distribution, modification, sale, offer to sell, transfer or import of Embedl
274
+ Deploy, any Contribution (alone or in combination), or the outputs or results
275
+ of Embedl Deploy, constitutes direct, indirect, vicarious, or contributory
276
+ infringement of intellectual property rights.
277
+
278
+ 5.3 By Us. We may on written notice terminate rights granted under these
279
+ Terms in the event the Embedl Deploy become subject to third party claims of
280
+ infringement or violation of intellectual property or other rights or
281
+ applicable law.
282
+
283
+ 5.4 Effect. Upon termination of rights for any reason, all licensed rights
284
+ granted under these Terms will terminate, and you will immediately cease all
285
+ use, and destroy all and not retain any copies, of Embedl Deploy. Our rights
286
+ under this section 5.0 are in addition to any other rights and remedies
287
+ permitted by law or under these Terms.
288
+
289
+ 6.0 GENERAL PROVISIONS
290
+
291
+ 6.1 Governing Law. These Terms and any disputes between us and related to or
292
+ concerning these Terms or Embedl Deploy (including tort as well as contract
293
+ claims, and whether pre-contractual or extra-contractual), notwithstanding the
294
+ choice of laws rules of any jurisdiction to the contrary, will be governed by
295
+ the procedural and substantive laws of Sweden. The United Nations Convention on
296
+ Contracts for the International Sale of Goods will not apply and is hereby
297
+ excluded.
298
+
299
+ 6.2 Arbitration. Subject to section 6.3, any disputes between or claims
300
+ brought by either party arising out of or related to Embedl Deploy or these
301
+ Terms shall be referred to and finally settled exclusively by binding
302
+ arbitration before the Arbitration Institute of the Stockholm Chamber of
303
+ Commerce (the "SCC"), and the venue for arbitration will be Stockholm, Sweden.
304
+ The Rules for Expedited Arbitrations will apply, unless the SCC in its
305
+ discretion determines, taking into account the complexity of the case, the
306
+ amount in dispute and other circumstances, that the Arbitration Rules will
307
+ apply. In the latter case, the SCC will also decide whether the Arbitral
308
+ Tribunal will be composed of one or three arbitrators. The arbitration will be
309
+ conducted by telephone, on-line and/or based solely upon written submissions
310
+ where no in-person appearance is required. If in-person appearance is required,
311
+ such hearings will be held in Stockholm, Sweden. The arbitrator will apply
312
+ the law specified in section 6.1. All awards may if necessary be enforced by
313
+ any court having jurisdiction. The existence of any dispute, the existence or
314
+ details of the arbitration proceeding, and all related documents, materials,
315
+ evidence, judgments and awards therein, shall be kept confidential. Except as
316
+ required by law, no party will make any public announcements with respect to
317
+ the other party or the proceeding or the award, except as required to enforce
318
+ same. All disputes will be arbitrated only on an individual basis and not in a
319
+ class, consolidated or representative action. The arbitrator does not have the
320
+ power to vary these provisions. Subject to section 6.3, all claims between the
321
+ parties must be resolved via arbitration in accordance with this section.
322
+ Should either party file an action contrary to this section, the other party
323
+ may recover lawyers' fees and costs associated with enforcing this section,
324
+ provided that the party seeking the award has notified the other party in
325
+ writing of the improperly filed claim, and the other party has failed to
326
+ withdraw the claim in a timely fashion.
327
+
328
+ 6.3 Injunctive Relief; Prevailing Party. You acknowledge and agree that
329
+ breach of these Terms, or any unauthorized use or distribution of Embedl
330
+ Deploy, may cause irreparable harm to us, the extent of which would be
331
+ difficult to ascertain, and that we will be entitled to seek immediate
332
+ injunctive relief and specific performance (in addition to any other available
333
+ remedies), and to enforce and/or seek redress for infringement of intellectual
334
+ property rights, in any court of competent jurisdiction under the applicable
335
+ laws thereto (and such proceeding, and our right to prosecute such a claim,
336
+ is not subject to arbitration). You hereby consent to the jurisdiction of
337
+ such courts for any such actions. A party prevailing in any litigation or
338
+ arbitration related to these Terms or Embedl Deploy will be entitled, in
339
+ addition to such other relief as may be granted, to an award of reasonable
340
+ attorneys' fees. This section is in addition and without prejudice to the
341
+ availability of attorneys' fees as a remedy under applicable law, including
342
+ laws pertaining to intellectual property rights.
343
+
344
+ 6.4 Important Final Terms. The Terms constitute the entire agreement between
345
+ you and us and govern your use of Embedl Deploy, superseding any prior
346
+ agreements, understandings, communications or proposals; except that these
347
+ Terms do not override, supersede, or modify the terms of any written license
348
+ agreement signed by you and us. If any provision of the Terms is found by a
349
+ court of competent jurisdiction to be invalid, the parties nevertheless agree
350
+ that the court should endeavor to give effect to the parties' intentions as
351
+ reflected in the provision, and the other provisions of the Terms will remain
352
+ in full force and effect. No waiver of any provision of these Terms will be
353
+ deemed a further waiver or continuing waiver or such provision or any other
354
+ provision, and our failure to assert any right or provision under these Terms
355
+ will not constitute a waiver of such right or provision. Nothing herein will
356
+ be deemed to create an agency, partnership, joint venture, or
357
+ franchisor-franchisee relationship of any kind between us and any user or other
358
+ person or entity, nor do these terms extend rights to any third party. These
359
+ Terms are in the English language only, which language will be controlling in
360
+ all respects, and all versions of these Terms in any other language will be
361
+ for accommodation only and will not be binding on you or us.
@@ -0,0 +1,13 @@
1
+ prune *
2
+ include src/embedl_deploy/__init__.py
3
+ include src/embedl_deploy/_internal/__init__.py
4
+ include src/embedl_deploy/version/__init__.py
5
+ include src/embedl_deploy/version/public.py
6
+ include LICENSE
7
+ include NOTICE
8
+ include README.md
9
+ graft src/embedl_deploy/tensorrt
10
+ graft src/embedl_deploy/_internal/tensorrt
11
+ global-exclude CLAUDE.md
12
+ global-exclude *.pyc
13
+ global-exclude __pycache__
@@ -0,0 +1 @@
1
+ Copyright (C) 2026 Embedl AB
@@ -0,0 +1,170 @@
1
+ Metadata-Version: 2.4
2
+ Name: embedl-deploy-tensorrt
3
+ Version: 0.4.0
4
+ Summary: TensorRT backend for embedl-deploy.
5
+ Author-email: Embedl AB <support@embedl.com>
6
+ Project-URL: Homepage, https://www.embedl.com/
7
+ Classifier: Programming Language :: Python :: 3
8
+ Classifier: Programming Language :: Python :: 3 :: Only
9
+ Classifier: License :: Other/Proprietary License
10
+ Classifier: Operating System :: POSIX :: Linux
11
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
12
+ Requires-Python: >=3.10
13
+ Description-Content-Type: text/markdown
14
+ License-File: LICENSE
15
+ License-File: NOTICE
16
+ Requires-Dist: tensorrt
17
+ Provides-Extra: core
18
+ Requires-Dist: embedl-deploy; extra == "core"
19
+ Dynamic: license-file
20
+
21
+ # embedl-deploy
22
+
23
+ Python package to make AI models deployment-ready for any hardware.
24
+
25
+ ## Why embedl-deploy
26
+
27
+ PyTorch models are flexible, but edge hardware is not. Hardware toolchains may
28
+ fail due to unsupported operators, apply implicit transformations and fusions
29
+ during compilation and quantization leading to deployment issues.
30
+
31
+ `embedl-deploy` eliminates these surprises by enforcing hardware and compiler
32
+ constraints directly in PyTorch, so what you build, train, and debug is what
33
+ actually runs on the device. It converts your models to be compatible for the
34
+ hardware target ensuring correct quantization and compilation.
35
+
36
+ ## Features
37
+
38
+ - **Hardware-accurate PyTorch Intermediate Representation (IR):**
39
+ Build models using a hardware-aware PyTorch intermediate representation that
40
+ mirrors the behavior of the compiled artifact, e.g., fused convolutions.
41
+ Unsupported operators and compatibility issues are surfaced early and
42
+ resolved explicitly before compilation, within PyTorch.
43
+
44
+ - **Quantization:**
45
+ Supports post-training quantization (PTQ) and quantization-aware-training
46
+ (QAT). Fake quantization is applied in PyTorch with explicit quantization
47
+ operator placements. PTQ methods are included and QAT can be applied directly
48
+ to the transformed and quantized models with no additional dependencies
49
+ required.
50
+
51
+ - **Guaranteed deployable artifacts:**
52
+ Produce optimized compilation artifacts ready for deployment on the target
53
+ device with predictable performance and accuracy.
54
+
55
+
56
+ ## Supported Backends
57
+
58
+ | Backend | Status |
59
+ |---------------------|-------------|
60
+ | NVIDIA TensorRT | Supported |
61
+
62
+ Contact us for other backends.
63
+
64
+ ## Installation
65
+
66
+ ```bash
67
+ pip install embedl-deploy
68
+ ```
69
+ Note that you may need to also install `onnx` and `onnx-simplifier` to export
70
+ and get the exported model compiled with TensorRT if using ONNX as an
71
+ intermediate.
72
+
73
+ ---
74
+
75
+ ## Quick Start
76
+
77
+ ```python
78
+ import torch
79
+ from embedl_deploy import transform
80
+ from embedl_deploy.quantize import quantize
81
+ from embedl_deploy.tensorrt import TENSORRT_PATTERNS
82
+ from torchvision.models import resnet18 as Model
83
+
84
+ # 1. Load a standard PyTorch model
85
+ model = Model().eval()
86
+ example_input = torch.randn(1, 3, 224, 224)
87
+
88
+ # 2. Transform — fuse and optimize for TensorRT in one call
89
+ res = transform(model, patterns=TENSORRT_PATTERNS)
90
+ print("Model\n", res.model.print_readable())
91
+ print("Matches", "\n".join([str(match) for match in res.matches]))
92
+
93
+
94
+ # 3. Quantize (PTQ)
95
+ def calibration_loop(model: torch.fx.GraphModule):
96
+ model.eval()
97
+ for _ in range(100):
98
+ model(torch.randn(1, 3, 224, 224))
99
+
100
+
101
+ quantized_model = quantize(
102
+ res.model, (example_input,), forward_loop=calibration_loop
103
+ )
104
+ quantized_model.eval()
105
+
106
+ # 4. Export as usual (dynamo exported models may have compilation issues)
107
+ torch.onnx.export(
108
+ quantized_model, (example_input,), "model.onnx", dynamo=False
109
+ )
110
+
111
+ # 5. Quantization-aware training with a training loop
112
+ qat_model = quantized_model.train()
113
+ # Freeze BatchNorm, or apply other QAT utilities as needed
114
+ # train(qat_model)
115
+
116
+ # Compile
117
+ # -------
118
+ # Compilation can be done with TensorRT's trtexec tool, which can take the ONNX
119
+ # model and compile it for inference. The exported layer info and profile can
120
+ # be used for debugging, optimization and visualization.
121
+ #
122
+ # Note: that the ONNX model might need to be simplified with onnx-simplifier to
123
+ # make trtexec compile it. Dynamo exported models may have compilation issues,
124
+ # so it's recommended to export with dynamo=False.
125
+ #
126
+ # We are working on a Aten-based export path that should be more robust and
127
+ # support more models in the future.
128
+
129
+ # >> onnxsim model.onnx model.onnx
130
+ # >> trtexec \
131
+ # --onnx=model.onnx \
132
+ # --exportLayerInfo=layer_info.json \
133
+ # --exportProfile=profile.json \
134
+ # --profilingVerbosity=detailed
135
+
136
+ # More benchmarking scripts can be found in the examples/ directory
137
+ ```
138
+
139
+ ## Design Principles
140
+
141
+ 1. **Patterns are the only abstraction.**
142
+ Every graph transformation — fusion, conversion, quantization — is a
143
+ `Pattern` subclass. Adding a new backend (TIDL, QNN, …) means defining a
144
+ new set of `Pattern` subclasses and fused modules with quantization
145
+ information. The core plan/apply machinery stays the same.
146
+
147
+ 2. **Plans are editable.**
148
+ `get_transformation_plan()` returns a plan the user can inspect and edit
149
+ before applying. Toggle `match.apply = False` to skip specific matches.
150
+ `transform()` is a convenience for the common case where you want
151
+ everything applied.
152
+
153
+ 3. **FX-graph-based.**
154
+ All graph analysis and surgery uses `torch.fx`. Models are traced once
155
+ and manipulated as `fx.GraphModule` objects. Support for Aten graphs
156
+ produced by `torch.export.export` is planned for the future.
157
+
158
+ ## Support
159
+
160
+ - [GitHub Issues](https://github.com/embedl/embedl-deploy/issues)
161
+ - Maintainers: The Embedl Team
162
+ - Contact: Shahnawaz Ahmed | shahnawaz@embedl.com | @quantshah
163
+
164
+ ## License
165
+
166
+ Free for non-commercial use within the Embedl Community License (v.1.0).
167
+
168
+ Please [Contact us](https://embedl.com/contact) for commercial licensing.
169
+
170
+ Copyright (C) 2026 Embedl AB