zenml-nightly 0.71.0.dev20250105__py3-none-any.whl → 0.71.0.dev20250107__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- zenml/VERSION +1 -1
- zenml/zen_server/auth.py +21 -2
- zenml/zen_server/routers/auth_endpoints.py +2 -1
- zenml/zen_stores/rest_zen_store.py +11 -1
- {zenml_nightly-0.71.0.dev20250105.dist-info → zenml_nightly-0.71.0.dev20250107.dist-info}/METADATA +2 -2
- {zenml_nightly-0.71.0.dev20250105.dist-info → zenml_nightly-0.71.0.dev20250107.dist-info}/RECORD +9 -16
- {zenml_nightly-0.71.0.dev20250105.dist-info → zenml_nightly-0.71.0.dev20250107.dist-info}/WHEEL +1 -1
- CLA.md +0 -110
- CODE-OF-CONDUCT.md +0 -132
- CONTRIBUTING.md +0 -260
- README.md +0 -342
- RELEASE_NOTES.md +0 -5722
- ROADMAP.md +0 -5
- SECURITY.md +0 -15
- {zenml_nightly-0.71.0.dev20250105.dist-info → zenml_nightly-0.71.0.dev20250107.dist-info}/LICENSE +0 -0
- {zenml_nightly-0.71.0.dev20250105.dist-info → zenml_nightly-0.71.0.dev20250107.dist-info}/entry_points.txt +0 -0
CONTRIBUTING.md
DELETED
@@ -1,260 +0,0 @@
|
|
1
|
-
# 🧑💻 Contributing to ZenML
|
2
|
-
|
3
|
-
A big welcome and thank you for considering contributing to ZenML! It’s people
|
4
|
-
like you that make it a reality for users
|
5
|
-
in our community.
|
6
|
-
|
7
|
-
Reading and following these guidelines will help us make the contribution
|
8
|
-
process easy and effective for everyone
|
9
|
-
involved. It also communicates that you agree to respect the developers' time
|
10
|
-
management and develop these open-source projects. In return, we will reciprocate that respect by reading your
|
11
|
-
issue, assessing changes, and helping
|
12
|
-
you finalize your pull requests.
|
13
|
-
|
14
|
-
## ⚡️ Quicklinks
|
15
|
-
|
16
|
-
- [🧑💻 Contributing to ZenML](#-contributing-to-zenml)
|
17
|
-
- [⚡️ Quicklinks](#-quicklinks)
|
18
|
-
- [🧑⚖️ Code of Conduct](#-code-of-conduct)
|
19
|
-
- [🛫 Getting Started](#-getting-started)
|
20
|
-
- [⁉️ Issues](#-issues)
|
21
|
-
- [🏷 Pull Requests: When to make one](#-pull-requests-when-to-make-one)
|
22
|
-
- [💯 Pull Requests: Workflow to Contribute](#-pull-requests-workflow-to-contribute)
|
23
|
-
- [🧱 Pull Requests: Rebase on develop](#-pull-requests-rebase-your-branch-on-develop)
|
24
|
-
- [🧐 Linting, formatting, and tests](#-linting-formatting-and-tests)
|
25
|
-
- [🚨 Reporting a Vulnerability](#-reporting-a-vulnerability)
|
26
|
-
- [Coding Conventions](#coding-conventions)
|
27
|
-
- [👷 Creating a new Integration](#-creating-a-new-integration)
|
28
|
-
- [🆘 Getting Help](#-getting-help)
|
29
|
-
|
30
|
-
## 🧑⚖️ Code of Conduct
|
31
|
-
|
32
|
-
We take our open-source community seriously and hold ourselves and other
|
33
|
-
contributors to high standards of communication.
|
34
|
-
By participating and contributing to this project, you agree to uphold
|
35
|
-
our [Code of Conduct](https://github.com/zenml-io/zenml/blob/master/CODE-OF-CONDUCT.md)
|
36
|
-
.
|
37
|
-
|
38
|
-
## 🛫 Getting Started
|
39
|
-
|
40
|
-
Contributions are made to this repo via Issues and Pull Requests (PRs). A few
|
41
|
-
general guidelines that cover both:
|
42
|
-
|
43
|
-
- To report security vulnerabilities, please get in touch
|
44
|
-
at [support@zenml.io](mailto:support@zenml.io), monitored by
|
45
|
-
our security team.
|
46
|
-
- Search for existing Issues and PRs before creating your own.
|
47
|
-
- We work hard to make sure issues are handled on time, but it could take a
|
48
|
-
while to investigate the root cause depending on the impact.
|
49
|
-
|
50
|
-
A friendly ping in the comment thread to the submitter or a contributor can help
|
51
|
-
draw attention if your issue is blocking.
|
52
|
-
|
53
|
-
### Good First Issues for New Contributors
|
54
|
-
|
55
|
-
The best way to start is to check the
|
56
|
-
[`good-first-issue`](https://github.com/issues?q=is%3Aopen+is%3Aissue+archived%3Afalse+user%3Azenml-io+label%3A%22good+first+issue%22)
|
57
|
-
label on the issue board. The core team creates these issues as necessary
|
58
|
-
smaller tasks that you can work on to get deeper into ZenML internals. These
|
59
|
-
should generally require relatively simple changes, probably affecting just one
|
60
|
-
or two files which we think are ideal for people new to ZenML.
|
61
|
-
|
62
|
-
The next step after that would be to look at the
|
63
|
-
[`good-second-issue`](https://github.com/issues?q=is%3Aopen+is%3Aissue+archived%3Afalse+user%3Azenml-io+label%3A%22good+second+issue%22)
|
64
|
-
label on the issue board. These are a bit more complex, might involve more
|
65
|
-
files, but should still be well-defined and achievable to people relatively new
|
66
|
-
to ZenML.
|
67
|
-
|
68
|
-
### ⁉️ Issues
|
69
|
-
|
70
|
-
Issues should be used to report problems with the library, request a new
|
71
|
-
feature, or to discuss potential changes before
|
72
|
-
a PR is created. When you create a new Issue, a template will be loaded that
|
73
|
-
will guide you through collecting and
|
74
|
-
providing the information we need to investigate.
|
75
|
-
|
76
|
-
If you find an Issue that addresses your problem, please add your own
|
77
|
-
reproduction information to the
|
78
|
-
existing issue rather than creating a new one. Adding
|
79
|
-
a [reaction](https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/)
|
80
|
-
can also help by
|
81
|
-
indicating to our maintainers that a particular issue is affecting more than
|
82
|
-
just the reporter.
|
83
|
-
|
84
|
-
### 🏷 Pull Requests: When to make one
|
85
|
-
|
86
|
-
Pull Requests (PRs) to ZenML are always welcome and can be a quick way to get your fix or
|
87
|
-
improvement slated for the next release. In
|
88
|
-
general, PRs should:
|
89
|
-
|
90
|
-
- Only fix/add the functionality in question **OR** address widespread
|
91
|
-
whitespace/style issues, not both.
|
92
|
-
- Add unit or integration tests for fixed or changed functionality (if a test
|
93
|
-
suite already exists).
|
94
|
-
- Address a single concern in the least number of changed lines as possible.
|
95
|
-
- Include documentation in the repo or in your Pull Request.
|
96
|
-
- Be accompanied by a filled-out Pull Request template (loaded automatically when
|
97
|
-
a PR is created).
|
98
|
-
|
99
|
-
For changes that address core functionality or would require breaking changes (e.g. a major release), it's best to open
|
100
|
-
an Issue to discuss your proposal first. This is not required but can save time
|
101
|
-
creating and reviewing changes.
|
102
|
-
|
103
|
-
### 💯 Pull Requests: Workflow to Contribute
|
104
|
-
|
105
|
-
<p class="callout warning">Please note that development in ZenML happens off of the <b>develop</b> branch, <b>not main</b>,
|
106
|
-
which is the default branch on GitHub. Therefore, please pay particular attention to step 5 and step 9 below. </p>
|
107
|
-
|
108
|
-
In general, we follow
|
109
|
-
the ["fork-and-pull" Git workflow](https://github.com/susam/gitpr)
|
110
|
-
|
111
|
-
1. Review and sign
|
112
|
-
the [Contributor License Agreement](https://cla-assistant.io/zenml-io/zenml) (
|
113
|
-
CLA).
|
114
|
-
2. Fork the repository to your own Github account.
|
115
|
-
3. Clone the project to your machine.
|
116
|
-
4. Checkout the **develop** branch <- `git checkout develop`.
|
117
|
-
5. Create a branch (again, off of the develop branch) locally with a succinct but descriptive name.
|
118
|
-
6. Commit changes to the branch
|
119
|
-
7. Follow the `Linting, formatting, and tests` guide to make sure your code adheres to the ZenML coding style (see below).
|
120
|
-
8. Push changes to your fork.
|
121
|
-
9. Open a PR in our repository (to the `develop` branch, **NOT** `main`) and
|
122
|
-
follow the PR template so that we can efficiently review the changes.
|
123
|
-
|
124
|
-
### 🧱 Pull Requests: Rebase Your Branch on Develop
|
125
|
-
|
126
|
-
1. When making pull requests to ZenML, you should always make your changes on a branch that is based on `develop`. You can create a new branch based on `develop` by running the following command:
|
127
|
-
```
|
128
|
-
git checkout -b <new-branch-name> develop
|
129
|
-
```
|
130
|
-
2. Fetch the latest changes from the remote `develop` branch:
|
131
|
-
```
|
132
|
-
git fetch origin develop
|
133
|
-
```
|
134
|
-
3. Switch to your branch:
|
135
|
-
```
|
136
|
-
git checkout <your-branch-name>
|
137
|
-
```
|
138
|
-
4. Rebase your branch on `develop`:
|
139
|
-
```
|
140
|
-
git rebase origin/develop
|
141
|
-
```
|
142
|
-
This will apply your branch's changes on top of the latest changes in `develop`, one commit at a time.
|
143
|
-
5. Resolve any conflicts that may arise during the rebase. Git will notify you if there are any conflicts that need to be resolved. Use a text editor to manually resolve the conflicts in the affected files.
|
144
|
-
6. After resolving the conflicts, stage the changes:
|
145
|
-
```
|
146
|
-
git add .
|
147
|
-
```
|
148
|
-
7. Continue the rebase for all of your commits and go to 5) if there are conflicts.
|
149
|
-
```
|
150
|
-
git rebase --continue
|
151
|
-
```
|
152
|
-
8. Push the rebased branch to your remote repository:
|
153
|
-
```
|
154
|
-
git push origin --force <your-branch-name>
|
155
|
-
```
|
156
|
-
9. Open a pull request targeting the `develop` branch. The changes from your rebased branch will now be based on the latest `develop` branch.
|
157
|
-
|
158
|
-
### 🧐 Linting, formatting, and tests
|
159
|
-
|
160
|
-
To install ZenML from your local checked out files including all core dev-dependencies, run:
|
161
|
-
|
162
|
-
```
|
163
|
-
pip install -e ".[server,dev]"
|
164
|
-
```
|
165
|
-
|
166
|
-
Optionally, you might want to run the following commands to ensure you have all
|
167
|
-
integrations for `mypy` checks:
|
168
|
-
|
169
|
-
```
|
170
|
-
zenml integration install -y -i feast
|
171
|
-
pip install click~=8.0.3
|
172
|
-
mypy --install-types
|
173
|
-
```
|
174
|
-
|
175
|
-
Warning: This might take a while for both (~ 15 minutes each, depending on your machine), however if you have
|
176
|
-
time, please run it as it will make the
|
177
|
-
next commands error-free. Note that the `zenml integration install` command
|
178
|
-
might also fail on account of dependency conflicts so you can just install the
|
179
|
-
specific integration you're working on and manually run the mypy command for the
|
180
|
-
files you've been working on.
|
181
|
-
|
182
|
-
You can now run the following scripts to automatically format your
|
183
|
-
code and to check whether the code formatting, linting, docstrings, and
|
184
|
-
spelling is in order:
|
185
|
-
|
186
|
-
```
|
187
|
-
bash scripts/format.sh
|
188
|
-
bash scripts/run-ci-checks.sh
|
189
|
-
```
|
190
|
-
|
191
|
-
If you're on Windows you might have to run the formatting script as `bash
|
192
|
-
scripts/format.sh --no-yamlfix` and run the yamlfix command separately as
|
193
|
-
`yamlfix .github -v`.
|
194
|
-
|
195
|
-
Tests can be run as follows:
|
196
|
-
|
197
|
-
```
|
198
|
-
bash scripts/test-coverage-xml.sh
|
199
|
-
```
|
200
|
-
|
201
|
-
Please note that it is good practice to run the above commands before submitting
|
202
|
-
any Pull Request: The CI GitHub Action
|
203
|
-
will run it anyway, so you might as well catch the errors locally!
|
204
|
-
|
205
|
-
### 🚨 Reporting a Vulnerability
|
206
|
-
|
207
|
-
Please refer to [our security / reporting instructions](./SECURITY.md) for
|
208
|
-
details on reporting vulnerabilities.
|
209
|
-
|
210
|
-
|
211
|
-
## Coding Conventions
|
212
|
-
|
213
|
-
The code within the repository is structured in the following way -
|
214
|
-
the most relevant places for contributors are highlighted with a `<-` arrow:
|
215
|
-
|
216
|
-
```
|
217
|
-
├── .github -- Definition of the GH action workflows
|
218
|
-
├── docker -- Dockerfiles used to build ZenML docker images
|
219
|
-
├── docs <- The ZenML docs, CLI docs and API docs live here
|
220
|
-
│ ├── book <- In case you make user facing changes, update docs here
|
221
|
-
│ └── mkdocs -- Some configurations for the API/CLI docs
|
222
|
-
├── examples <- When adding an integration, add an example here
|
223
|
-
├── scripts -- Scripts used by Github Actions or for local linting/testing
|
224
|
-
├── src/zenml <- The heart of ZenML
|
225
|
-
│ ├── <stack_component> <- Each stack component has its own directory
|
226
|
-
│ ├── cli <- Change and improve the CLI here
|
227
|
-
│ ├── config -- The ZenML config methods live here
|
228
|
-
│ ├── integrations <- Add new integrations here
|
229
|
-
│ ├── io -- File operation implementations
|
230
|
-
│ ├── materializers <- Materializers responsible for reading/writing artifacts
|
231
|
-
│ ├── pipelines <- The base pipeline and its decorator
|
232
|
-
│ ├── services -- Code responsible for managing services
|
233
|
-
│ ├── stack <- Stack, Stack Components and the flavor registry
|
234
|
-
│ ├── steps <- Steps and their decorators are defined here
|
235
|
-
│ ├── utils <- Collection on useful utils
|
236
|
-
│ ├── zen_server -- Code for running the Zen Server
|
237
|
-
│ └── zen_stores -- Code for storing stacks in multiple settings
|
238
|
-
└── test <- Don't forget to write unit tests for your code
|
239
|
-
```
|
240
|
-
|
241
|
-
## 👷 Creating a new Integration
|
242
|
-
|
243
|
-
In case you want to create an entirely new integration that you would like to
|
244
|
-
see supported by ZenML there are a few steps that you should follow:
|
245
|
-
|
246
|
-
1. Create the actual integration. Check out the
|
247
|
-
[Integrations README](src/zenml/integrations/README.md)
|
248
|
-
for detailed step-by-step instructions.
|
249
|
-
2. Create an example of how to use the integration. Check out the
|
250
|
-
[Examples README](examples/README.md)
|
251
|
-
to find out what to do.
|
252
|
-
3. All integrations deserve to be documented. Make sure to pay a visit to the
|
253
|
-
[Component Guide](https://docs.zenml.io/stack-components/component-guide)
|
254
|
-
in the docs and add your implementations.
|
255
|
-
|
256
|
-
## 🆘 Getting Help
|
257
|
-
|
258
|
-
Join us in the [ZenML Slack Community](https://zenml.io/slack-invite/) to
|
259
|
-
interact directly with the core team and community at large. This is a good
|
260
|
-
place to ideate, discuss concepts or ask for help.
|
README.md
DELETED
@@ -1,342 +0,0 @@
|
|
1
|
-
<div align="center">
|
2
|
-
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=0fcbab94-8fbe-4a38-93e8-c2348450a42e" />
|
3
|
-
<h1 align="center">Connecting data science teams seamlessly to cloud infrastructure.
|
4
|
-
</h1>
|
5
|
-
</div>
|
6
|
-
|
7
|
-
<!-- PROJECT SHIELDS -->
|
8
|
-
<!--
|
9
|
-
*** I'm using markdown "reference style" links for readability.
|
10
|
-
*** Reference links are enclosed in brackets [ ] instead of parentheses ( ).
|
11
|
-
*** See the bottom of this document for the declaration of the reference variables
|
12
|
-
*** for contributors-url, forks-url, etc. This is an optional, concise syntax you may use.
|
13
|
-
*** https://www.markdownguide.org/basic-syntax/#reference-style-links
|
14
|
-
-->
|
15
|
-
|
16
|
-
<div align="center">
|
17
|
-
|
18
|
-
<!-- PROJECT LOGO -->
|
19
|
-
<br />
|
20
|
-
<a href="https://zenml.io">
|
21
|
-
<img alt="ZenML Logo" src="docs/book/.gitbook/assets/header.png" alt="ZenML Logo">
|
22
|
-
</a>
|
23
|
-
<br />
|
24
|
-
|
25
|
-
[![PyPi][pypi-shield]][pypi-url]
|
26
|
-
[![PyPi][pypiversion-shield]][pypi-url]
|
27
|
-
[![PyPi][downloads-shield]][downloads-url]
|
28
|
-
[![Contributors][contributors-shield]][contributors-url]
|
29
|
-
[![License][license-shield]][license-url]
|
30
|
-
<!-- [![Build][build-shield]][build-url] -->
|
31
|
-
<!-- [![CodeCov][codecov-shield]][codecov-url] -->
|
32
|
-
|
33
|
-
</div>
|
34
|
-
|
35
|
-
<!-- MARKDOWN LINKS & IMAGES -->
|
36
|
-
<!-- https://www.markdownguide.org/basic-syntax/#reference-style-links -->
|
37
|
-
|
38
|
-
[pypi-shield]: https://img.shields.io/pypi/pyversions/zenml?color=281158
|
39
|
-
|
40
|
-
[pypi-url]: https://pypi.org/project/zenml/
|
41
|
-
|
42
|
-
[pypiversion-shield]: https://img.shields.io/pypi/v/zenml?color=361776
|
43
|
-
|
44
|
-
[downloads-shield]: https://img.shields.io/pypi/dm/zenml?color=431D93
|
45
|
-
|
46
|
-
[downloads-url]: https://pypi.org/project/zenml/
|
47
|
-
|
48
|
-
[codecov-shield]: https://img.shields.io/codecov/c/gh/zenml-io/zenml?color=7A3EF4
|
49
|
-
|
50
|
-
[codecov-url]: https://codecov.io/gh/zenml-io/zenml
|
51
|
-
|
52
|
-
[contributors-shield]: https://img.shields.io/github/contributors/zenml-io/zenml?color=7A3EF4
|
53
|
-
|
54
|
-
[contributors-url]: https://github.com/zenml-io/zenml/graphs/contributors
|
55
|
-
|
56
|
-
[license-shield]: https://img.shields.io/github/license/zenml-io/zenml?color=9565F6
|
57
|
-
|
58
|
-
[license-url]: https://github.com/zenml-io/zenml/blob/main/LICENSE
|
59
|
-
|
60
|
-
[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=for-the-badge&logo=linkedin&colorB=555
|
61
|
-
|
62
|
-
[linkedin-url]: https://www.linkedin.com/company/zenml/
|
63
|
-
|
64
|
-
[twitter-shield]: https://img.shields.io/twitter/follow/zenml_io?style=for-the-badge
|
65
|
-
|
66
|
-
[twitter-url]: https://twitter.com/zenml_io
|
67
|
-
|
68
|
-
[slack-shield]: https://img.shields.io/badge/-Slack-black.svg?style=for-the-badge&logo=linkedin&colorB=555
|
69
|
-
|
70
|
-
[slack-url]: https://zenml.io/slack-invite
|
71
|
-
|
72
|
-
[build-shield]: https://img.shields.io/github/workflow/status/zenml-io/zenml/Build,%20Lint,%20Unit%20&%20Integration%20Test/develop?logo=github&style=for-the-badge
|
73
|
-
|
74
|
-
[build-url]: https://github.com/zenml-io/zenml/actions/workflows/ci.yml
|
75
|
-
|
76
|
-
---
|
77
|
-
|
78
|
-
## ⭐️ Show Your Support
|
79
|
-
|
80
|
-
If you find ZenML helpful or interesting, please consider giving us a star on GitHub. Your support helps promote the project and lets others know that it's worth checking out.
|
81
|
-
|
82
|
-
Thank you for your support! 🌟
|
83
|
-
|
84
|
-
[](https://github.com/zenml-io/zenml/stargazers)
|
85
|
-
|
86
|
-
## 🤸 Quickstart
|
87
|
-
[](https://colab.research.google.com/github/zenml-io/zenml/blob/main/examples/quickstart/quickstart.ipynb)
|
88
|
-
|
89
|
-
[Install ZenML](https://docs.zenml.io/getting-started/installation) via [PyPI](https://pypi.org/project/zenml/). Python 3.9 - 3.12 is required:
|
90
|
-
|
91
|
-
```bash
|
92
|
-
pip install "zenml[server]" notebook
|
93
|
-
```
|
94
|
-
|
95
|
-
Take a tour with the guided quickstart by running:
|
96
|
-
|
97
|
-
```bash
|
98
|
-
zenml go
|
99
|
-
```
|
100
|
-
|
101
|
-
## 🪄 Simple, integrated, End-to-end MLOps
|
102
|
-
|
103
|
-
### Create machine learning pipelines with minimal code changes
|
104
|
-
|
105
|
-
ZenML is a MLOps framework intended for data scientists or ML engineers looking to standardize machine learning practices. Just add `@step` and `@pipeline` to your existing Python functions to get going. Here is a toy example:
|
106
|
-
|
107
|
-
```python
|
108
|
-
from zenml import pipeline, step
|
109
|
-
|
110
|
-
@step # Just add this decorator
|
111
|
-
def load_data() -> dict:
|
112
|
-
training_data = [[1, 2], [3, 4], [5, 6]]
|
113
|
-
labels = [0, 1, 0]
|
114
|
-
return {'features': training_data, 'labels': labels}
|
115
|
-
|
116
|
-
@step
|
117
|
-
def train_model(data: dict) -> None:
|
118
|
-
total_features = sum(map(sum, data['features']))
|
119
|
-
total_labels = sum(data['labels'])
|
120
|
-
|
121
|
-
print(f"Trained model using {len(data['features'])} data points. "
|
122
|
-
f"Feature sum is {total_features}, label sum is {total_labels}")
|
123
|
-
|
124
|
-
@pipeline # This function combines steps together
|
125
|
-
def simple_ml_pipeline():
|
126
|
-
dataset = load_data()
|
127
|
-
train_model(dataset)
|
128
|
-
|
129
|
-
if __name__ == "__main__":
|
130
|
-
run = simple_ml_pipeline() # call this to run the pipeline
|
131
|
-
|
132
|
-
```
|
133
|
-
|
134
|
-

|
135
|
-
|
136
|
-
### Easily provision an MLOps stack or reuse your existing infrastructure
|
137
|
-
|
138
|
-
The framework is a gentle entry point for practitioners to build complex ML pipelines with little knowledge required of the underlying infrastructure complexity. ZenML pipelines can be run on AWS, GCP, Azure, Airflow, Kubeflow and even on Kubernetes without having to change any code or know underlying internals.
|
139
|
-
|
140
|
-
ZenML provides different features to aid people to get started quickly on a remote setting as well. If you want to deploy a remote stack from scratch on your selected cloud provider, you can use the 1-click deployment feature either through the dashboard:
|
141
|
-
|
142
|
-

|
143
|
-
|
144
|
-
Or, through our CLI command:
|
145
|
-
|
146
|
-
```bash
|
147
|
-
zenml stack deploy --provider aws
|
148
|
-
```
|
149
|
-
|
150
|
-
Alternatively, if the necessary pieces of infrastructure are already deployed, you can register a cloud stack seamlessly through the stack wizard:
|
151
|
-
|
152
|
-
```bash
|
153
|
-
zenml stack register <STACK_NAME> --provider aws
|
154
|
-
```
|
155
|
-
|
156
|
-
Read more about [ZenML stacks](https://docs.zenml.io/user-guide/production-guide/understand-stacks).
|
157
|
-
|
158
|
-
### Run workloads easily on your production infrastructure
|
159
|
-
|
160
|
-
Once you have your MLOps stack configured, you can easily run workloads on it:
|
161
|
-
|
162
|
-
```bash
|
163
|
-
zenml stack set <STACK_NAME>
|
164
|
-
python run.py
|
165
|
-
```
|
166
|
-
|
167
|
-
```python
|
168
|
-
from zenml.config import ResourceSettings, DockerSettings
|
169
|
-
|
170
|
-
@step(
|
171
|
-
settings={
|
172
|
-
"resources": ResourceSettings(memory="16GB", gpu_count="1", cpu_count="8"),
|
173
|
-
"docker": DockerSettings(parent_image="pytorch/pytorch:1.12.1-cuda11.3-cudnn8-runtime")
|
174
|
-
}
|
175
|
-
)
|
176
|
-
def training(...):
|
177
|
-
...
|
178
|
-
```
|
179
|
-
|
180
|
-

|
181
|
-
|
182
|
-
### Track models, pipeline, and artifacts
|
183
|
-
|
184
|
-
Create a complete lineage of who, where, and what data and models are produced.
|
185
|
-
|
186
|
-
You’ll be able to find out who produced which model, at what time, with which data, and on which version of the code. This guarantees full reproducibility and auditability.
|
187
|
-
|
188
|
-
```python
|
189
|
-
from zenml import Model
|
190
|
-
|
191
|
-
@step(model=Model(name="classification"))
|
192
|
-
def trainer(training_df: pd.DataFrame) -> Annotated["model", torch.nn.Module]:
|
193
|
-
...
|
194
|
-
```
|
195
|
-
|
196
|
-

|
197
|
-
|
198
|
-
### Purpose built for machine learning with integrations to your favorite tools
|
199
|
-
|
200
|
-
While ZenML brings a lot of value out of the box, it also integrates into your existing tooling and infrastructure without you having to be locked in.
|
201
|
-
|
202
|
-
```python
|
203
|
-
from bentoml._internal.bento import bento
|
204
|
-
|
205
|
-
@step(on_failure=alert_slack, experiment_tracker="mlflow")
|
206
|
-
def train_and_deploy(training_df: pd.DataFrame) -> bento.Bento
|
207
|
-
mlflow.autolog()
|
208
|
-
...
|
209
|
-
return bento
|
210
|
-
```
|
211
|
-
|
212
|
-

|
213
|
-
|
214
|
-
## 🖼️ Learning
|
215
|
-
|
216
|
-
The best way to learn about ZenML is the [docs](https://docs.zenml.io/). We recommend beginning with the [Starter Guide](https://docs.zenml.io/user-guide/starter-guide) to get up and running quickly.
|
217
|
-
|
218
|
-
If you are a visual learner, this 11-minute video tutorial is also a great start:
|
219
|
-
|
220
|
-
[](https://www.youtube.com/watch?v=wEVwIkDvUPs)
|
221
|
-
|
222
|
-
And finally, here are some other examples and use cases for inspiration:
|
223
|
-
|
224
|
-
1. [E2E Batch Inference](examples/e2e/): Feature engineering, training, and inference pipelines for tabular machine learning.
|
225
|
-
2. [Basic NLP with BERT](examples/e2e_nlp/): Feature engineering, training, and inference focused on NLP.
|
226
|
-
3. [LLM RAG Pipeline with Langchain and OpenAI](https://github.com/zenml-io/zenml-projects/tree/main/llm-agents): Using Langchain to create a simple RAG pipeline.
|
227
|
-
4. [Huggingface Model to Sagemaker Endpoint](https://github.com/zenml-io/zenml-projects/tree/main/huggingface-sagemaker): Automated MLOps on Amazon Sagemaker and HuggingFace
|
228
|
-
5. [LLMops](https://github.com/zenml-io/zenml-projects/tree/main/llm-complete-guide): Complete guide to do LLM with ZenML
|
229
|
-
|
230
|
-
|
231
|
-
## 📚 Learn from Books
|
232
|
-
|
233
|
-
<div align="center">
|
234
|
-
<a href="https://www.amazon.com/LLM-Engineers-Handbook-engineering-production/dp/1836200072">
|
235
|
-
<img src="docs/book/.gitbook/assets/llm_engineering_handbook_cover.jpg" alt="LLM Engineer's Handbook Cover" width="200"/></img>
|
236
|
-
</a>
|
237
|
-
<a href="https://www.amazon.com/-/en/Andrew-McMahon/dp/1837631964">
|
238
|
-
<img src="docs/book/.gitbook/assets/ml_engineering_with_python.jpg" alt="Machine Learning Engineering with Python Cover" width="200"/></img>
|
239
|
-
</a>
|
240
|
-
</br></br>
|
241
|
-
</div>
|
242
|
-
|
243
|
-
ZenML is featured in these comprehensive guides to modern MLOps and LLM engineering. Learn how to build production-ready machine learning systems with real-world examples and best practices.
|
244
|
-
|
245
|
-
## 🔋 Deploy ZenML
|
246
|
-
|
247
|
-
For full functionality ZenML should be deployed on the cloud to
|
248
|
-
enable collaborative features as the central MLOps interface for teams.
|
249
|
-
|
250
|
-
Read more about various deployment options [here](https://docs.zenml.io/getting-started/deploying-zenml).
|
251
|
-
|
252
|
-
Or, sign up for [ZenML Pro to get a fully managed server on a free trial](https://cloud.zenml.io/?utm_source=readme&utm_medium=referral_link&utm_campaign=cloud_promotion&utm_content=signup_link).
|
253
|
-
|
254
|
-
## Use ZenML with VS Code
|
255
|
-
|
256
|
-
ZenML has a [VS Code extension](https://marketplace.visualstudio.com/items?itemName=ZenML.zenml-vscode) that allows you to inspect your stacks and pipeline runs directly from your editor. The extension also allows you to switch your stacks without needing to type any CLI commands.
|
257
|
-
|
258
|
-
<details>
|
259
|
-
<summary>🖥️ VS Code Extension in Action!</summary>
|
260
|
-
<div align="center">
|
261
|
-
<img width="60%" src="/docs/book/.gitbook/assets/zenml-extension-shortened.gif" alt="ZenML Extension">
|
262
|
-
</div>
|
263
|
-
</details>
|
264
|
-
|
265
|
-
## 🗺 Roadmap
|
266
|
-
|
267
|
-
ZenML is being built in public. The [roadmap](https://zenml.io/roadmap) is a regularly updated source of truth for the ZenML community to understand where the product is going in the short, medium, and long term.
|
268
|
-
|
269
|
-
ZenML is managed by a [core team](https://zenml.io/company) of developers that are responsible for making key decisions and incorporating feedback from the community. The team oversees feedback via various channels,
|
270
|
-
and you can directly influence the roadmap as follows:
|
271
|
-
|
272
|
-
- Vote on your most wanted feature on our [Discussion
|
273
|
-
board](https://zenml.io/discussion).
|
274
|
-
- Start a thread in our [Slack channel](https://zenml.io/slack).
|
275
|
-
- [Create an issue](https://github.com/zenml-io/zenml/issues/new/choose) on our GitHub repo.
|
276
|
-
|
277
|
-
## 🙌 Contributing and Community
|
278
|
-
|
279
|
-
We would love to develop ZenML together with our community! The best way to get
|
280
|
-
started is to select any issue from the `[good-first-issue`
|
281
|
-
label](https://github.com/issues?q=is%3Aopen+is%3Aissue+archived%3Afalse+user%3Azenml-io+label%3A%22good+first+issue%22)
|
282
|
-
and open up a Pull Request!
|
283
|
-
|
284
|
-
If you
|
285
|
-
would like to contribute, please review our [Contributing
|
286
|
-
Guide](CONTRIBUTING.md) for all relevant details.
|
287
|
-
|
288
|
-
## 🆘 Getting Help
|
289
|
-
|
290
|
-
The first point of call should
|
291
|
-
be [our Slack group](https://zenml.io/slack-invite/).
|
292
|
-
Ask your questions about bugs or specific use cases, and someone from
|
293
|
-
the [core team](https://zenml.io/company) will respond.
|
294
|
-
Or, if you
|
295
|
-
prefer, [open an issue](https://github.com/zenml-io/zenml/issues/new/choose) on
|
296
|
-
our GitHub repo.
|
297
|
-
|
298
|
-
## ⭐️ Show Your Support
|
299
|
-
|
300
|
-
If you find ZenML helpful or interesting, please consider giving us a star on GitHub. Your support helps promote the project and lets others know that it's worth checking out.
|
301
|
-
|
302
|
-
Thank you for your support! 🌟
|
303
|
-
|
304
|
-
[](https://github.com/zenml-io/zenml/stargazers)
|
305
|
-
|
306
|
-
## 📜 License
|
307
|
-
|
308
|
-
ZenML is distributed under the terms of the Apache License Version 2.0.
|
309
|
-
A complete version of the license is available in the [LICENSE](LICENSE) file in
|
310
|
-
this repository. Any contribution made to this project will be licensed under
|
311
|
-
the Apache License Version 2.0.
|
312
|
-
|
313
|
-
<div>
|
314
|
-
<p align="left">
|
315
|
-
<div align="left">
|
316
|
-
Join our <a href="https://zenml.io/slack" target="_blank">
|
317
|
-
<img width="18" src="https://cdn3.iconfinder.com/data/icons/logos-and-brands-adobe/512/306_Slack-512.png" alt="Slack"/>
|
318
|
-
<b>Slack Community</b> </a> and be part of the ZenML family.
|
319
|
-
</div>
|
320
|
-
<br />
|
321
|
-
<a href="https://zenml.io/features">Features</a>
|
322
|
-
·
|
323
|
-
<a href="https://zenml.io/roadmap">Roadmap</a>
|
324
|
-
·
|
325
|
-
<a href="https://github.com/zenml-io/zenml/issues">Report Bug</a>
|
326
|
-
·
|
327
|
-
<a href="https://zenml.io/pro">Sign up for ZenML Pro</a>
|
328
|
-
·
|
329
|
-
<a href="https://www.zenml.io/blog">Read Blog</a>
|
330
|
-
·
|
331
|
-
<a href="https://github.com/issues?q=is%3Aopen+is%3Aissue+archived%3Afalse+user%3Azenml-io+label%3A%22good+first+issue%22">Contribute to Open Source</a>
|
332
|
-
·
|
333
|
-
<a href="https://github.com/zenml-io/zenml-projects">Projects Showcase</a>
|
334
|
-
<br />
|
335
|
-
<br />
|
336
|
-
🎉 Version 0.71.0 is out. Check out the release notes
|
337
|
-
<a href="https://github.com/zenml-io/zenml/releases">here</a>.
|
338
|
-
<br />
|
339
|
-
🖥️ Download our VS Code Extension <a href="https://marketplace.visualstudio.com/items?itemName=ZenML.zenml-vscode">here</a>.
|
340
|
-
<br />
|
341
|
-
</p>
|
342
|
-
</div>
|