flyte 0.2.0b4__py3-none-any.whl → 0.2.0b7__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of flyte might be problematic. Click here for more details.

Files changed (42) hide show
  1. flyte/__init__.py +2 -1
  2. flyte/_build.py +3 -2
  3. flyte/_code_bundle/_utils.py +0 -16
  4. flyte/_code_bundle/bundle.py +1 -1
  5. flyte/_deploy.py +4 -4
  6. flyte/_initialize.py +69 -26
  7. flyte/_internal/controllers/remote/_core.py +1 -1
  8. flyte/_protos/workflow/common_pb2.py +27 -0
  9. flyte/_protos/workflow/common_pb2.pyi +14 -0
  10. flyte/_protos/workflow/common_pb2_grpc.py +4 -0
  11. flyte/_protos/workflow/run_definition_pb2.py +14 -14
  12. flyte/_protos/workflow/run_definition_pb2.pyi +4 -2
  13. flyte/_protos/workflow/task_definition_pb2.py +14 -13
  14. flyte/_protos/workflow/task_definition_pb2.pyi +7 -3
  15. flyte/_run.py +7 -5
  16. flyte/_trace.py +1 -6
  17. flyte/_version.py +2 -2
  18. flyte/cli/_common.py +23 -15
  19. flyte/cli/_run.py +12 -6
  20. flyte/cli/main.py +15 -9
  21. flyte/config/__init__.py +2 -189
  22. flyte/config/_config.py +181 -172
  23. flyte/config/_internal.py +1 -1
  24. flyte/config/_reader.py +207 -0
  25. flyte/io/_dir.py +2 -2
  26. flyte/io/_file.py +1 -4
  27. flyte/remote/_data.py +3 -3
  28. flyte/remote/_logs.py +49 -26
  29. flyte/remote/_project.py +8 -9
  30. flyte/remote/_run.py +106 -61
  31. flyte/remote/_secret.py +12 -12
  32. flyte/remote/_task.py +3 -3
  33. flyte/report/_report.py +4 -4
  34. flyte/syncify/__init__.py +5 -0
  35. flyte/syncify/_api.py +277 -0
  36. flyte-0.2.0b7.dist-info/METADATA +156 -0
  37. {flyte-0.2.0b4.dist-info → flyte-0.2.0b7.dist-info}/RECORD +40 -35
  38. flyte/_api_commons.py +0 -3
  39. flyte-0.2.0b4.dist-info/METADATA +0 -179
  40. {flyte-0.2.0b4.dist-info → flyte-0.2.0b7.dist-info}/WHEEL +0 -0
  41. {flyte-0.2.0b4.dist-info → flyte-0.2.0b7.dist-info}/entry_points.txt +0 -0
  42. {flyte-0.2.0b4.dist-info → flyte-0.2.0b7.dist-info}/top_level.txt +0 -0
@@ -1,179 +0,0 @@
1
- Metadata-Version: 2.4
2
- Name: flyte
3
- Version: 0.2.0b4
4
- Summary: Add your description here
5
- Author-email: Ketan Umare <kumare3@users.noreply.github.com>
6
- Requires-Python: >=3.10
7
- Description-Content-Type: text/markdown
8
- Requires-Dist: aiofiles>=24.1.0
9
- Requires-Dist: click>=8.2.1
10
- Requires-Dist: flyteidl==1.15.4b0
11
- Requires-Dist: cloudpickle>=3.1.1
12
- Requires-Dist: fsspec>=2025.3.0
13
- Requires-Dist: grpcio>=1.71.0
14
- Requires-Dist: obstore>=0.6.0
15
- Requires-Dist: protobuf>=6.30.1
16
- Requires-Dist: pydantic>=2.10.6
17
- Requires-Dist: pyyaml>=6.0.2
18
- Requires-Dist: rich-click>=1.8.9
19
- Requires-Dist: httpx>=0.28.1
20
- Requires-Dist: keyring>=25.6.0
21
- Requires-Dist: synchronicity>=0.9.11
22
- Requires-Dist: msgpack>=1.1.0
23
- Requires-Dist: toml>=0.10.2
24
- Requires-Dist: async-lru>=2.0.5
25
- Requires-Dist: mashumaro
26
- Requires-Dist: dataclasses_json
27
-
28
- # Flyte 2
29
-
30
- Next-gen of SDK for Flyte.
31
-
32
- ## Get started
33
-
34
- 1. Only async tasks are supported right now. Style recommended, `import flyte` and then `flyte.TaskEnvironment` etc
35
- 2. You have to create environment for even a single task
36
- 3. `flyte run` CLI does not yet support arguments (but will soon)
37
- 4. Support for dataclasses is still limited if present at all
38
- 5. look at examples/... for various examples.
39
- 6. For a single script recommend using uv run scripts with metadata headers.
40
-
41
- Hello world
42
-
43
- ```python
44
- import flyte
45
-
46
- env = flyte.TaskEnvironment(name="hello_world")
47
-
48
-
49
- @env.task
50
- async def say_hello(data: str) -> str:
51
- return f"Hello {data}"
52
-
53
-
54
- @env.task
55
- async def say_hello_nested(data: str) -> str:
56
- return await say_hello.override(resources=flyte.Resources(gpu="A100 80G:4")).execute(data)
57
-
58
-
59
- if __name__ == "__main__":
60
- import asyncio
61
-
62
- # to run pure python - the SDK is not invoked at all
63
- asyncio.run(say_hello_nested("test"))
64
-
65
- # To run locally, but run through type system etc
66
- flyte.init()
67
- flyte.run(say_hello_nested, "World")
68
-
69
- # To run remote
70
- flyte.init(endpoint="dns:///localhost:8090", insecure=True)
71
- flyte.run(say_hello_nested, "World")
72
- # It is possible to switch local and remote, but keeping init to have and endpoint, but , changing context during run
73
- flyte.with_runcontext(name="my-run-1", mode="local").run(...) # this will run locally only
74
- ```
75
-
76
- ## When testing now, you will have to build images with v2 sdk installed.
77
- To do this, you can use a nifty way to add the sdk to the image.
78
-
79
- ```python
80
- import flyte
81
- import pathlib
82
-
83
- env = flyte.TaskEnvironment(
84
- name="test",
85
- image=flyte.Image.auto(registry="ghcr.io/your-handle", name="package-name")
86
- .with_source_folder(pathlib.Path(
87
- __file__).parent.parent / "dist") # Assuming you have run `make dist` and the dist folder is in the directory above
88
- .with_local_v2(), # This will add the v2 sdk to the image from the local dist folder
89
- )
90
- ```
91
-
92
- ## Run using CLI
93
-
94
- ```bash
95
- flyte --endpoint dns:///localhost:8090 --insecure --org-override testorg run -p project -d development devbox_one.py say_hello_nested
96
- ```
97
- ### Retrieve runs
98
- ```bash
99
- flyte --endpoint dns:///localhost:8090 --insecure --org-override testorg get run --project testproject --domain development
100
- ```
101
- Output:
102
- ```bash
103
- ┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
104
- ┃ Run-name ┃ Name ┃ Start_time ┃ End_time ┃ Run_time ┃ Phase ┃ Task ┃
105
- ┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
106
- │ testrun │ a0 │ 2025-04-25T21:47:37.564471 │ None │ 41154 secs │ PHASE_QUEUED │ │
107
- │ random-run-5783fbc8 │ a0 │ 2025-04-29T04:39:33.871810 │ 2025-04-29T04:42:24 │ 170 secs │ PHASE_SUCCEEDED │ │
108
- │ random-run-e57d6195 │ a0 │ 2025-04-29T04:40:53.467206 │ 2025-04-29T04:42:54 │ 120 secs │ PHASE_SUCCEEDED │ say_hello_nested │
109
- │ random-run-752c52a0 │ a0 │ 2025-04-29T04:44:50.169879 │ 2025-04-29T04:47:34 │ 163 secs │ PHASE_SUCCEEDED │ failure_recovery │
110
- │ random-run-4f57d4af │ a0 │ 2025-04-29T05:04:28.780208 │ 2025-04-29T05:07:10 │ 161 secs │ PHASE_FAILED │ failure_recovery │
111
- │ random-run-f6c6405f │ a0 │ 2025-04-29T05:10:19.808186 │ 2025-04-29T05:13:07 │ 167 secs │ PHASE_FAILED │ failure_recovery │
112
- │ random-run-66448df4 │ a0 │ 2025-04-29T05:42:21.625827 │ 2025-04-29T05:45:01 │ 159 secs │ PHASE_SUCCEEDED │ failure_recovery │
113
- │ random-run-5c5c905f │ a0 │ 2025-04-29T05:50:32.623097 │ 2025-04-29T05:53:21 │ 168 secs │ PHASE_SUCCEEDED │ failure_recovery │
114
- │ random-run-99056dba │ a0 │ 2025-04-29T05:54:47.820520 │ 2025-04-29T05:57:31 │ 163 secs │ PHASE_SUCCEEDED │ failure_recovery │
115
- └─────────────────────┴──────┴────────────────────────────┴─────────────────────┴────────────┴─────────────────┴──────────────────┘
116
- ```
117
- ### Retrieve specific run details
118
- ```bash
119
- flyte --endpoint dns:///localhost:8090 --insecure --org-override testorg get run --project testproject --domain development random-run-66448df4
120
- ```
121
- Output:
122
- ```bash
123
- RunDetails(name='random-run-66448df4', start_time='2025-04-29T05:42:21.625827', end_time='2025-04-29T05:45:01', run_time='159 secs', phase='PHASE_SUCCEEDED', error=, task='failure_recovery')
124
- ```
125
-
126
- ### Retrieve actions for a run
127
- ```bash
128
- flyte --endpoint dns:///localhost:8090 --insecure --org-override testorg get action --project testproject --domain development random-run-99056dba
129
- ```
130
-
131
- Output:
132
- ```bash
133
- ┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
134
- ┃ Run-name ┃ Name ┃ Start_time ┃ End_time ┃ Run_time ┃ Phase ┃ Task ┃
135
- ┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
136
- │ random-run-99056dba │ a0 │ 2025-04-29T05:54:47.820520 │ 2025-04-29T05:57:31 │ 163 secs │ PHASE_SUCCEEDED │ failure_recovery │
137
- │ random-run-99056dba │ 5DYnhmntDf3WG9bm5B54E9 │ 2025-04-29T05:55:18.098842 │ 2025-04-29T05:55:51 │ 32 secs │ PHASE_FAILED │ always_fails │
138
- │ random-run-99056dba │ 7rg3HTmTfCjMqhLLq17ZjR │ 2025-04-29T05:55:51.933307 │ 2025-04-29T05:56:31 │ 39 secs │ PHASE_FAILED │ always_fails │
139
- │ random-run-99056dba │ 3avSConCJYUMmCuTpqUCdO │ 2025-04-29T05:56:31.939420 │ 2025-04-29T05:57:11 │ 39 secs │ PHASE_SUCCEEDED │ always_succeeds │
140
- └─────────────────────┴────────────────────────┴────────────────────────────┴─────────────────────┴──────────┴─────────────────┴──────────────────┘
141
- ```
142
-
143
- ### Retrieve action details
144
- ```bash
145
- flyte --endpoint dns:///localhost:8090 --insecure --org-override testorg get action --project testproject --domain development random-run-99056dba 5DYnhmntDf3WG9bm5B54E9
146
- ```
147
-
148
- Output:
149
- ```bash
150
- ActionDetails(name='random-run-99056dba', start_time='2025-04-29T05:55:18.098842', end_time='2025-04-29T05:55:51', run_time='32 secs', phase='PHASE_FAILED', error=, task='always_fails')
151
- ```
152
-
153
- ### Deploy
154
- ```bash
155
- flyte deploy --endpoint dns:///localhost:8090 --insecure --org-override testorg -p project -d development devbox_one.py say_hello_nested
156
- ```
157
- # Contributor Notes
158
- Some of these are limited to Union for now, because the new IDL has not yet been open-sourced.
159
-
160
- ## How to import proto stubs?
161
-
162
- Make sure that you have `uv` installed locally then run `make copy-protos <path-to-cloud-repo>`, where `<path-to-cloud-repo>`
163
- is optional, if it's not defined then it assumes by default that there's a sibling `cloud` directory.
164
-
165
- ## Iterating on the SDK itself.
166
- While there is a publicly available semantically version base image (similar to flytekit) for each flyte sdk release,
167
- it's probably helpful to create your own images if you're to iterate on this SDK itself. Anyone who wishes to modify
168
- this code and then run it on a Flyte cluster, will need to build their own image.
169
-
170
- To do this:
171
-
172
- 1. Build a wheel with your updated code by running `make dist`
173
-
174
- 2. (This step only works for Union employees for now) Create the image by running
175
- ```
176
- python maint_tools/build_default_image.py
177
- ```
178
- This will build the image using buildx, creating a buildx builder. The image will be tagged with a SHA that changes
179
- based on a hash of the built wheel inside the dist folder.