kaggle-fleet 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- kaggle_fleet-0.1.0/LICENSE +21 -0
- kaggle_fleet-0.1.0/PKG-INFO +364 -0
- kaggle_fleet-0.1.0/README.md +332 -0
- kaggle_fleet-0.1.0/pyproject.toml +67 -0
- kaggle_fleet-0.1.0/setup.cfg +4 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/__init__.py +21 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/__main__.py +12 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/account_manager.py +687 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/cli.py +860 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/config.py +117 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/kaggle_runner.py +399 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/models.py +187 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/notebook_manager.py +205 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/server/__init__.py +7 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/server/api_handlers.py +439 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/server/http_server.py +334 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/server/static/app.js +704 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/server/static/index.html +306 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/server/static/styles.css +310 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet/storage.py +132 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet.egg-info/PKG-INFO +364 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet.egg-info/SOURCES.txt +32 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet.egg-info/dependency_links.txt +1 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet.egg-info/entry_points.txt +3 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet.egg-info/requires.txt +6 -0
- kaggle_fleet-0.1.0/src/kaggle_fleet.egg-info/top_level.txt +1 -0
- kaggle_fleet-0.1.0/tests/test_cli_json.py +120 -0
- kaggle_fleet-0.1.0/tests/test_core.py +431 -0
- kaggle_fleet-0.1.0/tests/test_isolation_fixes.py +186 -0
- kaggle_fleet-0.1.0/tests/test_link_login.py +111 -0
- kaggle_fleet-0.1.0/tests/test_server_and_cli.py +427 -0
- kaggle_fleet-0.1.0/tests/test_server_e2e_verification.py +349 -0
- kaggle_fleet-0.1.0/tests/test_verification.py +389 -0
- kaggle_fleet-0.1.0/tests/test_web_api.py +92 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 The KaggleFleet contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,364 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: kaggle-fleet
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Run local Jupyter notebooks on Kaggle GPUs across a team of Kaggle accounts - from a CLI, a local web panel, or an AI agent.
|
|
5
|
+
Author: KaggleFleet contributors
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/towhaEL/KaggleFleet
|
|
8
|
+
Project-URL: Issues, https://github.com/towhaEL/KaggleFleet/issues
|
|
9
|
+
Project-URL: Changelog, https://github.com/towhaEL/KaggleFleet/blob/main/CHANGELOG.md
|
|
10
|
+
Keywords: kaggle,gpu,jupyter,notebook,kernels,research,multi-account,cli,ai-agents
|
|
11
|
+
Classifier: Development Status :: 4 - Beta
|
|
12
|
+
Classifier: Environment :: Console
|
|
13
|
+
Classifier: Environment :: Web Environment
|
|
14
|
+
Classifier: Intended Audience :: Science/Research
|
|
15
|
+
Classifier: Intended Audience :: Developers
|
|
16
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
17
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
18
|
+
Classifier: Operating System :: MacOS
|
|
19
|
+
Classifier: Programming Language :: Python :: 3
|
|
20
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
21
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
22
|
+
Classifier: Topic :: Utilities
|
|
23
|
+
Requires-Python: >=3.9
|
|
24
|
+
Description-Content-Type: text/markdown
|
|
25
|
+
License-File: LICENSE
|
|
26
|
+
Requires-Dist: kaggle>=1.7
|
|
27
|
+
Requires-Dist: six
|
|
28
|
+
Provides-Extra: dev
|
|
29
|
+
Requires-Dist: pytest>=7; extra == "dev"
|
|
30
|
+
Requires-Dist: ruff>=0.5; extra == "dev"
|
|
31
|
+
Dynamic: license-file
|
|
32
|
+
|
|
33
|
+
<p align="center">
|
|
34
|
+
<img src="https://raw.githubusercontent.com/towhaEL/KaggleFleet/main/docs/images/banner.png"
|
|
35
|
+
alt="KaggleFleet: run local Jupyter notebooks on Kaggle GPUs across your research team's own accounts" width="100%">
|
|
36
|
+
</p>
|
|
37
|
+
|
|
38
|
+
<p align="center">
|
|
39
|
+
<a href="https://pypi.org/project/kaggle-fleet/"><img src="https://img.shields.io/pypi/v/kaggle-fleet?color=6d5dfc" alt="PyPI version"></a>
|
|
40
|
+
<a href="https://pypi.org/project/kaggle-fleet/"><img src="https://img.shields.io/pypi/pyversions/kaggle-fleet" alt="Python versions"></a>
|
|
41
|
+
<a href="https://github.com/towhaEL/KaggleFleet/actions/workflows/ci.yml"><img src="https://github.com/towhaEL/KaggleFleet/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
|
|
42
|
+
<a href="https://github.com/towhaEL/KaggleFleet/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-green" alt="MIT license"></a>
|
|
43
|
+
<img src="https://img.shields.io/badge/platform-Linux%20%7C%20macOS%20%7C%20WSL-lightgrey" alt="Linux, macOS, WSL">
|
|
44
|
+
</p>
|
|
45
|
+
|
|
46
|
+
<p align="center">
|
|
47
|
+
<b>Run your local Jupyter notebooks on Kaggle GPUs, across your research team's own Kaggle accounts,<br>
|
|
48
|
+
from the terminal, a local web panel, or an AI agent.</b>
|
|
49
|
+
</p>
|
|
50
|
+
|
|
51
|
+
<p align="center">
|
|
52
|
+
<a href="#install">Install</a> ·
|
|
53
|
+
<a href="#quick-start">Quick start</a> ·
|
|
54
|
+
<a href="#web-panel">Web panel</a> ·
|
|
55
|
+
<a href="#using-kagglefleet-from-ai-agents-and-scripts">AI agents</a> ·
|
|
56
|
+
<a href="https://github.com/towhaEL/KaggleFleet/blob/main/CHANGELOG.md">Changelog</a>
|
|
57
|
+
</p>
|
|
58
|
+
|
|
59
|
+
<p align="center">
|
|
60
|
+
<img src="https://raw.githubusercontent.com/towhaEL/KaggleFleet/main/docs/images/demo.gif"
|
|
61
|
+
alt="Terminal demo: list accounts and notebooks, push a notebook to a co-author's account, check status and logs, download the outputs" width="860">
|
|
62
|
+
</p>
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
pipx install kaggle-fleet
|
|
66
|
+
kfleet account login --label "Me"
|
|
67
|
+
kfleet run push train.ipynb --watch # runs on a Kaggle GPU, downloads outputs when done
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## Why KaggleFleet?
|
|
71
|
+
|
|
72
|
+
Kaggle gives every account a free weekly GPU quota. A research group with several
|
|
73
|
+
co-authors has several quotas, but using them means logging in and out of Kaggle,
|
|
74
|
+
copying notebooks around, and tracking who ran what. KaggleFleet does that part for you:
|
|
75
|
+
|
|
76
|
+
- **Keep your notebooks local.** Edit in your own editor and git repo; KaggleFleet stages and pushes them as Kaggle kernels.
|
|
77
|
+
- **One machine, many co-author accounts.** Each Kaggle account lives in its own credential sandbox, so runs can go to different accounts at the same time without leaking into each other.
|
|
78
|
+
- **Push, watch, collect.** Pick an account and accelerator (T4, P100, TPU or CPU), push, follow status and logs, and download the outputs when the run finishes.
|
|
79
|
+
- **A web panel for the team.** Accounts, approximate GPU hours used per account this week, live job status, logs, and one-click launches.
|
|
80
|
+
- **Built for AI agents.** Every CLI command has `--json` output and meaningful exit codes, and [`AGENTS.md`](https://github.com/towhaEL/KaggleFleet/blob/main/AGENTS.md) documents the workflow for coding agents.
|
|
81
|
+
- **No runtime dependencies** besides the official `kaggle` CLI. The tool itself is pure Python standard library.
|
|
82
|
+
|
|
83
|
+
| Accounts | Jobs |
|
|
84
|
+
|---|---|
|
|
85
|
+
|  |  |
|
|
86
|
+
|
|
87
|
+
> [!IMPORTANT]
|
|
88
|
+
> KaggleFleet is for **teams of real people**, each using **their own** Kaggle account with their consent
|
|
89
|
+
> (for example co-authors on a paper). Kaggle's [Terms of Use](https://www.kaggle.com/terms) allow one account
|
|
90
|
+
> per person. Do not create extra accounts to get more GPU quota. KaggleFleet is not affiliated with Kaggle or Google.
|
|
91
|
+
|
|
92
|
+
---
|
|
93
|
+
|
|
94
|
+
## Contents
|
|
95
|
+
|
|
96
|
+
- [Install](#install)
|
|
97
|
+
- [Quick start](#quick-start)
|
|
98
|
+
- [Adding co-author accounts](#adding-co-author-accounts)
|
|
99
|
+
- [Running notebooks](#running-notebooks)
|
|
100
|
+
- [Web panel](#web-panel)
|
|
101
|
+
- [Using KaggleFleet from AI agents and scripts](#using-kagglefleet-from-ai-agents-and-scripts)
|
|
102
|
+
- [Configuration](#configuration)
|
|
103
|
+
- [How credential isolation works](#how-credential-isolation-works)
|
|
104
|
+
- [CLI reference](#cli-reference)
|
|
105
|
+
- [Limitations](#limitations)
|
|
106
|
+
- [Development](#development)
|
|
107
|
+
- [Contributing and releases](#contributing-and-releases)
|
|
108
|
+
|
|
109
|
+
## Install
|
|
110
|
+
|
|
111
|
+
You need Python 3.9 or newer on **Linux or macOS**. On Windows, use WSL.
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
pipx install kaggle-fleet # recommended: isolated, puts `kfleet` on your PATH
|
|
115
|
+
# or
|
|
116
|
+
pip install kaggle-fleet
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
To try the latest unreleased code: `pipx install git+https://github.com/towhaEL/KaggleFleet.git`.
|
|
120
|
+
|
|
121
|
+
This installs two commands, `kfleet` and its longer alias `kaggle-fleet`, and pulls in the official `kaggle` CLI.
|
|
122
|
+
Browser sign-in (`kfleet account login`) requires Kaggle CLI 2.x. API-key accounts work with 1.7 and newer.
|
|
123
|
+
|
|
124
|
+
Check your setup:
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
kfleet doctor
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
## Quick start
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
cd my-research-project/ # a folder with your notebooks, or a ./notebooks subfolder
|
|
134
|
+
|
|
135
|
+
kfleet account login --label "Me" # sign in with your Kaggle account in the browser
|
|
136
|
+
kfleet notebook list # see what KaggleFleet found
|
|
137
|
+
kfleet run push train.ipynb --watch
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
To try it on something small first, push [`examples/hello_gpu.ipynb`](https://github.com/towhaEL/KaggleFleet/blob/main/examples/hello_gpu.ipynb), which reports the GPU it got.
|
|
141
|
+
|
|
142
|
+
With `--watch`, KaggleFleet polls the run until it finishes and then downloads the kernel's
|
|
143
|
+
output files to `./outputs/<kernel-slug>/`. Without it, the push returns right away and you can
|
|
144
|
+
check on the run later:
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
kfleet run list
|
|
148
|
+
kfleet run status # latest job; or pass a job id / kernel slug
|
|
149
|
+
kfleet run logs
|
|
150
|
+
kfleet run download
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
Or open the web panel:
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
kfleet serve --open # http://127.0.0.1:8080
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
## Adding co-author accounts
|
|
160
|
+
|
|
161
|
+
Each co-author adds their own account once, on the machine that runs KaggleFleet.
|
|
162
|
+
|
|
163
|
+
### Option A: browser sign-in (recommended)
|
|
164
|
+
|
|
165
|
+
```bash
|
|
166
|
+
kfleet account login --label "Co-author B"
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
This opens Kaggle's sign-in page in your normal browser. **Whichever Kaggle account is signed in
|
|
170
|
+
there is the account that gets added**, so switch the browser to the co-author's account first,
|
|
171
|
+
or use `--no-browser` to get a link you can open in a private window or on another device and
|
|
172
|
+
paste the verification code back.
|
|
173
|
+
|
|
174
|
+
The login runs inside a throwaway sandbox. Your own `~/.kaggle` login is never read or changed,
|
|
175
|
+
and adding an account that is already configured is refused.
|
|
176
|
+
|
|
177
|
+
In the web panel, **Sign in with browser** offers the same flow as a link plus a code field, which
|
|
178
|
+
works well when a co-author signs in from their own laptop.
|
|
179
|
+
|
|
180
|
+
### Option B: API key (`kaggle.json`)
|
|
181
|
+
|
|
182
|
+
The co-author creates a token at **kaggle.com → Settings → API → Create New Token** and sends you
|
|
183
|
+
the `kaggle.json` file, or its username and key:
|
|
184
|
+
|
|
185
|
+
```bash
|
|
186
|
+
kfleet account add --username alice --key 0123456789abcdef... --label "Alice"
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
### Option C: import what is already on this machine
|
|
190
|
+
|
|
191
|
+
```bash
|
|
192
|
+
kfleet account import # picks up ~/.kaggle/kaggle.json and ~/.kaggle/credentials.json
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
### Managing accounts
|
|
196
|
+
|
|
197
|
+
```bash
|
|
198
|
+
kfleet account list
|
|
199
|
+
kfleet account test alice # checks that the credentials authenticate as *alice*, not just as someone
|
|
200
|
+
kfleet account switch alice # change the default account for pushes
|
|
201
|
+
kfleet account remove alice # deletes the local sandbox only; the Kaggle account is untouched
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
## Running notebooks
|
|
205
|
+
|
|
206
|
+
```bash
|
|
207
|
+
kfleet run push 02_train.ipynb \
|
|
208
|
+
--account bob \
|
|
209
|
+
--accelerator NvidiaTeslaP100 \
|
|
210
|
+
--dataset bob/preprocessed-data \
|
|
211
|
+
--kernel-source alice/nb-00-data-prep \
|
|
212
|
+
--max-runtime 7200 \
|
|
213
|
+
--watch
|
|
214
|
+
```
|
|
215
|
+
|
|
216
|
+
| Option | Meaning |
|
|
217
|
+
|---|---|
|
|
218
|
+
| `--account`, `-a` | Account id or username. Defaults to the active account. |
|
|
219
|
+
| `--accelerator` | `NvidiaTeslaT4` (default), `NvidiaTeslaP100`, `TpuV5E8` or `None` |
|
|
220
|
+
| `--no-gpu` | Run on CPU |
|
|
221
|
+
| `--no-internet` | Disable internet access inside the kernel |
|
|
222
|
+
| `--dataset`, `--competition`, `--kernel-source`, `--model` | Attach Kaggle inputs. Each flag can be repeated. |
|
|
223
|
+
| `--title`, `--slug` | Override the kernel title and slug, which are derived from the notebook by default |
|
|
224
|
+
| `--max-runtime SECONDS` | Ask Kaggle to stop the kernel after this long. The default is Kaggle's own session limit. |
|
|
225
|
+
| `--watch`, `-w` | Wait for completion, then download outputs (turn this off with `--no-download`) |
|
|
226
|
+
| `--timeout SECONDS` | How long `--watch` waits. The kernel itself is not affected. |
|
|
227
|
+
|
|
228
|
+
**Kernel naming.** The kernel title comes from the notebook's first `# heading`. The slug comes from the
|
|
229
|
+
file name: `02_train.ipynb` becomes `<username>/nb-02-train`. File names that start with a digit get
|
|
230
|
+
a prefix, `nb-` by default; set `KAGGLE_FLEET_SLUG_PREFIX=myproject` to use your project's name instead.
|
|
231
|
+
Kernels are always created as **private**.
|
|
232
|
+
|
|
233
|
+
**Re-running a notebook** pushes a new version of the same kernel. Every push starts a new run on
|
|
234
|
+
Kaggle and uses GPU quota.
|
|
235
|
+
|
|
236
|
+
## Web panel
|
|
237
|
+
|
|
238
|
+
```bash
|
|
239
|
+
kfleet serve # http://127.0.0.1:8080
|
|
240
|
+
kfleet serve --port 9000 --open
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
The panel has three views:
|
|
244
|
+
|
|
245
|
+
- **Accounts:** add, test, switch and remove accounts, with each account's estimated GPU hours for the last 7 days.
|
|
246
|
+
- **Launch notebook:** pick a notebook, an account and hardware, then push.
|
|
247
|
+
- **Jobs:** live status, logs, refresh and download for every run.
|
|
248
|
+
|
|
249
|
+
The panel has **no login**. It listens on `127.0.0.1` and rejects requests from other websites
|
|
250
|
+
(it checks the `Origin` and `Host` headers and requires JSON request bodies). To use it from another
|
|
251
|
+
machine, prefer an SSH tunnel over exposing it:
|
|
252
|
+
|
|
253
|
+
```bash
|
|
254
|
+
ssh -L 8080:127.0.0.1:8080 your-gpu-box # then open http://127.0.0.1:8080 locally
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
The panel is backed by a small JSON REST API, documented in [`docs/rest-api.md`](https://github.com/towhaEL/KaggleFleet/blob/main/docs/rest-api.md).
|
|
258
|
+
|
|
259
|
+
## Using KaggleFleet from AI agents and scripts
|
|
260
|
+
|
|
261
|
+
Add `--json` to any command, before or after the subcommand, to get a single JSON document on stdout.
|
|
262
|
+
Failures print `{"error": "..."}` and exit with a non-zero code. Account secrets are never printed.
|
|
263
|
+
|
|
264
|
+
```bash
|
|
265
|
+
$ kfleet run push 02_train.ipynb --account bob --json
|
|
266
|
+
{
|
|
267
|
+
"success": true,
|
|
268
|
+
"job": {
|
|
269
|
+
"id": "job_20261121_101500_nb-02-train_3f2a",
|
|
270
|
+
"kernel_id": "bob/nb-02-train",
|
|
271
|
+
"status": "RUNNING",
|
|
272
|
+
...
|
|
273
|
+
}
|
|
274
|
+
}
|
|
275
|
+
|
|
276
|
+
$ kfleet run status job_20261121_101500_nb-02-train_3f2a --json
|
|
277
|
+
$ kfleet run logs job_20261121_101500_nb-02-train_3f2a --json
|
|
278
|
+
$ kfleet run download job_20261121_101500_nb-02-train_3f2a --json # lists the downloaded files
|
|
279
|
+
```
|
|
280
|
+
|
|
281
|
+
[`AGENTS.md`](https://github.com/towhaEL/KaggleFleet/blob/main/AGENTS.md) is a guide written for coding agents such as Claude Code or Codex:
|
|
282
|
+
the workflow, the JSON shapes, how to pick an account with spare GPU quota, and what not to do.
|
|
283
|
+
Copy it, or link to it, from your research repo's own `AGENTS.md` / `CLAUDE.md`.
|
|
284
|
+
|
|
285
|
+
## Configuration
|
|
286
|
+
|
|
287
|
+
| Environment variable | Default | Purpose |
|
|
288
|
+
|---|---|---|
|
|
289
|
+
| `KAGGLE_FLEET_DIR` | `~/.kaggle_fleet` | State: accounts, credential sandboxes, job history, staging |
|
|
290
|
+
| `KAGGLE_FLEET_NOTEBOOKS_DIR` | `./notebooks` if it exists, else `.` | Where notebooks are discovered |
|
|
291
|
+
| `KAGGLE_FLEET_OUTPUTS_DIR` | `./outputs` | Where kernel outputs are downloaded |
|
|
292
|
+
| `KAGGLE_FLEET_KAGGLE_BIN` | auto-detected | Path to the `kaggle` executable |
|
|
293
|
+
| `KAGGLE_FLEET_SLUG_PREFIX` | `nb` | Prefix for slugs derived from file names that start with a digit |
|
|
294
|
+
|
|
295
|
+
`--notebooks-dir` and `--outputs-dir` override the matching variables for a single command.
|
|
296
|
+
|
|
297
|
+
## How credential isolation works
|
|
298
|
+
|
|
299
|
+
Every account gets a private directory, `~/.kaggle_fleet/envs/<account-id>/` (mode `0700`, files `0600`).
|
|
300
|
+
Each `kaggle` command for that account runs with:
|
|
301
|
+
|
|
302
|
+
- `KAGGLE_CONFIG_DIR` pointing at that directory, and
|
|
303
|
+
- `HOME` set to a per-account home whose `.kaggle` links to that directory. The Kaggle SDK reads OAuth tokens
|
|
304
|
+
from the hard-coded path `~/.kaggle/credentials.json` and ignores `KAGGLE_CONFIG_DIR`, so without a
|
|
305
|
+
separate `HOME`, every OAuth account would silently run as whoever is logged in to your real `~/.kaggle`.
|
|
306
|
+
- Any `KAGGLE_KEY` or `KAGGLE_API_TOKEN` from your shell removed, because those take priority over all other credentials.
|
|
307
|
+
|
|
308
|
+
When the Kaggle CLI refreshes an OAuth token, the new token is copied back into `accounts.json`.
|
|
309
|
+
`kfleet account test` checks the identity Kaggle reports (`kaggle config view`), so a credential
|
|
310
|
+
leaking from another account would be caught instead of passing silently.
|
|
311
|
+
|
|
312
|
+
Credentials are stored **unencrypted** on disk and protected only by file permissions, the same way
|
|
313
|
+
the Kaggle CLI stores them. Run KaggleFleet on a machine you trust. See [`SECURITY.md`](https://github.com/towhaEL/KaggleFleet/blob/main/SECURITY.md).
|
|
314
|
+
|
|
315
|
+
## CLI reference
|
|
316
|
+
|
|
317
|
+
```
|
|
318
|
+
kfleet [--json] [--notebooks-dir DIR] [--outputs-dir DIR] <command>
|
|
319
|
+
|
|
320
|
+
account list | add | add-oauth | login | switch | test | remove | import
|
|
321
|
+
notebook list | inspect <notebook>
|
|
322
|
+
run push <notebook> | list | status [job] | logs [job] | download [job]
|
|
323
|
+
serve [--host] [--port] [--open] [--allow-host HOST]
|
|
324
|
+
doctor
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
Run `kfleet <command> <subcommand> --help` for every option.
|
|
328
|
+
`python -m kaggle_fleet` works the same way as `kfleet`.
|
|
329
|
+
|
|
330
|
+
## Limitations
|
|
331
|
+
|
|
332
|
+
- **The GPU hours shown are estimates.** They count only runs launched through KaggleFleet, and a run's end time is when KaggleFleet noticed it finished. Kaggle's own quota page is authoritative.
|
|
333
|
+
- **Status is polled.** Status comes from `kaggle kernels status`, so runs are refreshed while you watch them (CLI `--watch` or the web panel) or when you ask (`run status`).
|
|
334
|
+
- **Kernels cannot be cancelled from here** because the Kaggle CLI does not support it. Stop a run on kaggle.com.
|
|
335
|
+
- **Linux and macOS only** (symlink-based sandboxes). On Windows, use WSL.
|
|
336
|
+
|
|
337
|
+
## Development
|
|
338
|
+
|
|
339
|
+
```bash
|
|
340
|
+
git clone https://github.com/towhaEL/KaggleFleet.git
|
|
341
|
+
cd KaggleFleet
|
|
342
|
+
python -m venv .venv && source .venv/bin/activate
|
|
343
|
+
pip install -e ".[dev]"
|
|
344
|
+
|
|
345
|
+
pytest # or: python -m unittest discover -s tests -t .
|
|
346
|
+
ruff check src tests
|
|
347
|
+
```
|
|
348
|
+
|
|
349
|
+
The test suite never contacts Kaggle: the `kaggle` CLI is mocked or replaced with a fake script.
|
|
350
|
+
See [`CONTRIBUTING.md`](https://github.com/towhaEL/KaggleFleet/blob/main/CONTRIBUTING.md).
|
|
351
|
+
|
|
352
|
+
## Contributing and releases
|
|
353
|
+
|
|
354
|
+
Bug reports, feature ideas and pull requests are welcome. Please read
|
|
355
|
+
[`CONTRIBUTING.md`](https://github.com/towhaEL/KaggleFleet/blob/main/CONTRIBUTING.md) first, and report
|
|
356
|
+
security issues privately as described in [`SECURITY.md`](https://github.com/towhaEL/KaggleFleet/blob/main/SECURITY.md).
|
|
357
|
+
|
|
358
|
+
Releases are listed on the [GitHub releases page](https://github.com/towhaEL/KaggleFleet/releases) and
|
|
359
|
+
[PyPI](https://pypi.org/project/kaggle-fleet/). Maintainers: see
|
|
360
|
+
[`docs/RELEASING.md`](https://github.com/towhaEL/KaggleFleet/blob/main/docs/RELEASING.md).
|
|
361
|
+
|
|
362
|
+
## License
|
|
363
|
+
|
|
364
|
+
[MIT](https://github.com/towhaEL/KaggleFleet/blob/main/LICENSE)
|