runlab-client 0.2.0__tar.gz → 0.2.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {runlab_client-0.2.0/src/runlab_client.egg-info → runlab_client-0.2.1}/PKG-INFO +25 -10
- {runlab_client-0.2.0 → runlab_client-0.2.1}/README.md +24 -9
- {runlab_client-0.2.0 → runlab_client-0.2.1}/pyproject.toml +3 -1
- {runlab_client-0.2.0 → runlab_client-0.2.1}/src/runlab_client/__init__.py +15 -7
- runlab_client-0.2.1/src/runlab_client/__main__.py +6 -0
- {runlab_client-0.2.0 → runlab_client-0.2.1/src/runlab_client.egg-info}/PKG-INFO +25 -10
- {runlab_client-0.2.0 → runlab_client-0.2.1}/src/runlab_client.egg-info/SOURCES.txt +1 -0
- {runlab_client-0.2.0 → runlab_client-0.2.1}/src/runlab_client.egg-info/entry_points.txt +1 -0
- {runlab_client-0.2.0 → runlab_client-0.2.1}/LICENSE +0 -0
- {runlab_client-0.2.0 → runlab_client-0.2.1}/setup.cfg +0 -0
- {runlab_client-0.2.0 → runlab_client-0.2.1}/src/runlab_client.egg-info/dependency_links.txt +0 -0
- {runlab_client-0.2.0 → runlab_client-0.2.1}/src/runlab_client.egg-info/top_level.txt +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: runlab-client
|
|
3
|
-
Version: 0.2.
|
|
3
|
+
Version: 0.2.1
|
|
4
4
|
Summary: Python client for the RunLab research API: pre-registered A/B comparisons over seeds, verdicts with confidence intervals. Standard library only.
|
|
5
5
|
Author: omer-nachman
|
|
6
6
|
License-Expression: MIT
|
|
@@ -19,8 +19,13 @@ its baseline. RunLab gives back a verdict you can quote, built from:
|
|
|
19
19
|
- a paired test across seeds, with a 95 % confidence interval;
|
|
20
20
|
- checks that refuse a comparison that would mislead, e.g. arms tuned with unequal budgets.
|
|
21
21
|
|
|
22
|
-
|
|
23
|
-
|
|
22
|
+
The RunLab Python client gives access to the RunLab API using only the standard library,
|
|
23
|
+
on Python 3.10+.
|
|
24
|
+
|
|
25
|
+
## Documentation
|
|
26
|
+
|
|
27
|
+
- [API reference](https://omer-nachman.github.io/runlab-python/api.html)
|
|
28
|
+
- [Examples](https://github.com/omer-nachman/runlab-python/tree/main/examples): Python and curl
|
|
24
29
|
|
|
25
30
|
## Get started
|
|
26
31
|
|
|
@@ -33,6 +38,9 @@ runlab login # sign in with GitHub, approve this termina
|
|
|
33
38
|
runlab whoami # ada (agent) at https://runlab.tools
|
|
34
39
|
```
|
|
35
40
|
|
|
41
|
+
- Another program called `runlab` first on your PATH? Every command here also runs as
|
|
42
|
+
`runlab-client login | whoami | logout | worker` (or `python -m runlab_client ...`), and
|
|
43
|
+
`runlab-client --version` names the version.
|
|
36
44
|
- `runlab login` opens a browser when this machine has a display. On an SSH session or a
|
|
37
45
|
headless GPU box, open the printed link on any device (your laptop or phone).
|
|
38
46
|
- The page asks "Authorize this terminal?" and shows the code, the address and host name
|
|
@@ -76,8 +84,10 @@ for c in rl.report("resnet-aug")["claims"]:
|
|
|
76
84
|
```
|
|
77
85
|
|
|
78
86
|
`train` is your own training run. The numbers above are an illustration. A runnable version
|
|
79
|
-
with a simulated `train` is
|
|
80
|
-
[examples/
|
|
87
|
+
with a simulated `train` is
|
|
88
|
+
[examples/quickstart.py](https://github.com/omer-nachman/runlab-python/blob/main/examples/quickstart.py),
|
|
89
|
+
and [examples/curl.sh](https://github.com/omer-nachman/runlab-python/blob/main/examples/curl.sh)
|
|
90
|
+
does the same loop over plain HTTP.
|
|
81
91
|
|
|
82
92
|
## The loop
|
|
83
93
|
|
|
@@ -240,8 +250,9 @@ harmless anyway.
|
|
|
240
250
|
## Client reference
|
|
241
251
|
|
|
242
252
|
`RunLab(url=None, token=None, *, timeout=30, retries=3, backoff_s=0.5)`. Every method returns
|
|
243
|
-
the decoded JSON of one operation in
|
|
244
|
-
|
|
253
|
+
the decoded JSON of one operation in
|
|
254
|
+
[openapi.json](https://github.com/omer-nachman/runlab-python/blob/main/openapi.json).
|
|
255
|
+
With no arguments it uses the key `runlab login` saved (see [Get started](#get-started)).
|
|
245
256
|
|
|
246
257
|
| Method | Operation | Returns |
|
|
247
258
|
|---|---|---|
|
|
@@ -268,12 +279,16 @@ the key `runlab login` saved (see [Get started](#get-started)).
|
|
|
268
279
|
| `runlab whoami [--url URL]` | Who the key signs in as, and where. Exit 1 when not logged in. |
|
|
269
280
|
| `runlab logout` | Revoke the saved key and delete it. If RunLab cannot be reached, the key stays saved and the exit code is 1. |
|
|
270
281
|
| `runlab worker ...` | Run a study's seed pairs here (see above). |
|
|
282
|
+
| `runlab --version` | The client's version. |
|
|
283
|
+
|
|
284
|
+
Each command also runs as `runlab-client ...` and `python -m runlab_client ...`.
|
|
271
285
|
|
|
272
286
|
## Coding agents
|
|
273
287
|
|
|
274
|
-
[AGENTS.md](AGENTS.md)
|
|
275
|
-
|
|
288
|
+
[AGENTS.md](https://github.com/omer-nachman/runlab-python/blob/main/AGENTS.md)
|
|
289
|
+
is the short version for Claude Code, Codex and other agents.
|
|
290
|
+
[llms.txt](https://github.com/omer-nachman/runlab-python/blob/main/llms.txt) indexes this repo for LLM tools.
|
|
276
291
|
|
|
277
292
|
## License
|
|
278
293
|
|
|
279
|
-
MIT ([LICENSE](LICENSE)).
|
|
294
|
+
MIT ([LICENSE](https://github.com/omer-nachman/runlab-python/blob/main/LICENSE)).
|
|
@@ -7,8 +7,13 @@ its baseline. RunLab gives back a verdict you can quote, built from:
|
|
|
7
7
|
- a paired test across seeds, with a 95 % confidence interval;
|
|
8
8
|
- checks that refuse a comparison that would mislead, e.g. arms tuned with unequal budgets.
|
|
9
9
|
|
|
10
|
-
|
|
11
|
-
|
|
10
|
+
The RunLab Python client gives access to the RunLab API using only the standard library,
|
|
11
|
+
on Python 3.10+.
|
|
12
|
+
|
|
13
|
+
## Documentation
|
|
14
|
+
|
|
15
|
+
- [API reference](https://omer-nachman.github.io/runlab-python/api.html)
|
|
16
|
+
- [Examples](https://github.com/omer-nachman/runlab-python/tree/main/examples): Python and curl
|
|
12
17
|
|
|
13
18
|
## Get started
|
|
14
19
|
|
|
@@ -21,6 +26,9 @@ runlab login # sign in with GitHub, approve this termina
|
|
|
21
26
|
runlab whoami # ada (agent) at https://runlab.tools
|
|
22
27
|
```
|
|
23
28
|
|
|
29
|
+
- Another program called `runlab` first on your PATH? Every command here also runs as
|
|
30
|
+
`runlab-client login | whoami | logout | worker` (or `python -m runlab_client ...`), and
|
|
31
|
+
`runlab-client --version` names the version.
|
|
24
32
|
- `runlab login` opens a browser when this machine has a display. On an SSH session or a
|
|
25
33
|
headless GPU box, open the printed link on any device (your laptop or phone).
|
|
26
34
|
- The page asks "Authorize this terminal?" and shows the code, the address and host name
|
|
@@ -64,8 +72,10 @@ for c in rl.report("resnet-aug")["claims"]:
|
|
|
64
72
|
```
|
|
65
73
|
|
|
66
74
|
`train` is your own training run. The numbers above are an illustration. A runnable version
|
|
67
|
-
with a simulated `train` is
|
|
68
|
-
[examples/
|
|
75
|
+
with a simulated `train` is
|
|
76
|
+
[examples/quickstart.py](https://github.com/omer-nachman/runlab-python/blob/main/examples/quickstart.py),
|
|
77
|
+
and [examples/curl.sh](https://github.com/omer-nachman/runlab-python/blob/main/examples/curl.sh)
|
|
78
|
+
does the same loop over plain HTTP.
|
|
69
79
|
|
|
70
80
|
## The loop
|
|
71
81
|
|
|
@@ -228,8 +238,9 @@ harmless anyway.
|
|
|
228
238
|
## Client reference
|
|
229
239
|
|
|
230
240
|
`RunLab(url=None, token=None, *, timeout=30, retries=3, backoff_s=0.5)`. Every method returns
|
|
231
|
-
the decoded JSON of one operation in
|
|
232
|
-
|
|
241
|
+
the decoded JSON of one operation in
|
|
242
|
+
[openapi.json](https://github.com/omer-nachman/runlab-python/blob/main/openapi.json).
|
|
243
|
+
With no arguments it uses the key `runlab login` saved (see [Get started](#get-started)).
|
|
233
244
|
|
|
234
245
|
| Method | Operation | Returns |
|
|
235
246
|
|---|---|---|
|
|
@@ -256,12 +267,16 @@ the key `runlab login` saved (see [Get started](#get-started)).
|
|
|
256
267
|
| `runlab whoami [--url URL]` | Who the key signs in as, and where. Exit 1 when not logged in. |
|
|
257
268
|
| `runlab logout` | Revoke the saved key and delete it. If RunLab cannot be reached, the key stays saved and the exit code is 1. |
|
|
258
269
|
| `runlab worker ...` | Run a study's seed pairs here (see above). |
|
|
270
|
+
| `runlab --version` | The client's version. |
|
|
271
|
+
|
|
272
|
+
Each command also runs as `runlab-client ...` and `python -m runlab_client ...`.
|
|
259
273
|
|
|
260
274
|
## Coding agents
|
|
261
275
|
|
|
262
|
-
[AGENTS.md](AGENTS.md)
|
|
263
|
-
|
|
276
|
+
[AGENTS.md](https://github.com/omer-nachman/runlab-python/blob/main/AGENTS.md)
|
|
277
|
+
is the short version for Claude Code, Codex and other agents.
|
|
278
|
+
[llms.txt](https://github.com/omer-nachman/runlab-python/blob/main/llms.txt) indexes this repo for LLM tools.
|
|
264
279
|
|
|
265
280
|
## License
|
|
266
281
|
|
|
267
|
-
MIT ([LICENSE](LICENSE)).
|
|
282
|
+
MIT ([LICENSE](https://github.com/omer-nachman/runlab-python/blob/main/LICENSE)).
|
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "runlab-client"
|
|
7
|
-
version = "0.2.
|
|
7
|
+
version = "0.2.1"
|
|
8
8
|
description = "Python client for the RunLab research API: pre-registered A/B comparisons over seeds, verdicts with confidence intervals. Standard library only."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.10"
|
|
@@ -14,6 +14,8 @@ dependencies = []
|
|
|
14
14
|
|
|
15
15
|
[project.scripts]
|
|
16
16
|
runlab = "runlab_client:main"
|
|
17
|
+
# the same commands under a name no other `runlab` program on PATH takes
|
|
18
|
+
runlab-client = "runlab_client:main"
|
|
17
19
|
|
|
18
20
|
[project.urls]
|
|
19
21
|
Homepage = "https://github.com/omer-nachman/runlab-python"
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
$ pip install runlab-client
|
|
4
4
|
$ runlab login # sign in with GitHub; the key stays on this machine
|
|
5
|
+
$ runlab-client login # the same, when another `runlab` comes first on PATH
|
|
5
6
|
|
|
6
7
|
from runlab_client import RunLab
|
|
7
8
|
|
|
@@ -50,7 +51,7 @@ from pathlib import Path
|
|
|
50
51
|
from typing import Any
|
|
51
52
|
|
|
52
53
|
__all__ = ["RunLab", "RunLabError", "login", "result_line"]
|
|
53
|
-
__version__ = "0.2.
|
|
54
|
+
__version__ = "0.2.1"
|
|
54
55
|
|
|
55
56
|
DEFAULT_URL = "https://runlab.tools"
|
|
56
57
|
|
|
@@ -580,12 +581,12 @@ def _cmd_login(args: Any) -> int:
|
|
|
580
581
|
def _cmd_whoami(args: Any) -> int:
|
|
581
582
|
rl = RunLab(args.url)
|
|
582
583
|
if not rl.token:
|
|
583
|
-
print(f"not logged in to {rl.url}: run `
|
|
584
|
+
print(f"not logged in to {rl.url}: run `{args.prog} login`", file=sys.stderr)
|
|
584
585
|
return 1
|
|
585
586
|
try:
|
|
586
587
|
me = rl.me()
|
|
587
588
|
except RunLabError as exc:
|
|
588
|
-
hint = ": run `
|
|
589
|
+
hint = f": run `{args.prog} login`" if exc.status == 401 else ""
|
|
589
590
|
print(f"error: {exc}{hint}", file=sys.stderr)
|
|
590
591
|
return 1
|
|
591
592
|
print(f"{me.get('name')} ({me.get('role')}) at {rl.url}")
|
|
@@ -603,7 +604,7 @@ def _cmd_logout(args: Any) -> int:
|
|
|
603
604
|
except RunLabError as exc:
|
|
604
605
|
if exc.status != 401:
|
|
605
606
|
print(f"error: cannot revoke the key at {url}: {exc.message}. It stays saved: run "
|
|
606
|
-
f"`
|
|
607
|
+
f"`{args.prog} logout` again, or delete {config_dir() / 'token'} to forget it "
|
|
607
608
|
"here only", file=sys.stderr)
|
|
608
609
|
return 1
|
|
609
610
|
how = "the key had already stopped working"
|
|
@@ -616,11 +617,18 @@ def _cmd_logout(args: Any) -> int:
|
|
|
616
617
|
|
|
617
618
|
# ---------------------------------------------------------------- command line
|
|
618
619
|
|
|
619
|
-
def main(argv: list[str] | None = None) -> int:
|
|
620
|
-
"""`runlab login | whoami | logout | worker ...` (see the module doc)
|
|
620
|
+
def main(argv: list[str] | None = None, prog: str | None = None) -> int:
|
|
621
|
+
"""`runlab login | whoami | logout | worker ...` (see the module doc); also installed as
|
|
622
|
+
`runlab-client`, and run by `python -m runlab_client`. Every hint names `prog`, the
|
|
623
|
+
command that was run (default: `runlab-client` when run under that name, else `runlab`),
|
|
624
|
+
so following it reaches this client and not another `runlab` on PATH."""
|
|
621
625
|
import argparse
|
|
622
626
|
|
|
623
|
-
|
|
627
|
+
if prog is None:
|
|
628
|
+
prog = "runlab-client" if Path(sys.argv[0]).name.startswith("runlab-client") else "runlab"
|
|
629
|
+
ap = argparse.ArgumentParser(prog=prog, description="RunLab from the command line.")
|
|
630
|
+
ap.add_argument("--version", action="version", version=f"runlab-client {__version__}")
|
|
631
|
+
ap.set_defaults(prog=prog)
|
|
624
632
|
sub = ap.add_subparsers(dest="command", required=True)
|
|
625
633
|
lg = sub.add_parser("login", help="sign this machine in with GitHub",
|
|
626
634
|
description="Print a link and a code, open the link when this machine "
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: runlab-client
|
|
3
|
-
Version: 0.2.
|
|
3
|
+
Version: 0.2.1
|
|
4
4
|
Summary: Python client for the RunLab research API: pre-registered A/B comparisons over seeds, verdicts with confidence intervals. Standard library only.
|
|
5
5
|
Author: omer-nachman
|
|
6
6
|
License-Expression: MIT
|
|
@@ -19,8 +19,13 @@ its baseline. RunLab gives back a verdict you can quote, built from:
|
|
|
19
19
|
- a paired test across seeds, with a 95 % confidence interval;
|
|
20
20
|
- checks that refuse a comparison that would mislead, e.g. arms tuned with unequal budgets.
|
|
21
21
|
|
|
22
|
-
|
|
23
|
-
|
|
22
|
+
The RunLab Python client gives access to the RunLab API using only the standard library,
|
|
23
|
+
on Python 3.10+.
|
|
24
|
+
|
|
25
|
+
## Documentation
|
|
26
|
+
|
|
27
|
+
- [API reference](https://omer-nachman.github.io/runlab-python/api.html)
|
|
28
|
+
- [Examples](https://github.com/omer-nachman/runlab-python/tree/main/examples): Python and curl
|
|
24
29
|
|
|
25
30
|
## Get started
|
|
26
31
|
|
|
@@ -33,6 +38,9 @@ runlab login # sign in with GitHub, approve this termina
|
|
|
33
38
|
runlab whoami # ada (agent) at https://runlab.tools
|
|
34
39
|
```
|
|
35
40
|
|
|
41
|
+
- Another program called `runlab` first on your PATH? Every command here also runs as
|
|
42
|
+
`runlab-client login | whoami | logout | worker` (or `python -m runlab_client ...`), and
|
|
43
|
+
`runlab-client --version` names the version.
|
|
36
44
|
- `runlab login` opens a browser when this machine has a display. On an SSH session or a
|
|
37
45
|
headless GPU box, open the printed link on any device (your laptop or phone).
|
|
38
46
|
- The page asks "Authorize this terminal?" and shows the code, the address and host name
|
|
@@ -76,8 +84,10 @@ for c in rl.report("resnet-aug")["claims"]:
|
|
|
76
84
|
```
|
|
77
85
|
|
|
78
86
|
`train` is your own training run. The numbers above are an illustration. A runnable version
|
|
79
|
-
with a simulated `train` is
|
|
80
|
-
[examples/
|
|
87
|
+
with a simulated `train` is
|
|
88
|
+
[examples/quickstart.py](https://github.com/omer-nachman/runlab-python/blob/main/examples/quickstart.py),
|
|
89
|
+
and [examples/curl.sh](https://github.com/omer-nachman/runlab-python/blob/main/examples/curl.sh)
|
|
90
|
+
does the same loop over plain HTTP.
|
|
81
91
|
|
|
82
92
|
## The loop
|
|
83
93
|
|
|
@@ -240,8 +250,9 @@ harmless anyway.
|
|
|
240
250
|
## Client reference
|
|
241
251
|
|
|
242
252
|
`RunLab(url=None, token=None, *, timeout=30, retries=3, backoff_s=0.5)`. Every method returns
|
|
243
|
-
the decoded JSON of one operation in
|
|
244
|
-
|
|
253
|
+
the decoded JSON of one operation in
|
|
254
|
+
[openapi.json](https://github.com/omer-nachman/runlab-python/blob/main/openapi.json).
|
|
255
|
+
With no arguments it uses the key `runlab login` saved (see [Get started](#get-started)).
|
|
245
256
|
|
|
246
257
|
| Method | Operation | Returns |
|
|
247
258
|
|---|---|---|
|
|
@@ -268,12 +279,16 @@ the key `runlab login` saved (see [Get started](#get-started)).
|
|
|
268
279
|
| `runlab whoami [--url URL]` | Who the key signs in as, and where. Exit 1 when not logged in. |
|
|
269
280
|
| `runlab logout` | Revoke the saved key and delete it. If RunLab cannot be reached, the key stays saved and the exit code is 1. |
|
|
270
281
|
| `runlab worker ...` | Run a study's seed pairs here (see above). |
|
|
282
|
+
| `runlab --version` | The client's version. |
|
|
283
|
+
|
|
284
|
+
Each command also runs as `runlab-client ...` and `python -m runlab_client ...`.
|
|
271
285
|
|
|
272
286
|
## Coding agents
|
|
273
287
|
|
|
274
|
-
[AGENTS.md](AGENTS.md)
|
|
275
|
-
|
|
288
|
+
[AGENTS.md](https://github.com/omer-nachman/runlab-python/blob/main/AGENTS.md)
|
|
289
|
+
is the short version for Claude Code, Codex and other agents.
|
|
290
|
+
[llms.txt](https://github.com/omer-nachman/runlab-python/blob/main/llms.txt) indexes this repo for LLM tools.
|
|
276
291
|
|
|
277
292
|
## License
|
|
278
293
|
|
|
279
|
-
MIT ([LICENSE](LICENSE)).
|
|
294
|
+
MIT ([LICENSE](https://github.com/omer-nachman/runlab-python/blob/main/LICENSE)).
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|