technode-cli 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- technode_cli-0.1.0/LICENSE +21 -0
- technode_cli-0.1.0/PKG-INFO +59 -0
- technode_cli-0.1.0/README.md +45 -0
- technode_cli-0.1.0/pyproject.toml +28 -0
- technode_cli-0.1.0/setup.cfg +4 -0
- technode_cli-0.1.0/technode/__init__.py +3 -0
- technode_cli-0.1.0/technode/__main__.py +6 -0
- technode_cli-0.1.0/technode/cli.py +423 -0
- technode_cli-0.1.0/technode_cli.egg-info/PKG-INFO +59 -0
- technode_cli-0.1.0/technode_cli.egg-info/SOURCES.txt +11 -0
- technode_cli-0.1.0/technode_cli.egg-info/dependency_links.txt +1 -0
- technode_cli-0.1.0/technode_cli.egg-info/entry_points.txt +2 -0
- technode_cli-0.1.0/technode_cli.egg-info/top_level.txt +1 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 TechNode
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: technode-cli
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: TechNode CLI — run inference on TechNode's compressed open-model GPU grid (RunPod-compatible).
|
|
5
|
+
Author-email: TechNode <smlee3636@gmail.com>
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://technode.network
|
|
8
|
+
Project-URL: API docs, https://technode.network/developers
|
|
9
|
+
Keywords: llm,inference,gpu,runpod,quantized,technode
|
|
10
|
+
Requires-Python: >=3.8
|
|
11
|
+
Description-Content-Type: text/markdown
|
|
12
|
+
License-File: LICENSE
|
|
13
|
+
Dynamic: license-file
|
|
14
|
+
|
|
15
|
+
# technode
|
|
16
|
+
|
|
17
|
+
Run inference on **TechNode** — a GPU grid serving *compressed* open models
|
|
18
|
+
(Qwen, Granite, gpt-oss, Devstral, Gemma, EXAONE…) at consumer-GPU prices.
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
pip install technode-cli # the command is `technode`
|
|
22
|
+
technode login # paste your tn_test_… key (or set TECHNODE_API_KEY)
|
|
23
|
+
technode models # list the compressed catalog
|
|
24
|
+
technode infer "Explain quantization in one line." --model qwen2.5-7b
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Zero dependencies — pure Python stdlib, runs anywhere Python ≥3.8 does.
|
|
28
|
+
|
|
29
|
+
## Commands
|
|
30
|
+
|
|
31
|
+
| Command | What it does |
|
|
32
|
+
|---|---|
|
|
33
|
+
| `technode login [key]` | Save your API key to `~/.technode/config.json` (chmod 600). |
|
|
34
|
+
| `technode logout` | Remove the saved key. |
|
|
35
|
+
| `technode models [--json]` | List available models (id, quantization, role). |
|
|
36
|
+
| `technode infer PROMPT [-m MODEL] [-n MAX_TOKENS] [-t TEMP] [--json] [-q]` | Text generation. `-` or piped stdin reads the prompt from stdin. |
|
|
37
|
+
| `technode whoami` | Show the active key (masked) + endpoint. |
|
|
38
|
+
|
|
39
|
+
## Configuration
|
|
40
|
+
|
|
41
|
+
| Setting | Env var | Default |
|
|
42
|
+
|---|---|---|
|
|
43
|
+
| API key | `TECHNODE_API_KEY` | — (from `technode login`) |
|
|
44
|
+
| Endpoint | `TECHNODE_BASE_URL` | `https://technode.network` |
|
|
45
|
+
|
|
46
|
+
Get a key (free beta): <https://technode.network/developers>
|
|
47
|
+
|
|
48
|
+
## Examples
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
# pick a coder model
|
|
52
|
+
technode infer "Write a Python one-liner to flatten a list of lists." -m qwen2.5-coder-7b
|
|
53
|
+
|
|
54
|
+
# read the prompt from a file / pipe
|
|
55
|
+
cat prompt.txt | technode infer -
|
|
56
|
+
|
|
57
|
+
# machine-readable
|
|
58
|
+
technode infer "hi" --json
|
|
59
|
+
```
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# technode
|
|
2
|
+
|
|
3
|
+
Run inference on **TechNode** — a GPU grid serving *compressed* open models
|
|
4
|
+
(Qwen, Granite, gpt-oss, Devstral, Gemma, EXAONE…) at consumer-GPU prices.
|
|
5
|
+
|
|
6
|
+
```bash
|
|
7
|
+
pip install technode-cli # the command is `technode`
|
|
8
|
+
technode login # paste your tn_test_… key (or set TECHNODE_API_KEY)
|
|
9
|
+
technode models # list the compressed catalog
|
|
10
|
+
technode infer "Explain quantization in one line." --model qwen2.5-7b
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Zero dependencies — pure Python stdlib, runs anywhere Python ≥3.8 does.
|
|
14
|
+
|
|
15
|
+
## Commands
|
|
16
|
+
|
|
17
|
+
| Command | What it does |
|
|
18
|
+
|---|---|
|
|
19
|
+
| `technode login [key]` | Save your API key to `~/.technode/config.json` (chmod 600). |
|
|
20
|
+
| `technode logout` | Remove the saved key. |
|
|
21
|
+
| `technode models [--json]` | List available models (id, quantization, role). |
|
|
22
|
+
| `technode infer PROMPT [-m MODEL] [-n MAX_TOKENS] [-t TEMP] [--json] [-q]` | Text generation. `-` or piped stdin reads the prompt from stdin. |
|
|
23
|
+
| `technode whoami` | Show the active key (masked) + endpoint. |
|
|
24
|
+
|
|
25
|
+
## Configuration
|
|
26
|
+
|
|
27
|
+
| Setting | Env var | Default |
|
|
28
|
+
|---|---|---|
|
|
29
|
+
| API key | `TECHNODE_API_KEY` | — (from `technode login`) |
|
|
30
|
+
| Endpoint | `TECHNODE_BASE_URL` | `https://technode.network` |
|
|
31
|
+
|
|
32
|
+
Get a key (free beta): <https://technode.network/developers>
|
|
33
|
+
|
|
34
|
+
## Examples
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
# pick a coder model
|
|
38
|
+
technode infer "Write a Python one-liner to flatten a list of lists." -m qwen2.5-coder-7b
|
|
39
|
+
|
|
40
|
+
# read the prompt from a file / pipe
|
|
41
|
+
cat prompt.txt | technode infer -
|
|
42
|
+
|
|
43
|
+
# machine-readable
|
|
44
|
+
technode infer "hi" --json
|
|
45
|
+
```
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=61"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
# Distribution name on PyPI. The console command is still `technode`
|
|
7
|
+
# (see [project.scripts]); PyPI disallows the bare name "technode".
|
|
8
|
+
name = "technode-cli"
|
|
9
|
+
version = "0.1.0"
|
|
10
|
+
description = "TechNode CLI — run inference on TechNode's compressed open-model GPU grid (RunPod-compatible)."
|
|
11
|
+
readme = "README.md"
|
|
12
|
+
requires-python = ">=3.8"
|
|
13
|
+
license = { text = "MIT" }
|
|
14
|
+
authors = [{ name = "TechNode", email = "smlee3636@gmail.com" }]
|
|
15
|
+
keywords = ["llm", "inference", "gpu", "runpod", "quantized", "technode"]
|
|
16
|
+
# Zero runtime dependencies — pure stdlib (urllib). Keeps `pip install technode`
|
|
17
|
+
# light and lets the same client run inside minimal/embedded Python.
|
|
18
|
+
dependencies = []
|
|
19
|
+
|
|
20
|
+
[project.urls]
|
|
21
|
+
Homepage = "https://technode.network"
|
|
22
|
+
"API docs" = "https://technode.network/developers"
|
|
23
|
+
|
|
24
|
+
[project.scripts]
|
|
25
|
+
technode = "technode.cli:main"
|
|
26
|
+
|
|
27
|
+
[tool.setuptools]
|
|
28
|
+
packages = ["technode"]
|
|
@@ -0,0 +1,423 @@
|
|
|
1
|
+
"""TechNode CLI.
|
|
2
|
+
|
|
3
|
+
A tiny, dependency-free client for TechNode's compressed open-model GPU grid.
|
|
4
|
+
|
|
5
|
+
pip install technode
|
|
6
|
+
technode login # paste your tn_test_... key (or set TECHNODE_API_KEY)
|
|
7
|
+
technode models # list the compressed catalog
|
|
8
|
+
technode infer "Explain quantization in one line." --model qwen2.5-7b
|
|
9
|
+
|
|
10
|
+
Design notes
|
|
11
|
+
------------
|
|
12
|
+
* Pure stdlib (urllib) — no third-party deps, so it runs anywhere Python does.
|
|
13
|
+
* The authenticated path goes through the website API (https://technode.network),
|
|
14
|
+
which validates the key + charges credit server-side and forwards to the GPU
|
|
15
|
+
broker. Secrets never touch the CLI beyond the user's own key.
|
|
16
|
+
* The model catalog is public, so `models` reads it without auth.
|
|
17
|
+
"""
|
|
18
|
+
|
|
19
|
+
import argparse
|
|
20
|
+
import getpass
|
|
21
|
+
import json
|
|
22
|
+
import os
|
|
23
|
+
import sys
|
|
24
|
+
import time
|
|
25
|
+
import urllib.error
|
|
26
|
+
import urllib.request
|
|
27
|
+
|
|
28
|
+
from . import __version__
|
|
29
|
+
|
|
30
|
+
DEFAULT_BASE_URL = "https://technode.network"
|
|
31
|
+
# Public, unauthenticated catalog (read-only). Used as a fallback if the website
|
|
32
|
+
# proxy (/api/models) is unavailable.
|
|
33
|
+
CATALOG_FALLBACK_URL = "https://broker.technode.network/models/available"
|
|
34
|
+
CONFIG_DIR = os.path.join(os.path.expanduser("~"), ".technode")
|
|
35
|
+
CONFIG_PATH = os.path.join(CONFIG_DIR, "config.json")
|
|
36
|
+
USER_AGENT = "technode-cli/" + __version__
|
|
37
|
+
|
|
38
|
+
|
|
39
|
+
# --------------------------------------------------------------------------- #
|
|
40
|
+
# config
|
|
41
|
+
# --------------------------------------------------------------------------- #
|
|
42
|
+
def _load_config() -> dict:
|
|
43
|
+
try:
|
|
44
|
+
with open(CONFIG_PATH, "r", encoding="utf-8") as fh:
|
|
45
|
+
data = json.load(fh)
|
|
46
|
+
return data if isinstance(data, dict) else {}
|
|
47
|
+
except (FileNotFoundError, ValueError, OSError):
|
|
48
|
+
return {}
|
|
49
|
+
|
|
50
|
+
|
|
51
|
+
def _save_config(cfg: dict) -> None:
|
|
52
|
+
os.makedirs(CONFIG_DIR, exist_ok=True)
|
|
53
|
+
tmp = CONFIG_PATH + ".tmp"
|
|
54
|
+
with open(tmp, "w", encoding="utf-8") as fh:
|
|
55
|
+
json.dump(cfg, fh, indent=2)
|
|
56
|
+
os.replace(tmp, CONFIG_PATH)
|
|
57
|
+
try:
|
|
58
|
+
os.chmod(CONFIG_PATH, 0o600) # the key is a secret; lock the file down
|
|
59
|
+
except OSError:
|
|
60
|
+
pass
|
|
61
|
+
|
|
62
|
+
|
|
63
|
+
def _api_key() -> str:
|
|
64
|
+
"""Resolve the API key: env wins, then the saved config."""
|
|
65
|
+
return (os.environ.get("TECHNODE_API_KEY", "").strip()
|
|
66
|
+
or str(_load_config().get("api_key", "")).strip())
|
|
67
|
+
|
|
68
|
+
|
|
69
|
+
def _base_url() -> str:
|
|
70
|
+
return (os.environ.get("TECHNODE_BASE_URL", "").strip()
|
|
71
|
+
or str(_load_config().get("base_url", "")).strip()
|
|
72
|
+
or DEFAULT_BASE_URL).rstrip("/")
|
|
73
|
+
|
|
74
|
+
|
|
75
|
+
# --------------------------------------------------------------------------- #
|
|
76
|
+
# http
|
|
77
|
+
# --------------------------------------------------------------------------- #
|
|
78
|
+
class ApiError(Exception):
|
|
79
|
+
def __init__(self, status: int, payload):
|
|
80
|
+
self.status = status
|
|
81
|
+
self.payload = payload
|
|
82
|
+
super().__init__(f"HTTP {status}: {payload}")
|
|
83
|
+
|
|
84
|
+
|
|
85
|
+
def _request(method: str, url: str, *, token: str = "", body=None, timeout: float = 70.0):
|
|
86
|
+
data = None
|
|
87
|
+
headers = {"User-Agent": USER_AGENT, "Accept": "application/json"}
|
|
88
|
+
if body is not None:
|
|
89
|
+
data = json.dumps(body).encode("utf-8")
|
|
90
|
+
headers["Content-Type"] = "application/json"
|
|
91
|
+
if token:
|
|
92
|
+
headers["Authorization"] = "Bearer " + token
|
|
93
|
+
req = urllib.request.Request(url, data=data, method=method, headers=headers)
|
|
94
|
+
try:
|
|
95
|
+
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
|
96
|
+
raw = resp.read().decode("utf-8") or "{}"
|
|
97
|
+
return json.loads(raw)
|
|
98
|
+
except urllib.error.HTTPError as exc:
|
|
99
|
+
raw = exc.read().decode("utf-8", "replace")
|
|
100
|
+
try:
|
|
101
|
+
payload = json.loads(raw)
|
|
102
|
+
except ValueError:
|
|
103
|
+
payload = {"error": raw[:400] or exc.reason}
|
|
104
|
+
raise ApiError(exc.code, payload)
|
|
105
|
+
except urllib.error.URLError as exc:
|
|
106
|
+
raise ApiError(0, {"error": "network error", "reason": str(exc.reason)})
|
|
107
|
+
|
|
108
|
+
|
|
109
|
+
def _die(msg: str, code: int = 1):
|
|
110
|
+
print("technode: " + msg, file=sys.stderr)
|
|
111
|
+
raise SystemExit(code)
|
|
112
|
+
|
|
113
|
+
|
|
114
|
+
def _require_key() -> str:
|
|
115
|
+
key = _api_key()
|
|
116
|
+
if not key:
|
|
117
|
+
_die("no API key. Run `technode login` or set TECHNODE_API_KEY.\n"
|
|
118
|
+
" Get a key at https://technode.network/developers")
|
|
119
|
+
return key
|
|
120
|
+
|
|
121
|
+
|
|
122
|
+
# --------------------------------------------------------------------------- #
|
|
123
|
+
# commands
|
|
124
|
+
# --------------------------------------------------------------------------- #
|
|
125
|
+
def cmd_login(args) -> int:
|
|
126
|
+
key = (args.key or "").strip()
|
|
127
|
+
if not key:
|
|
128
|
+
try:
|
|
129
|
+
key = getpass.getpass("TechNode API key (tn_test_… / tn_live_…): ").strip()
|
|
130
|
+
except (EOFError, KeyboardInterrupt):
|
|
131
|
+
print()
|
|
132
|
+
return 1
|
|
133
|
+
if not key:
|
|
134
|
+
_die("no key provided.")
|
|
135
|
+
if not (key.startswith("tn_test_") or key.startswith("tn_live_")):
|
|
136
|
+
print("technode: warning — key does not start with tn_test_/tn_live_; saving anyway.",
|
|
137
|
+
file=sys.stderr)
|
|
138
|
+
cfg = _load_config()
|
|
139
|
+
cfg["api_key"] = key
|
|
140
|
+
_save_config(cfg)
|
|
141
|
+
print(f"Saved key to {CONFIG_PATH} (chmod 600).")
|
|
142
|
+
print("Try: technode models then technode infer \"hello\"")
|
|
143
|
+
return 0
|
|
144
|
+
|
|
145
|
+
|
|
146
|
+
def cmd_logout(args) -> int:
|
|
147
|
+
cfg = _load_config()
|
|
148
|
+
if "api_key" in cfg:
|
|
149
|
+
del cfg["api_key"]
|
|
150
|
+
_save_config(cfg)
|
|
151
|
+
print("Removed saved API key.")
|
|
152
|
+
else:
|
|
153
|
+
print("No saved API key.")
|
|
154
|
+
return 0
|
|
155
|
+
|
|
156
|
+
|
|
157
|
+
def cmd_models(args) -> int:
|
|
158
|
+
base = _base_url()
|
|
159
|
+
data = None
|
|
160
|
+
for url in (base + "/api/models", CATALOG_FALLBACK_URL):
|
|
161
|
+
try:
|
|
162
|
+
data = _request("GET", url, timeout=15)
|
|
163
|
+
break
|
|
164
|
+
except ApiError:
|
|
165
|
+
continue
|
|
166
|
+
if data is None:
|
|
167
|
+
_die("could not reach the model catalog.")
|
|
168
|
+
models = data.get("models", data) if isinstance(data, dict) else data
|
|
169
|
+
if not models:
|
|
170
|
+
print("No models currently available.")
|
|
171
|
+
return 0
|
|
172
|
+
if args.json:
|
|
173
|
+
print(json.dumps(models, indent=2, ensure_ascii=False))
|
|
174
|
+
return 0
|
|
175
|
+
print(f"{'MODEL':22} {'QUANT':8} {'ROLE':22} {'STATUS':10} AVAIL")
|
|
176
|
+
print("-" * 72)
|
|
177
|
+
for m in models:
|
|
178
|
+
role = m.get("role")
|
|
179
|
+
role = ",".join(role) if isinstance(role, list) else str(role or "")
|
|
180
|
+
avail = "yes" if m.get("available", m.get("serving")) else "no"
|
|
181
|
+
warm = " (warm)" if m.get("warm") else ""
|
|
182
|
+
print(f"{str(m.get('id','')):22} {str(m.get('quant','')):8} "
|
|
183
|
+
f"{role[:22]:22} {str(m.get('status','')):10} {avail}{warm}")
|
|
184
|
+
return 0
|
|
185
|
+
|
|
186
|
+
|
|
187
|
+
def cmd_infer(args) -> int:
|
|
188
|
+
key = _require_key()
|
|
189
|
+
prompt = args.prompt
|
|
190
|
+
if prompt == "-" or (prompt is None and not sys.stdin.isatty()):
|
|
191
|
+
prompt = sys.stdin.read()
|
|
192
|
+
if not prompt or not prompt.strip():
|
|
193
|
+
_die("empty prompt.")
|
|
194
|
+
body = {"prompt": prompt}
|
|
195
|
+
if args.model:
|
|
196
|
+
body["model"] = args.model
|
|
197
|
+
if args.max_tokens is not None:
|
|
198
|
+
body["max_new_tokens"] = args.max_tokens
|
|
199
|
+
if args.temperature is not None:
|
|
200
|
+
body["temperature"] = args.temperature
|
|
201
|
+
url = _base_url() + "/api/dispatch"
|
|
202
|
+
t0 = time.time()
|
|
203
|
+
try:
|
|
204
|
+
res = _request("POST", url, token=key, body=body)
|
|
205
|
+
except ApiError as exc:
|
|
206
|
+
return _handle_api_error(exc)
|
|
207
|
+
if args.json:
|
|
208
|
+
print(json.dumps(res, indent=2, ensure_ascii=False))
|
|
209
|
+
return 0
|
|
210
|
+
text = res.get("generated") or res.get("text") or ""
|
|
211
|
+
print(text.rstrip("\n"))
|
|
212
|
+
if not args.quiet:
|
|
213
|
+
ms = res.get("e2e_ms") or int((time.time() - t0) * 1000)
|
|
214
|
+
meta = [f"model={res.get('model', args.model or '?')}"]
|
|
215
|
+
if res.get("device"):
|
|
216
|
+
meta.append(f"device={res['device']}")
|
|
217
|
+
meta.append(f"{ms}ms")
|
|
218
|
+
if res.get("cost_krw") is not None:
|
|
219
|
+
meta.append(f"cost=₩{res['cost_krw']}")
|
|
220
|
+
if res.get("balance_after_krw") is not None:
|
|
221
|
+
meta.append(f"balance=₩{res['balance_after_krw']}")
|
|
222
|
+
print(" · " + " ".join(meta), file=sys.stderr)
|
|
223
|
+
return 0
|
|
224
|
+
|
|
225
|
+
|
|
226
|
+
def _handle_api_error(exc: ApiError) -> int:
|
|
227
|
+
p = exc.payload if isinstance(exc.payload, dict) else {"error": exc.payload}
|
|
228
|
+
err = p.get("error", "request failed")
|
|
229
|
+
reason = p.get("reason") or p.get("hint")
|
|
230
|
+
if exc.status == 401:
|
|
231
|
+
_die(f"unauthorized — {reason or err}.\n"
|
|
232
|
+
" Check your key: technode login")
|
|
233
|
+
if exc.status == 402:
|
|
234
|
+
_die(f"out of credit — {p.get('hint') or err}.\n"
|
|
235
|
+
" Top up at https://technode.network/app/customer")
|
|
236
|
+
if exc.status == 0:
|
|
237
|
+
_die(f"network error — {reason or err}.")
|
|
238
|
+
msg = f"{err}" + (f" ({reason})" if reason else "")
|
|
239
|
+
_die(f"server returned {exc.status}: {msg}")
|
|
240
|
+
return 1 # unreachable
|
|
241
|
+
|
|
242
|
+
|
|
243
|
+
def _fmt_lease(res: dict) -> None:
|
|
244
|
+
rid = res.get("lease_id", "?")
|
|
245
|
+
gpu = res.get("gpu_name") or res.get("provider_id") or "?"
|
|
246
|
+
jhref = res.get("lab_join_url") or res.get("lab_url") or ""
|
|
247
|
+
token = res.get("lab_token") or ""
|
|
248
|
+
rem = res.get("remaining_s")
|
|
249
|
+
print("GPU leased ✓")
|
|
250
|
+
print(f" lease id: {rid}")
|
|
251
|
+
print(f" gpu: {gpu}")
|
|
252
|
+
if jhref:
|
|
253
|
+
print(f" jupyter: {jhref}")
|
|
254
|
+
if token:
|
|
255
|
+
print(f" token: {token}")
|
|
256
|
+
if rem is not None:
|
|
257
|
+
print(f" expires: in {int(rem) // 60} min")
|
|
258
|
+
print(f"\nStatus: technode gpu status {rid}")
|
|
259
|
+
print(f"Release: technode gpu release {rid}")
|
|
260
|
+
|
|
261
|
+
|
|
262
|
+
def cmd_gpu_lease(args) -> int:
|
|
263
|
+
key = _require_key()
|
|
264
|
+
body = {}
|
|
265
|
+
if args.gpu:
|
|
266
|
+
body["gpu_id"] = args.gpu
|
|
267
|
+
try:
|
|
268
|
+
res = _request("POST", _base_url() + "/api/lease", token=key, body=body, timeout=30)
|
|
269
|
+
except ApiError as exc:
|
|
270
|
+
if exc.status == 409:
|
|
271
|
+
p = exc.payload if isinstance(exc.payload, dict) else {}
|
|
272
|
+
_die(f"no free GPU slot right now — {p.get('error', 'all slots in use')}.\n"
|
|
273
|
+
" Try again shortly (`technode gpu list` shows active leases).")
|
|
274
|
+
return _handle_api_error(exc)
|
|
275
|
+
if args.json:
|
|
276
|
+
print(json.dumps(res, indent=2, ensure_ascii=False))
|
|
277
|
+
return 0
|
|
278
|
+
_fmt_lease(res)
|
|
279
|
+
return 0
|
|
280
|
+
|
|
281
|
+
|
|
282
|
+
def cmd_gpu_status(args) -> int:
|
|
283
|
+
try:
|
|
284
|
+
res = _request("GET", _base_url() + "/api/lease?id=" + args.lease_id, timeout=15)
|
|
285
|
+
except ApiError as exc:
|
|
286
|
+
return _handle_api_error(exc)
|
|
287
|
+
if args.json:
|
|
288
|
+
print(json.dumps(res, indent=2, ensure_ascii=False))
|
|
289
|
+
return 0
|
|
290
|
+
rem = res.get("remaining_s")
|
|
291
|
+
state = res.get("status") or ("active" if rem else "unknown")
|
|
292
|
+
print(f"lease {args.lease_id}: {state}"
|
|
293
|
+
+ (f" · {int(rem) // 60} min left" if rem is not None else ""))
|
|
294
|
+
return 0
|
|
295
|
+
|
|
296
|
+
|
|
297
|
+
def cmd_gpu_release(args) -> int:
|
|
298
|
+
try:
|
|
299
|
+
res = _request("DELETE", _base_url() + "/api/lease?id=" + args.lease_id, timeout=15)
|
|
300
|
+
except ApiError as exc:
|
|
301
|
+
return _handle_api_error(exc)
|
|
302
|
+
if args.json:
|
|
303
|
+
print(json.dumps(res, indent=2, ensure_ascii=False))
|
|
304
|
+
return 0
|
|
305
|
+
print(f"Released lease {args.lease_id}.")
|
|
306
|
+
return 0
|
|
307
|
+
|
|
308
|
+
|
|
309
|
+
def cmd_gpu_list(args) -> int:
|
|
310
|
+
try:
|
|
311
|
+
res = _request("GET", _base_url() + "/api/leases", timeout=15)
|
|
312
|
+
except ApiError as exc:
|
|
313
|
+
return _handle_api_error(exc)
|
|
314
|
+
leases = res.get("leases", res) if isinstance(res, dict) else res
|
|
315
|
+
if args.json:
|
|
316
|
+
print(json.dumps(leases, indent=2, ensure_ascii=False))
|
|
317
|
+
return 0
|
|
318
|
+
if not leases:
|
|
319
|
+
print("No active GPU leases.")
|
|
320
|
+
return 0
|
|
321
|
+
print(f"{'LEASE ID':40} {'GPU':20} REMAINING")
|
|
322
|
+
print("-" * 72)
|
|
323
|
+
for l in leases:
|
|
324
|
+
rem = l.get("remaining_s")
|
|
325
|
+
print(f"{str(l.get('lease_id','')):40} {str(l.get('gpu_name','')):20} "
|
|
326
|
+
+ (f"{int(rem) // 60} min" if rem is not None else "?"))
|
|
327
|
+
return 0
|
|
328
|
+
|
|
329
|
+
|
|
330
|
+
def cmd_whoami(args) -> int:
|
|
331
|
+
key = _api_key()
|
|
332
|
+
base = _base_url()
|
|
333
|
+
if not key:
|
|
334
|
+
print("Not logged in. Run `technode login`.")
|
|
335
|
+
print(f"Endpoint: {base}")
|
|
336
|
+
return 1
|
|
337
|
+
masked = key[:12] + "…" + key[-4:] if len(key) > 18 else key
|
|
338
|
+
src = "env TECHNODE_API_KEY" if os.environ.get("TECHNODE_API_KEY") else CONFIG_PATH
|
|
339
|
+
print(f"Key: {masked} (from {src})")
|
|
340
|
+
print(f"Endpoint: {base}")
|
|
341
|
+
return 0
|
|
342
|
+
|
|
343
|
+
|
|
344
|
+
# --------------------------------------------------------------------------- #
|
|
345
|
+
# parser
|
|
346
|
+
# --------------------------------------------------------------------------- #
|
|
347
|
+
def build_parser() -> argparse.ArgumentParser:
|
|
348
|
+
p = argparse.ArgumentParser(
|
|
349
|
+
prog="technode",
|
|
350
|
+
description="Run inference on TechNode's compressed open-model GPU grid.",
|
|
351
|
+
)
|
|
352
|
+
p.add_argument("--version", action="version",
|
|
353
|
+
version=f"technode {__version__}")
|
|
354
|
+
sub = p.add_subparsers(dest="command")
|
|
355
|
+
|
|
356
|
+
sp = sub.add_parser("login", help="save your API key locally")
|
|
357
|
+
sp.add_argument("key", nargs="?", help="tn_test_… key (prompted if omitted)")
|
|
358
|
+
sp.set_defaults(func=cmd_login)
|
|
359
|
+
|
|
360
|
+
sp = sub.add_parser("logout", help="remove the saved API key")
|
|
361
|
+
sp.set_defaults(func=cmd_logout)
|
|
362
|
+
|
|
363
|
+
sp = sub.add_parser("models", help="list the available compressed models")
|
|
364
|
+
sp.add_argument("--json", action="store_true", help="raw JSON output")
|
|
365
|
+
sp.set_defaults(func=cmd_models)
|
|
366
|
+
|
|
367
|
+
sp = sub.add_parser("infer", help="run a text generation request")
|
|
368
|
+
sp.add_argument("prompt", nargs="?", help="prompt text ('-' or piped stdin to read stdin)")
|
|
369
|
+
sp.add_argument("-m", "--model", help="model id (see `technode models`)")
|
|
370
|
+
sp.add_argument("-n", "--max-tokens", type=int, dest="max_tokens",
|
|
371
|
+
help="max new tokens")
|
|
372
|
+
sp.add_argument("-t", "--temperature", type=float, help="sampling temperature")
|
|
373
|
+
sp.add_argument("--json", action="store_true", help="raw JSON output")
|
|
374
|
+
sp.add_argument("-q", "--quiet", action="store_true",
|
|
375
|
+
help="suppress the timing/cost footer")
|
|
376
|
+
sp.set_defaults(func=cmd_infer)
|
|
377
|
+
|
|
378
|
+
sp = sub.add_parser("whoami", help="show the active key + endpoint")
|
|
379
|
+
sp.set_defaults(func=cmd_whoami)
|
|
380
|
+
|
|
381
|
+
# gpu: rent a whole GPU (Jupyter session), separate from `infer`.
|
|
382
|
+
gpu = sub.add_parser("gpu", help="rent a GPU (Jupyter lab session)")
|
|
383
|
+
gsub = gpu.add_subparsers(dest="gpu_command")
|
|
384
|
+
|
|
385
|
+
g = gsub.add_parser("lease", help="lease a GPU slot (~60 min)")
|
|
386
|
+
g.add_argument("--gpu", help="GPU id hint (optional)")
|
|
387
|
+
g.add_argument("--json", action="store_true", help="raw JSON output")
|
|
388
|
+
g.set_defaults(func=cmd_gpu_lease)
|
|
389
|
+
|
|
390
|
+
g = gsub.add_parser("status", help="check a lease")
|
|
391
|
+
g.add_argument("lease_id")
|
|
392
|
+
g.add_argument("--json", action="store_true", help="raw JSON output")
|
|
393
|
+
g.set_defaults(func=cmd_gpu_status)
|
|
394
|
+
|
|
395
|
+
g = gsub.add_parser("release", help="release a lease early")
|
|
396
|
+
g.add_argument("lease_id")
|
|
397
|
+
g.add_argument("--json", action="store_true", help="raw JSON output")
|
|
398
|
+
g.set_defaults(func=cmd_gpu_release)
|
|
399
|
+
|
|
400
|
+
g = gsub.add_parser("list", help="list active GPU leases")
|
|
401
|
+
g.add_argument("--json", action="store_true", help="raw JSON output")
|
|
402
|
+
g.set_defaults(func=cmd_gpu_list)
|
|
403
|
+
|
|
404
|
+
gpu.set_defaults(func=lambda a: (gpu.print_help() or 0))
|
|
405
|
+
|
|
406
|
+
return p
|
|
407
|
+
|
|
408
|
+
|
|
409
|
+
def main(argv=None) -> int:
|
|
410
|
+
parser = build_parser()
|
|
411
|
+
args = parser.parse_args(argv)
|
|
412
|
+
if not getattr(args, "command", None):
|
|
413
|
+
parser.print_help()
|
|
414
|
+
return 0
|
|
415
|
+
try:
|
|
416
|
+
return args.func(args)
|
|
417
|
+
except KeyboardInterrupt:
|
|
418
|
+
print()
|
|
419
|
+
return 130
|
|
420
|
+
|
|
421
|
+
|
|
422
|
+
if __name__ == "__main__":
|
|
423
|
+
raise SystemExit(main())
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: technode-cli
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: TechNode CLI — run inference on TechNode's compressed open-model GPU grid (RunPod-compatible).
|
|
5
|
+
Author-email: TechNode <smlee3636@gmail.com>
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://technode.network
|
|
8
|
+
Project-URL: API docs, https://technode.network/developers
|
|
9
|
+
Keywords: llm,inference,gpu,runpod,quantized,technode
|
|
10
|
+
Requires-Python: >=3.8
|
|
11
|
+
Description-Content-Type: text/markdown
|
|
12
|
+
License-File: LICENSE
|
|
13
|
+
Dynamic: license-file
|
|
14
|
+
|
|
15
|
+
# technode
|
|
16
|
+
|
|
17
|
+
Run inference on **TechNode** — a GPU grid serving *compressed* open models
|
|
18
|
+
(Qwen, Granite, gpt-oss, Devstral, Gemma, EXAONE…) at consumer-GPU prices.
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
pip install technode-cli # the command is `technode`
|
|
22
|
+
technode login # paste your tn_test_… key (or set TECHNODE_API_KEY)
|
|
23
|
+
technode models # list the compressed catalog
|
|
24
|
+
technode infer "Explain quantization in one line." --model qwen2.5-7b
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Zero dependencies — pure Python stdlib, runs anywhere Python ≥3.8 does.
|
|
28
|
+
|
|
29
|
+
## Commands
|
|
30
|
+
|
|
31
|
+
| Command | What it does |
|
|
32
|
+
|---|---|
|
|
33
|
+
| `technode login [key]` | Save your API key to `~/.technode/config.json` (chmod 600). |
|
|
34
|
+
| `technode logout` | Remove the saved key. |
|
|
35
|
+
| `technode models [--json]` | List available models (id, quantization, role). |
|
|
36
|
+
| `technode infer PROMPT [-m MODEL] [-n MAX_TOKENS] [-t TEMP] [--json] [-q]` | Text generation. `-` or piped stdin reads the prompt from stdin. |
|
|
37
|
+
| `technode whoami` | Show the active key (masked) + endpoint. |
|
|
38
|
+
|
|
39
|
+
## Configuration
|
|
40
|
+
|
|
41
|
+
| Setting | Env var | Default |
|
|
42
|
+
|---|---|---|
|
|
43
|
+
| API key | `TECHNODE_API_KEY` | — (from `technode login`) |
|
|
44
|
+
| Endpoint | `TECHNODE_BASE_URL` | `https://technode.network` |
|
|
45
|
+
|
|
46
|
+
Get a key (free beta): <https://technode.network/developers>
|
|
47
|
+
|
|
48
|
+
## Examples
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
# pick a coder model
|
|
52
|
+
technode infer "Write a Python one-liner to flatten a list of lists." -m qwen2.5-coder-7b
|
|
53
|
+
|
|
54
|
+
# read the prompt from a file / pipe
|
|
55
|
+
cat prompt.txt | technode infer -
|
|
56
|
+
|
|
57
|
+
# machine-readable
|
|
58
|
+
technode infer "hi" --json
|
|
59
|
+
```
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
LICENSE
|
|
2
|
+
README.md
|
|
3
|
+
pyproject.toml
|
|
4
|
+
technode/__init__.py
|
|
5
|
+
technode/__main__.py
|
|
6
|
+
technode/cli.py
|
|
7
|
+
technode_cli.egg-info/PKG-INFO
|
|
8
|
+
technode_cli.egg-info/SOURCES.txt
|
|
9
|
+
technode_cli.egg-info/dependency_links.txt
|
|
10
|
+
technode_cli.egg-info/entry_points.txt
|
|
11
|
+
technode_cli.egg-info/top_level.txt
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
technode
|