kalavai-client 0.5.19__tar.gz → 0.5.20__tar.gz

{kalavai_client-0.5.19 → kalavai_client-0.5.20}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: kalavai-client
-Version: 0.5.19
+Version: 0.5.20
 Summary: Client app for kalavai platform
 License: Apache-2.0
 Keywords: LLM,platform
@@ -71,6 +71,19 @@ Description-Content-Type: text/markdown
 
 Kalavai's goal is to make using LLMs in real applications accessible and affordable to all. It's a _magic box_ that **integrates all the components required to make LLM useful in the age of massive computing**, from sourcing computing power, managing distributed infrastructure and storage, using industry-standard model engines and orchestration of LLMs. 
 
+### Core features
+
+- Manage **multiple devices resources as one**. One pool of RAM, CPUs and GPUs
+- **Deploy Large Language Models seamlessly across devices**, wherever they are (multiple clouds, on premises, personal devices)
+- Auto-discovery: all **models are automatically exposed** through a single OpenAI-like API and a ChatGPT-like UI playground
+- Compatible with [most popular model engines](#support-for-llm-engines)
+- [Easy to expand](https://github.com/kalavai-net/kube-watcher/tree/main/templates) to custom workloads
+
+
+<details>
+
+**<summary>Video tutorials</summary>**
+
 ### Aggregate multiple devices in an LLM pool
 
 https://github.com/user-attachments/assets/4be59886-1b76-4400-ab5c-c803e3e414ec
@@ -88,12 +101,16 @@ https://github.com/user-attachments/assets/7df73bbc-d129-46aa-8ce5-0735177dedeb
 https://github.com/user-attachments/assets/0d2316f3-79ea-46ac-b41e-8ef720f52672
 
 
-### News updates
+</details>
 
-<img src="docs/docs/assets/images/DeepSeek-Emblem.png" width="100">
+### Latest updates
 
+- 20 February 2025: New shiny GUI interface to control LLM pools and deploy models
 - 6 February 2025: 🔥🔥🔥 Access  **DeepSeek R1 model for free** when you join our [public LLM pool](https://kalavai-net.github.io/kalavai-client/public_llm_pool/)
 - 31 January 2025: `kalavai-client` is now a [PyPI package](https://pypi.org/project/kalavai-client/), easier to install than ever!
+<details>
+<summary>More news</summary>
+
 - 27 January 2025: Support for accessing pools from remote computers
 - 9 January 2025: Added support for [Aphrodite Engine](https://github.com/aphrodite-engine/aphrodite-engine) models
 - 8 January 2025: Release of [a free, public, shared pool](/docs/docs/public_llm_pool.md) for community LLM deployment
@@ -102,6 +119,7 @@ https://github.com/user-attachments/assets/0d2316f3-79ea-46ac-b41e-8ef720f52672
 - 24 November 2024: Common pools with private user spaces
 - 30 October 2024: Release of our [public pool platform](https://platform.kalavai.net)
 
+</details>
 
 ### Support for LLM engines
 
@@ -139,6 +157,10 @@ The `kalavai-client` is the main tool to interact with the Kalavai platform, to
 From release **v0.5.0, you can now install `kalavai-client` in non-worker computers**. You can run a pool on a set of machines and have the client on a remote computer from which you access the LLM pool. Because the client only requires having python installed, this means more computers are now supported to run it.
 
 
+<details>
+
+<summary>Requirements</summary>
+
 ### Requirements
 
 For workers sharing resources with the pool:
@@ -150,8 +172,11 @@ For workers sharing resources with the pool:
 
 Any system that runs python 3.6+ is able to run the `kalavai-client` and therefore connect and operate an LLM pool, [without sharing with the pool](). Your computer won't be adding its capacity to the pool, but it wil be able to deploy jobs and interact with models.
 
+</details>
+
+<details>
 
-#### Common issues
+<summary> Common issues</summary>
 
 If you see the following error:
 
@@ -175,6 +200,7 @@ Upgrade your setuptools:
 ```bash
 pip install -U setuptools
 ```
+</details>
 
 ### Install the client
 
@@ -184,54 +210,32 @@ The client is a python package and can be installed with one command:
 pip install kalavai-client
 ```
 
-## Public LLM pools: crowdsource community resources
-
-This is the **easiest and most powerful** way to experience Kalavai. It affords users the full resource capabilities of the community and access to all its deployed LLMs, via an [OpenAI-compatible endpoint](https://kalavai-net.github.io/kalavai-client/public_llm_pool/#single-api-endpoint) as well as a [UI-based playground](https://kalavai-net.github.io/kalavai-client/public_llm_pool/#ui-playground).
-
-Check out [our guide](https://kalavai-net.github.io/kalavai-client/public_llm_pool/) on how to join and start deploying LLMs.
-
 
 ## Createa a local, private LLM pool
 
-Kalavai is **free to use, no caps, for both commercial and non-commercial purposes**. All you need to get started is one or more computers that can see each other (i.e. within the same network), and you are good to go. If you wish to join computers in different locations / networks, check [managed kalavai](#public-pools-crowdsource-community-resources).
+> Kalavai is **free to use, no caps, for both commercial and non-commercial purposes**. All you need to get started is one or more computers that can see each other (i.e. within the same network), and you are good to go. If you are interested in join computers in different locations / networks, [contact us](mailto:info@kalavai.net) or [book a demo](https://app.onecal.io/b/kalavai/book-a-demo) with the founders.
 
-### 1. Start a seed node
-
-Simply use the client to start your seed node:
+You can create and manage your pools with the new kalavai GUI, which can be started with:
 
 ```bash
-kalavai pool start <pool-name>
+kalavai gui start
 ```
 
-Now you are ready to add worker nodes to this seed. To do so, generate a joining token:
-```bash
-$ kalavai pool token --user
+This will expose the GUI and the backend services in localhost. By default, the GUI is accessible via [http://localhost:3000](http://localhost:3000). In the UI users can create and join LLM pools, monitor devices, deploy LLMs and more.
 
-Join token: <token>
-```
-
-### 2. Add worker nodes
-
-Increase the power of your AI pool by inviting others to join.
-
-Copy the joining token. On the worker node, run:
+![Kalavai logo](docs/docs/assets/images/ui_dashboard_multiple.png)
 
-```bash
-kalavai pool join <token>
-```
+Check out our [getting started guide](https://kalavai-net.github.io/kalavai-client/getting_started/) for next steps.
 
-### 3. Attach more clients
 
-You can now connect to an existing pool from any computer -not just from worker nodes. To connect to a pool, run:
+## Public LLM pools: crowdsource community resources
 
-```bash
-kalavai pool attach <token>
-```
+This is the **easiest and most powerful** way to experience Kalavai. It affords users the full resource capabilities of the community and access to all its deployed LLMs, via an [OpenAI-compatible endpoint](https://kalavai-net.github.io/kalavai-client/public_llm_pool/#single-api-endpoint) as well as a [UI-based playground](https://kalavai-net.github.io/kalavai-client/public_llm_pool/#ui-playground).
 
-This won't add the machine as a worker, but you will be able to operate in the pool as if you were. This is ideal for remote access to the pool, and to use the pool from machines that cannot run workers (docker container limitations).
+Check out [our guide](https://kalavai-net.github.io/kalavai-client/public_llm_pool/) on how to join and start deploying LLMs.
 
 
-### Enough already, let's run stuff!
+## Enough already, let's run stuff!
 
 Check our [examples](examples/) to put your new AI pool to good use!
 - [Single node vLLM GPU LLM](examples/singlenode_gpu_vllm.md) deployment
@@ -244,6 +248,10 @@ Check our [examples](examples/) to put your new AI pool to good use!
 
 If your system is not currently supported, [open an issue](https://github.com/kalavai-net/kalavai-client/issues) and request it. We are expanding this list constantly.
 
+<details>
+
+**<summary>Hardware and OS compatibility </summary>**
+
 ### OS compatibility
 
 Since **worker nodes** run inside docker, any machine that can run docker **should** be compatible with Kalavai. Here are instructions for [linux](https://docs.docker.com/engine/install/), [Windows](https://docs.docker.com/desktop/setup/install/windows-install/) and [MacOS](https://docs.docker.com/desktop/setup/install/mac-install/).
@@ -257,6 +265,7 @@ The kalavai client, which controls and access pools, can be installed on any mac
 - NVIDIA GPU
 - AMD and Intel GPUs are currently not supported ([interested in helping us test it?](https://kalavai-net.github.io/kalavai-client/compatibility/#help-testing-amd-gpus))
 
+</details>
 
 ## Roadmap
 
@@ -268,6 +277,7 @@ The kalavai client, which controls and access pools, can be installed on any mac
 - [x] Collaborative LLM deployment
 - [x] Ray cluster support
 - [x] Kalavai client on Mac
+- [x] Kalavai pools UI
 - [ ] [TEMPLATE] [GPUStack](https://github.com/gpustack/gpustack) support
 - [ ] [TEMPLATE] [exo](https://github.com/exo-explore/exo) support
 - [ ] Support for AMD GPUs
@@ -293,7 +303,9 @@ Anything missing here? Give us a shout in the [discussion board](https://github.
 
 ## Build from source
 
-### Requirements
+<details>
+
+<summary>Expand</summary>
 
 Python version >= 3.6.
 
@@ -313,6 +325,7 @@ Build python wheels:
 bash publish.sh build
 ```
 
+</details>
 
 ### Unit tests
 
@@ -322,5 +335,3 @@ To run the unit tests, use:
 python -m unittest
 ```
 
-docker run --rm --net=host -v   /root/.cache/kalavai/:/root/.cache/kalavai/  ghcr.io/helmfile/helmfile:v0.169.2 helmfile sync --file  /root/.cache/kalavai/apps.yaml --kubeconfig /root/.cache/kalavai/kubeconfig
-

{kalavai_client-0.5.19 → kalavai_client-0.5.20}/README.md

@@ -27,6 +27,19 @@
 
 Kalavai's goal is to make using LLMs in real applications accessible and affordable to all. It's a _magic box_ that **integrates all the components required to make LLM useful in the age of massive computing**, from sourcing computing power, managing distributed infrastructure and storage, using industry-standard model engines and orchestration of LLMs. 
 
+### Core features
+
+- Manage **multiple devices resources as one**. One pool of RAM, CPUs and GPUs
+- **Deploy Large Language Models seamlessly across devices**, wherever they are (multiple clouds, on premises, personal devices)
+- Auto-discovery: all **models are automatically exposed** through a single OpenAI-like API and a ChatGPT-like UI playground
+- Compatible with [most popular model engines](#support-for-llm-engines)
+- [Easy to expand](https://github.com/kalavai-net/kube-watcher/tree/main/templates) to custom workloads
+
+
+<details>
+
+**<summary>Video tutorials</summary>**
+
 ### Aggregate multiple devices in an LLM pool
 
 https://github.com/user-attachments/assets/4be59886-1b76-4400-ab5c-c803e3e414ec
@@ -44,12 +57,16 @@ https://github.com/user-attachments/assets/7df73bbc-d129-46aa-8ce5-0735177dedeb
 https://github.com/user-attachments/assets/0d2316f3-79ea-46ac-b41e-8ef720f52672
 
 
-### News updates
+</details>
 
-<img src="docs/docs/assets/images/DeepSeek-Emblem.png" width="100">
+### Latest updates
 
+- 20 February 2025: New shiny GUI interface to control LLM pools and deploy models
 - 6 February 2025: 🔥🔥🔥 Access  **DeepSeek R1 model for free** when you join our [public LLM pool](https://kalavai-net.github.io/kalavai-client/public_llm_pool/)
 - 31 January 2025: `kalavai-client` is now a [PyPI package](https://pypi.org/project/kalavai-client/), easier to install than ever!
+<details>
+<summary>More news</summary>
+
 - 27 January 2025: Support for accessing pools from remote computers
 - 9 January 2025: Added support for [Aphrodite Engine](https://github.com/aphrodite-engine/aphrodite-engine) models
 - 8 January 2025: Release of [a free, public, shared pool](/docs/docs/public_llm_pool.md) for community LLM deployment
@@ -58,6 +75,7 @@ https://github.com/user-attachments/assets/0d2316f3-79ea-46ac-b41e-8ef720f52672
 - 24 November 2024: Common pools with private user spaces
 - 30 October 2024: Release of our [public pool platform](https://platform.kalavai.net)
 
+</details>
 
 ### Support for LLM engines
 
@@ -95,6 +113,10 @@ The `kalavai-client` is the main tool to interact with the Kalavai platform, to
 From release **v0.5.0, you can now install `kalavai-client` in non-worker computers**. You can run a pool on a set of machines and have the client on a remote computer from which you access the LLM pool. Because the client only requires having python installed, this means more computers are now supported to run it.
 
 
+<details>
+
+<summary>Requirements</summary>
+
 ### Requirements
 
 For workers sharing resources with the pool:
@@ -106,8 +128,11 @@ For workers sharing resources with the pool:
 
 Any system that runs python 3.6+ is able to run the `kalavai-client` and therefore connect and operate an LLM pool, [without sharing with the pool](). Your computer won't be adding its capacity to the pool, but it wil be able to deploy jobs and interact with models.
 
+</details>
+
+<details>
 
-#### Common issues
+<summary> Common issues</summary>
 
 If you see the following error:
 
@@ -131,6 +156,7 @@ Upgrade your setuptools:
 ```bash
 pip install -U setuptools
 ```
+</details>
 
 ### Install the client
 
@@ -140,54 +166,32 @@ The client is a python package and can be installed with one command:
 pip install kalavai-client
 ```
 
-## Public LLM pools: crowdsource community resources
-
-This is the **easiest and most powerful** way to experience Kalavai. It affords users the full resource capabilities of the community and access to all its deployed LLMs, via an [OpenAI-compatible endpoint](https://kalavai-net.github.io/kalavai-client/public_llm_pool/#single-api-endpoint) as well as a [UI-based playground](https://kalavai-net.github.io/kalavai-client/public_llm_pool/#ui-playground).
-
-Check out [our guide](https://kalavai-net.github.io/kalavai-client/public_llm_pool/) on how to join and start deploying LLMs.
-
 
 ## Createa a local, private LLM pool
 
-Kalavai is **free to use, no caps, for both commercial and non-commercial purposes**. All you need to get started is one or more computers that can see each other (i.e. within the same network), and you are good to go. If you wish to join computers in different locations / networks, check [managed kalavai](#public-pools-crowdsource-community-resources).
+> Kalavai is **free to use, no caps, for both commercial and non-commercial purposes**. All you need to get started is one or more computers that can see each other (i.e. within the same network), and you are good to go. If you are interested in join computers in different locations / networks, [contact us](mailto:info@kalavai.net) or [book a demo](https://app.onecal.io/b/kalavai/book-a-demo) with the founders.
 
-### 1. Start a seed node
-
-Simply use the client to start your seed node:
+You can create and manage your pools with the new kalavai GUI, which can be started with:
 
 ```bash
-kalavai pool start <pool-name>
+kalavai gui start
 ```
 
-Now you are ready to add worker nodes to this seed. To do so, generate a joining token:
-```bash
-$ kalavai pool token --user
+This will expose the GUI and the backend services in localhost. By default, the GUI is accessible via [http://localhost:3000](http://localhost:3000). In the UI users can create and join LLM pools, monitor devices, deploy LLMs and more.
 
-Join token: <token>
-```
-
-### 2. Add worker nodes
-
-Increase the power of your AI pool by inviting others to join.
-
-Copy the joining token. On the worker node, run:
+![Kalavai logo](docs/docs/assets/images/ui_dashboard_multiple.png)
 
-```bash
-kalavai pool join <token>
-```
+Check out our [getting started guide](https://kalavai-net.github.io/kalavai-client/getting_started/) for next steps.
 
-### 3. Attach more clients
 
-You can now connect to an existing pool from any computer -not just from worker nodes. To connect to a pool, run:
+## Public LLM pools: crowdsource community resources
 
-```bash
-kalavai pool attach <token>
-```
+This is the **easiest and most powerful** way to experience Kalavai. It affords users the full resource capabilities of the community and access to all its deployed LLMs, via an [OpenAI-compatible endpoint](https://kalavai-net.github.io/kalavai-client/public_llm_pool/#single-api-endpoint) as well as a [UI-based playground](https://kalavai-net.github.io/kalavai-client/public_llm_pool/#ui-playground).
 
-This won't add the machine as a worker, but you will be able to operate in the pool as if you were. This is ideal for remote access to the pool, and to use the pool from machines that cannot run workers (docker container limitations).
+Check out [our guide](https://kalavai-net.github.io/kalavai-client/public_llm_pool/) on how to join and start deploying LLMs.
 
 
-### Enough already, let's run stuff!
+## Enough already, let's run stuff!
 
 Check our [examples](examples/) to put your new AI pool to good use!
 - [Single node vLLM GPU LLM](examples/singlenode_gpu_vllm.md) deployment
@@ -200,6 +204,10 @@ Check our [examples](examples/) to put your new AI pool to good use!
 
 If your system is not currently supported, [open an issue](https://github.com/kalavai-net/kalavai-client/issues) and request it. We are expanding this list constantly.
 
+<details>
+
+**<summary>Hardware and OS compatibility </summary>**
+
 ### OS compatibility
 
 Since **worker nodes** run inside docker, any machine that can run docker **should** be compatible with Kalavai. Here are instructions for [linux](https://docs.docker.com/engine/install/), [Windows](https://docs.docker.com/desktop/setup/install/windows-install/) and [MacOS](https://docs.docker.com/desktop/setup/install/mac-install/).
@@ -213,6 +221,7 @@ The kalavai client, which controls and access pools, can be installed on any mac
 - NVIDIA GPU
 - AMD and Intel GPUs are currently not supported ([interested in helping us test it?](https://kalavai-net.github.io/kalavai-client/compatibility/#help-testing-amd-gpus))
 
+</details>
 
 ## Roadmap
 
@@ -224,6 +233,7 @@ The kalavai client, which controls and access pools, can be installed on any mac
 - [x] Collaborative LLM deployment
 - [x] Ray cluster support
 - [x] Kalavai client on Mac
+- [x] Kalavai pools UI
 - [ ] [TEMPLATE] [GPUStack](https://github.com/gpustack/gpustack) support
 - [ ] [TEMPLATE] [exo](https://github.com/exo-explore/exo) support
 - [ ] Support for AMD GPUs
@@ -249,7 +259,9 @@ Anything missing here? Give us a shout in the [discussion board](https://github.
 
 ## Build from source
 
-### Requirements
+<details>
+
+<summary>Expand</summary>
 
 Python version >= 3.6.
 
@@ -269,6 +281,7 @@ Build python wheels:
 bash publish.sh build
 ```
 
+</details>
 
 ### Unit tests
 
@@ -277,5 +290,3 @@ To run the unit tests, use:
 ```bash
 python -m unittest
 ```
-
-docker run --rm --net=host -v   /root/.cache/kalavai/:/root/.cache/kalavai/  ghcr.io/helmfile/helmfile:v0.169.2 helmfile sync --file  /root/.cache/kalavai/apps.yaml --kubeconfig /root/.cache/kalavai/kubeconfig

kalavai_client-0.5.20/kalavai_client/__init__.py

@@ -0,0 +1,2 @@
+
+__version__ = "0.5.20"

kalavai_client-0.5.20/kalavai_client/bridge_api.py

@@ -0,0 +1,216 @@
+"""
+Core kalavai service.
+Used as a bridge between the kalavai-client app and the reflex frontend
+"""
+from fastapi import FastAPI
+import uvicorn
+
+from kalavai_client.bridge_models import (
+    CreatePoolRequest,
+    JoinPoolRequest,
+    StopPoolRequest,
+    DeployJobRequest,
+    DeleteJobRequest,
+    JobDetailsRequest,
+    DeleteNodesRequest
+)
+from kalavai_client.core import (
+    create_pool,
+    join_pool,
+    attach_to_pool,
+    stop_pool,
+    fetch_devices,
+    fetch_resources,
+    fetch_job_names,
+    fetch_gpus,
+    fetch_job_details,
+    fetch_job_logs,
+    fetch_job_templates,
+    fetch_job_defaults,
+    deploy_job,
+    delete_job,
+    authenticate_user,
+    load_user_session,
+    user_logout,
+    is_connected,
+    list_available_pools,
+    is_agent_running,
+    is_server,
+    pause_agent,
+    resume_agent,
+    get_ip_addresses,
+    get_pool_token,
+    delete_nodes,
+    TokenType
+)
+
+app = FastAPI()
+
+@app.post("/create_pool")
+def pool_create(request: CreatePoolRequest):
+    result = create_pool(
+        cluster_name=request.cluster_name,
+        ip_address=request.ip_address,
+        app_values=request.app_values,
+        num_gpus=request.num_gpus,
+        node_name=request.node_name,
+        only_registered_users=request.only_registered_users,
+        location=request.location
+    )
+    return result
+
+@app.post("/join_pool")
+def pool_join(request: JoinPoolRequest):
+    result = join_pool(
+        token=request.token,
+        num_gpus=request.num_gpus,
+        node_name=request.node_name
+    )
+    return result
+
+@app.post("/attach_to_pool")
+def pool_attach(request: JoinPoolRequest):
+    result = attach_to_pool(
+        token=request.token,
+        node_name=request.node_name
+    )
+    return result
+
+@app.post("/stop_pool")
+def pool_stop(request: StopPoolRequest):
+    result = stop_pool(
+        skip_node_deletion=request.skip_node_deletion
+    )
+    return result
+
+@app.post("/delete_nodes")
+def device_delete(request: DeleteNodesRequest):
+    result = delete_nodes(
+        nodes=request.nodes
+    )
+    return result
+
+@app.get("/get_pool_token")
+def devices(mode: int):
+
+    return get_pool_token(mode=TokenType(mode))
+
+@app.get("/fetch_devices")
+def devices():
+    return fetch_devices()
+
+@app.get("/fetch_resources")
+def resources():
+    return fetch_resources()
+
+@app.get("/fetch_job_names")
+def job_names():
+    return fetch_job_names()
+
+@app.get("/fetch_gpus")
+def gpus(available: bool = False):
+    return fetch_gpus(available=available)
+
+@app.post("/fetch_job_details")
+def job_details(request: JobDetailsRequest):
+    return fetch_job_details(jobs=request.jobs)
+
+@app.get("/fetch_job_logs")
+def job_logs(job_name: str, force_namespace: str=None, pod_name: str=None, tail: int=100):
+    return fetch_job_logs(
+        job_name=job_name,
+        force_namespace=force_namespace,
+        pod_name=pod_name,
+        tail=tail
+    )
+
+@app.get("/fetch_job_templates")
+def job_templates():
+    return fetch_job_templates()
+
+@app.get("/fetch_job_defaults")
+def job_templates(name: str):
+    return fetch_job_defaults(name=name)
+
+@app.post("/deploy_job")
+def job_deploy(request: DeployJobRequest):
+    result = deploy_job(
+        template_name=request.template_name,
+        values_dict=request.values,
+        force_namespace=request.force_namespace
+    )
+    return result
+
+@app.post("/delete_job")
+def job_delete(request: DeleteJobRequest):
+    result = delete_job(
+        name=request.name,
+        force_namespace=request.force_namespace
+    )
+    return result
+
+@app.get("/authenticate_user")
+def user_authenticate(username: str, password: str):
+    result = authenticate_user(
+        username=username,
+        password=password
+    )
+    return result
+
+@app.get("/load_user_session")
+def user_session():
+    result = load_user_session()
+    return result
+
+@app.get("/user_logout")
+def logout_user():
+    result = user_logout()
+    return result
+
+@app.get("/is_connected")
+def pool_connected():
+    result = is_connected()
+    return result
+
+@app.get("/is_agent_running")
+def agent_running():
+    result = is_agent_running()
+    return result
+
+@app.get("/is_server")
+def server():
+    result = is_server()
+    return result
+
+@app.post("/pause_agent")
+def agent_pause():
+    result = pause_agent()
+    return result
+
+@app.post("/resume_agent")
+def agent_resume():
+    result = resume_agent()
+    return result
+
+@app.get("/get_ip_addresses")
+def ip_addresses(subnet: str=None):
+    result = get_ip_addresses(subnet=subnet)
+    return result
+
+@app.get("/list_available_pools")
+def pool_connected(user_only: bool=False):
+    result = list_available_pools(user_only=user_only)
+    return result
+
+
+def run_api(host="0.0.0.0", port=8001, log_level="critical"):
+    uvicorn.run(
+        app,
+        host=host,
+        port=port,
+        log_level=log_level
+    )
+
+if __name__ == "__main__":
+    run_api()
+    

kalavai_client-0.5.20/kalavai_client/bridge_models.py

@@ -0,0 +1,37 @@
+from pydantic import BaseModel
+
+from kalavai_client.core import Job
+
+
+class CreatePoolRequest(BaseModel):
+    cluster_name: str
+    ip_address: str
+    app_values: dict = None
+    num_gpus: int = None
+    node_name: str = None
+    only_registered_users: bool = False
+    location: str = None
+
+class DeleteNodesRequest(BaseModel):
+    nodes: list[str]
+
+class JoinPoolRequest(BaseModel):
+    token: str
+    node_name: str = None
+    num_gpus: int = None
+
+class JobDetailsRequest(BaseModel):
+    jobs: list[Job]
+
+
+class StopPoolRequest(BaseModel):
+    skip_node_deletion: bool = False
+
+class DeployJobRequest(BaseModel):
+    template_name: str
+    values: dict
+    force_namespace: str = None
+
+class DeleteJobRequest(BaseModel):
+    name: str
+    force_namespace: str = None

{kalavai_client-0.5.19 → kalavai_client-0.5.20}/kalavai_client/cli.py

@@ -15,6 +15,7 @@ import arguably
 from rich.console import Console
 
 from kalavai_client.cluster import CLUSTER
+from kalavai_client.bridge_api import run_api
 from kalavai_client.env import (
     USER_COOKIE,
     USER_LOCAL_SERVER_FILE,
@@ -50,13 +51,15 @@ from kalavai_client.core import (
     create_pool,
     get_ip_addresses,
     pause_agent,
-    resume_agent
+    resume_agent,
+    get_pool_token,
+    delete_nodes,
+    TokenType
 )
 from kalavai_client.utils import (
     check_gpu_drivers,
     load_template,
     run_cmd,
-    generate_join_token,
     user_confirm,
     generate_table,
     request_to_server,
@@ -71,11 +74,6 @@ from kalavai_client.utils import (
     get_public_seeds,
     load_user_session,
     SERVER_IP_KEY,
-    AUTH_KEY,
-    WATCHER_SERVICE_KEY,
-    READONLY_AUTH_KEY,
-    WRITE_AUTH_KEY,
-    PUBLIC_LOCATION_KEY,
     NODE_NAME_KEY,
     CLUSTER_NAME_KEY
 )
@@ -225,29 +223,30 @@ def input_gpus():
 ##################
 
 @arguably.command
-def gui__start(*others):
-    """Run GUI"""
-    values = {
-        "path": user_path("")
-    }
-    compose_yaml = load_template(
-        template_path=DOCKER_COMPOSE_GUI,
-        values=values)
-    with open(USER_GUI_COMPOSE_FILE, "w") as f:
-        f.write(compose_yaml)
-    
-    run_cmd(f"docker compose --file {USER_GUI_COMPOSE_FILE} up -d")
-
-    console.log(f"[green]Loading GUI, may take a few minutes. It will be available at http://localhost:3000")
-
-@arguably.command
-def gui__stop(*others):
-    """Stop GUI"""
-    run_cmd(f"docker compose --file {USER_GUI_COMPOSE_FILE} down")
+def gui__start(*others, backend_only=False, gui_frontend_port=3000, gui_backend_port=8000, bridge_port=8001):
+    """Run GUI (docker) and kalavai core backend (api)"""
+
+    if not backend_only:
+        values = {
+            "gui_frontend_port": gui_frontend_port,
+            "gui_backend_port": gui_backend_port,
+            "path": user_path("")
+        }
+        compose_yaml = load_template(
+            template_path=DOCKER_COMPOSE_GUI,
+            values=values)
+        with open(USER_GUI_COMPOSE_FILE, "w") as f:
+            f.write(compose_yaml)
+        
+        run_cmd(f"docker compose --file {USER_GUI_COMPOSE_FILE} up -d")
 
+        console.log(f"[green]Loading GUI, may take a few minutes. It will be available at http://localhost:{gui_frontend_port}")
+    run_api(port=bridge_port)
+    
+    if not backend_only:
+        run_cmd(f"docker compose --file {USER_GUI_COMPOSE_FILE} down")
     console.log("[green]Kalavai GUI has been stopped")
 
-
 @arguably.command
 def login(*others,  username: str=None):
     """
@@ -451,32 +450,19 @@ def pool__token(*others, admin=False, user=False, worker=False):
         return
     
     if admin:
-        auth_key = load_server_info(data_key=AUTH_KEY, file=USER_LOCAL_SERVER_FILE)
+        mode = TokenType.ADMIN
     elif user:
-        auth_key = load_server_info(data_key=WRITE_AUTH_KEY, file=USER_LOCAL_SERVER_FILE)
+        mode = TokenType.USER
     else:
-        auth_key = load_server_info(data_key=READONLY_AUTH_KEY, file=USER_LOCAL_SERVER_FILE)
-    
-    watcher_service = load_server_info(data_key=WATCHER_SERVICE_KEY, file=USER_LOCAL_SERVER_FILE)
-    public_location = load_server_info(data_key=PUBLIC_LOCATION_KEY, file=USER_LOCAL_SERVER_FILE)
-
-    cluster_token = CLUSTER.get_cluster_token()
-
-    ip_address = load_server_info(SERVER_IP_KEY, file=USER_LOCAL_SERVER_FILE)
-    cluster_name = load_server_info(data_key=CLUSTER_NAME_KEY, file=USER_LOCAL_SERVER_FILE)
+        mode = TokenType.WORKER
 
-    join_token = generate_join_token(
-        cluster_ip=ip_address,
-        cluster_name=cluster_name,
-        cluster_token=cluster_token,
-        auth_key=auth_key,
-        watcher_service=watcher_service,
-        public_location=public_location
-    )
-
-    console.log("[green]Join token:")
-    print(join_token)
+    join_token = get_pool_token(mode=mode)
 
+    if "error" in join_token:
+        console.log(f"[red]{join_token}")
+    else:
+        console.log("[green]Join token:")
+        print(join_token)
     return join_token
 
 @arguably.command
@@ -949,24 +935,12 @@ def node__delete(name, *others):
         console.log(f"[red]Problems with your pool: {str(e)}")
         return
     
-    data = {
-        "node_names": [name]
-    }
-    try:
-        result = request_to_server(
-            method="post",
-            endpoint="/v1/delete_nodes",
-            data=data,
-            server_creds=USER_LOCAL_SERVER_FILE,
-            user_cookie=USER_COOKIE
-        )
-        if result is None or result is True:
-            console.log(f"Node {name} deleted successfully")
-        else:
-            console.log(f"{result}")
-    except Exception as e:
-        console.log(f"[yellow](ignore if stopping worker from dead server). Error when removing node {name}: {str(e)}")
-
+    result = delete_nodes(nodes=[name])
+    
+    if "error" in result:
+        console.log(f"[red]{result}")
+    else:
+        console.log(f"[green]{result}")
 
 @arguably.command
 def node__cordon(node_name, *others):

{kalavai_client-0.5.19 → kalavai_client-0.5.20}/kalavai_client/core.py

@@ -6,11 +6,14 @@ import uuid
 import socket
 import ipaddress
 import netifaces as ni
-
+from typing import Optional
 from pydantic import BaseModel
+from enum import Enum
 
 from kalavai_client.cluster import CLUSTER
 from kalavai_client.utils import (
+    check_gpu_drivers,
+    generate_join_token,
     request_to_server,
     load_server_info,
     decode_dict,
@@ -68,11 +71,11 @@ from kalavai_client.env import (
 )
 
 class Job(BaseModel):
-    owner: str = None
-    name: str = None
-    workers: str = None
-    endpoint: str = None
-    status: str = None
+    owner: Optional[str] = None
+    name: Optional[str] = None
+    workers: Optional[str] = None
+    endpoint: Optional[str] = None
+    status: Optional[str] = None
 
 class DeviceStatus(BaseModel):
     name: str
@@ -89,6 +92,11 @@ class GPU(BaseModel):
     ready: bool
     model: str
 
+class TokenType(Enum):
+    ADMIN = 0
+    USER = 1
+    WORKER = 2
+
 
 def init_user_workspace(force_namespace=None):
     
@@ -461,6 +469,25 @@ def check_token(token, public=False):
         return {"status": True}
     except Exception as e:
         return {"error": str(e)}
+    
+def delete_nodes(nodes):
+    data = {
+        "node_names": nodes
+    }
+    try:
+        result = request_to_server(
+            method="post",
+            endpoint="/v1/delete_nodes",
+            data=data,
+            server_creds=USER_LOCAL_SERVER_FILE,
+            user_cookie=USER_COOKIE
+        )
+        if result is None or result is True:
+            return {"success": nodes}
+        else:
+            return {"error": result}
+    except Exception as e:
+        return {"error": f"Error when removing nodes {nodes}: {str(e)}"}
 
 def attach_to_pool(token, node_name=None):
     if node_name is None:
@@ -530,11 +557,24 @@ def attach_to_pool(token, node_name=None):
 
     return cluster_name
 
-def join_pool(token, num_gpus=0, node_name=None):
+def get_max_gpus():
+    try:
+        has_gpus = check_gpu_drivers()
+        if has_gpus:
+            return int(run_cmd("nvidia-smi -L | wc -l").decode())
+        else:
+            return 0
+    except:
+        return 0
+
+def join_pool(token, num_gpus=None, node_name=None):
     compatibility = check_worker_compatibility()
     if len(compatibility["issues"]) > 0:
         return {"error": compatibility["issues"]}
-    
+
+    if num_gpus is None:
+        num_gpus = get_max_gpus()
+
     if node_name is None:
         node_name = f"{socket.gethostname()}-{uuid.uuid4().hex[:6]}"
     
@@ -751,6 +791,40 @@ def create_pool(cluster_name: str, ip_address: str, app_values: str=None, pool_c
     
     return {"success"}
 
+def get_pool_token(mode: TokenType):
+
+    try:
+        match mode:
+            case TokenType.ADMIN:
+                auth_key = load_server_info(data_key=AUTH_KEY, file=USER_LOCAL_SERVER_FILE)
+            case TokenType.USER:
+                auth_key = load_server_info(data_key=WRITE_AUTH_KEY, file=USER_LOCAL_SERVER_FILE)
+            case _:
+                auth_key = load_server_info(data_key=READONLY_AUTH_KEY, file=USER_LOCAL_SERVER_FILE)
+        if auth_key is None:
+            return {"error": "Cannot generate selected token mode. Are you the seed node?"}
+
+        watcher_service = load_server_info(data_key=WATCHER_SERVICE_KEY, file=USER_LOCAL_SERVER_FILE)
+        public_location = load_server_info(data_key=PUBLIC_LOCATION_KEY, file=USER_LOCAL_SERVER_FILE)
+
+        cluster_token = CLUSTER.get_cluster_token()
+
+        ip_address = load_server_info(SERVER_IP_KEY, file=USER_LOCAL_SERVER_FILE)
+        cluster_name = load_server_info(data_key=CLUSTER_NAME_KEY, file=USER_LOCAL_SERVER_FILE)
+
+        join_token = generate_join_token(
+            cluster_ip=ip_address,
+            cluster_name=cluster_name,
+            cluster_token=cluster_token,
+            auth_key=auth_key,
+            watcher_service=watcher_service,
+            public_location=public_location
+        )
+
+        return {"token": join_token}
+    except Exception as e:
+        return {"error": f"Error when generating token: {str(e)}"}
+
 def pool_init(pool_config_values_path=None):
     """Deploy configured objects to initialise pool"""
     if pool_config_values_path is None:

{kalavai_client-0.5.19 → kalavai_client-0.5.20}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name            = "kalavai-client"
-version         = "0.5.19"
+version         = "0.5.20"
 authors = [
   {name = "Carlos Fernandez Musoles", email = "carlos@kalavai.net"}
 ]

kalavai_client-0.5.19/kalavai_client/__init__.py

@@ -1,2 +0,0 @@
-
-__version__ = "0.5.19"