kalavai-client 0.5.9__tar.gz → 0.5.12__tar.gz

{kalavai_client-0.5.9 → kalavai_client-0.5.12}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: kalavai-client
-Version: 0.5.9
+Version: 0.5.12
 Summary: Client app for kalavai platform
 License: Apache-2.0
 Keywords: LLM,platform
@@ -8,10 +8,8 @@ Author: Carlos Fernandez Musoles
 Author-email: carlos@kalavai.net
 Maintainer: Carlos Fernandez Musoles
 Maintainer-email: carlos@kalavai.net
-Requires-Python: <3.12
+Requires-Python: >=3.4
 Classifier: License :: OSI Approved :: Apache Software License
-Classifier: Programming Language :: Python :: 2
-Classifier: Programming Language :: Python :: 2.7
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.4
 Classifier: Programming Language :: Python :: 3.5
@@ -21,6 +19,8 @@ Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Provides-Extra: dev
 Requires-Dist: Pillow (==10.3.0)
 Requires-Dist: anvil-uplink (==0.5.1)
@@ -89,6 +89,7 @@ https://github.com/user-attachments/assets/0d2316f3-79ea-46ac-b41e-8ef720f52672
 
 ### News updates
 
+- 31 January 2025: `kalavai-client` is now a [PyPI package](https://pypi.org/project/kalavai-client/), easier to install than ever!
 - 27 January 2025: Support for accessing pools from remote computers
 - 9 January 2025: Added support for [Aphrodite Engine](https://github.com/aphrodite-engine/aphrodite-engine) models
 - 8 January 2025: Release of [a free, public, shared pool](/docs/docs/public_llm_pool.md) for community LLM deployment
@@ -129,20 +130,24 @@ Not what you were looking for? [Tell us](https://github.com/kalavai-net/kalavai-
 
 ## Getting started
 
-The `kalavai` client is the main tool to interact with the Kalavai platform, to create and manage both local and public pools and also to interact with them (e.g. deploy models). Let's go over its installation. 
+The `kalavai-client` is the main tool to interact with the Kalavai platform, to create and manage both local and public pools and also to interact with them (e.g. deploy models). Let's go over its installation. 
 
-From release **v0.5.0, you can now install `kalavai` client in non-worker computers**. You can run a pool on a set of machines and have the client on a remote computer from which you access the LLM pool. Because the client only requires having python installed, this means more computers are now supported to run it.
+From release **v0.5.0, you can now install `kalavai-client` in non-worker computers**. You can run a pool on a set of machines and have the client on a remote computer from which you access the LLM pool. Because the client only requires having python installed, this means more computers are now supported to run it.
 
 
-### Requirements for a worker machine
+### Requirements
+
+For workers sharing resources with the pool:
 
 - A laptop, desktop or Virtual Machine
 - Docker engine installed (for [linux](https://docs.docker.com/engine/install/), [Windows and MacOS](https://docs.docker.com/desktop/)) with [privilege access](https://docs.docker.com/engine/containers/run/#runtime-privilege-and-linux-capabilities).
 
+> **Support for Windows and MacOS workers is experimental**: kalavai workers run on docker containers that require access to the host network interfaces, thus systems that do not support containers natively (Windows and MacOS) may have difficulties finding each other.
+
+Any system that runs python 3.6+ is able to run the `kalavai-client` and therefore connect and operate an LLM pool, [without sharing with the pool](). Your computer won't be adding its capacity to the pool, but it wil be able to deploy jobs and interact with models.
 
-### Requirements to run the client
 
-- Python 3.10+
+#### Common issues
 
 If you see the following error:
 
@@ -211,6 +216,17 @@ Copy the joining token. On the worker node, run:
 kalavai pool join <token>
 ```
 
+### 3. Attach more clients
+
+You can now connect to an existing pool from any computer -not just from worker nodes. To connect to a pool, run:
+
+```bash
+kalavai pool attach <token>
+```
+
+This won't add the machine as a worker, but you will be able to operate in the pool as if you were. This is ideal for remote access to the pool, and to use the pool from machines that cannot run workers (docker container limitations).
+
+
 ### Enough already, let's run stuff!
 
 Check our [examples](examples/) to put your new AI pool to good use!
@@ -275,7 +291,7 @@ Anything missing here? Give us a shout in the [discussion board](https://github.
 
 ### Requirements
 
-Python version <= 3.12.
+Python version >= 3.6.
 
 ```bash
 sudo add-apt-repository ppa:deadsnakes/ppa

{kalavai_client-0.5.9 → kalavai_client-0.5.12}/README.md

@@ -46,6 +46,7 @@ https://github.com/user-attachments/assets/0d2316f3-79ea-46ac-b41e-8ef720f52672
 
 ### News updates
 
+- 31 January 2025: `kalavai-client` is now a [PyPI package](https://pypi.org/project/kalavai-client/), easier to install than ever!
 - 27 January 2025: Support for accessing pools from remote computers
 - 9 January 2025: Added support for [Aphrodite Engine](https://github.com/aphrodite-engine/aphrodite-engine) models
 - 8 January 2025: Release of [a free, public, shared pool](/docs/docs/public_llm_pool.md) for community LLM deployment
@@ -86,20 +87,24 @@ Not what you were looking for? [Tell us](https://github.com/kalavai-net/kalavai-
 
 ## Getting started
 
-The `kalavai` client is the main tool to interact with the Kalavai platform, to create and manage both local and public pools and also to interact with them (e.g. deploy models). Let's go over its installation. 
+The `kalavai-client` is the main tool to interact with the Kalavai platform, to create and manage both local and public pools and also to interact with them (e.g. deploy models). Let's go over its installation. 
 
-From release **v0.5.0, you can now install `kalavai` client in non-worker computers**. You can run a pool on a set of machines and have the client on a remote computer from which you access the LLM pool. Because the client only requires having python installed, this means more computers are now supported to run it.
+From release **v0.5.0, you can now install `kalavai-client` in non-worker computers**. You can run a pool on a set of machines and have the client on a remote computer from which you access the LLM pool. Because the client only requires having python installed, this means more computers are now supported to run it.
 
 
-### Requirements for a worker machine
+### Requirements
+
+For workers sharing resources with the pool:
 
 - A laptop, desktop or Virtual Machine
 - Docker engine installed (for [linux](https://docs.docker.com/engine/install/), [Windows and MacOS](https://docs.docker.com/desktop/)) with [privilege access](https://docs.docker.com/engine/containers/run/#runtime-privilege-and-linux-capabilities).
 
+> **Support for Windows and MacOS workers is experimental**: kalavai workers run on docker containers that require access to the host network interfaces, thus systems that do not support containers natively (Windows and MacOS) may have difficulties finding each other.
+
+Any system that runs python 3.6+ is able to run the `kalavai-client` and therefore connect and operate an LLM pool, [without sharing with the pool](). Your computer won't be adding its capacity to the pool, but it wil be able to deploy jobs and interact with models.
 
-### Requirements to run the client
 
-- Python 3.10+
+#### Common issues
 
 If you see the following error:
 
@@ -168,6 +173,17 @@ Copy the joining token. On the worker node, run:
 kalavai pool join <token>
 ```
 
+### 3. Attach more clients
+
+You can now connect to an existing pool from any computer -not just from worker nodes. To connect to a pool, run:
+
+```bash
+kalavai pool attach <token>
+```
+
+This won't add the machine as a worker, but you will be able to operate in the pool as if you were. This is ideal for remote access to the pool, and to use the pool from machines that cannot run workers (docker container limitations).
+
+
 ### Enough already, let's run stuff!
 
 Check our [examples](examples/) to put your new AI pool to good use!
@@ -232,7 +248,7 @@ Anything missing here? Give us a shout in the [discussion board](https://github.
 
 ### Requirements
 
-Python version <= 3.12.
+Python version >= 3.6.
 
 ```bash
 sudo add-apt-repository ppa:deadsnakes/ppa

kalavai_client-0.5.12/kalavai_client/__init__.py

@@ -0,0 +1,2 @@
+
+__version__ = "0.5.12"

{kalavai_client-0.5.9 → kalavai_client-0.5.12}/kalavai_client/assets/apps.yaml

@@ -139,7 +139,7 @@ releases:
     - name: replicas
       value: 2
     - name: image_tag
-      value: "v2025.01"
+      value: "v2025.01.1"
     - name: deployment.in_cluster
       value: "True"
     - name: deployment.use_auth_key

{kalavai_client-0.5.9 → kalavai_client-0.5.12}/kalavai_client/assets/docker-compose-template.yaml

@@ -1,8 +1,9 @@
 services:
 {% if vpn %}
   {{vpn_name}}:
-    image: gravitl/netclient:v0.30.0
+    image: gravitl/netclient:v0.24.3
     container_name: {{vpn_name}}
+    #privileged: true
     cap_add:
       - NET_ADMIN
       - SYS_MODULE

{kalavai_client-0.5.9 → kalavai_client-0.5.12}/kalavai_client/cli.py

@@ -337,18 +337,19 @@ def select_token_type():
             break
     return {"admin": choice == 0, "user": choice == 1, "worker": choice == 2}
 
-def generate_compose_config(role, node_name, is_public, node_labels=None, pool_ip=None, vpn_token=None, pool_token=None):
+def generate_compose_config(role, node_name, is_public, use_gpus=True, node_labels=None, pool_ip=None, vpn_token=None, pool_token=None):
     num_gpus = 0
-    try:
-        has_gpus = check_gpu_drivers()
-        if has_gpus:
-            max_gpus = int(run_cmd("nvidia-smi -L | wc -l").decode())
-            num_gpus = user_confirm(
-                question=f"{max_gpus} NVIDIA GPU(s) detected. How many GPUs would you like to include?",
-                options=range(max_gpus+1)
-            ) 
-    except:
-        console.log(f"[red]WARNING: error when fetching NVIDIA GPU info. GPUs will not be used on this local machine")
+    if use_gpus:
+        try:
+            has_gpus = check_gpu_drivers()
+            if has_gpus:
+                max_gpus = int(run_cmd("nvidia-smi -L | wc -l").decode())
+                num_gpus = user_confirm(
+                    question=f"{max_gpus} NVIDIA GPU(s) detected. How many GPUs would you like to include?",
+                    options=range(max_gpus+1)
+                ) 
+        except:
+            console.log(f"[red]WARNING: error when fetching NVIDIA GPU info. GPUs will not be used on this local machine")
     if node_labels is not None:
         node_labels = " ".join([f"--node-label {key}={value}" for key, value in node_labels.items()])
     compose_values = {
@@ -1102,6 +1103,10 @@ def pool__attach(token, *others, node_name=None):
     """
     Set creds in token on the local instance
     """
+
+    if node_name is None:
+        node_name = socket.gethostname()
+    
     # check that is not attached to another instance
     if os.path.exists(USER_LOCAL_SERVER_FILE):
         option = user_confirm(
@@ -1160,6 +1165,7 @@ def pool__attach(token, *others, node_name=None):
     
     # Generate docker compose recipe
     generate_compose_config(
+        use_gpus=False,
         role="",
         vpn_token=vpn["key"],
         node_name=node_name,
@@ -1510,6 +1516,62 @@ def job__run(template_name, *others, values: str=None, force_namespace: str=None
         console.log(f"[red]Error when connecting to kalavai service: {str(e)}")
         return
 
+@arguably.command
+def job__test(local_template_dir, *others, values, defaults, force_namespace: str=None):
+    """
+    Helper to test local templates, useful for development
+    """
+    try:
+        CLUSTER.validate_cluster()
+    except Exception as e:
+        console.log(f"[red]Problems with your pool: {str(e)}")
+        return
+    
+    if not os.path.isdir(local_template_dir):
+        console.log(f"[red]--local_template_dir ({local_template_dir}) is not a directory")
+        return
+    
+    # load template
+    with open(os.path.join(local_template_dir, "template.yaml"), "r") as f:
+        template_str = f.read()
+    
+    # load values
+    if not os.path.isfile(values):
+        console.log(f"[red]--values ({values}) is not a valid local file")
+        return
+    with open(values, "r") as f:
+        values_dict = yaml.safe_load(f)
+    # load defaults
+    if not os.path.isfile(defaults):
+        console.log(f"[red]--defaults ({defaults}) is not a valid local file")
+        return
+    with open(defaults, "r") as f:
+        defaults = f.read()
+    
+    # submit custom deployment
+    data = {
+        "template": template_str,
+        "template_values": values_dict,
+        "default_values": defaults
+    }
+    if force_namespace is not None:
+        data["force_namespace"] = force_namespace
+
+    try:
+        result = request_to_server(
+            method="post",
+            endpoint="/v1/deploy_custom_job",
+            data=data,
+            server_creds=USER_LOCAL_SERVER_FILE,
+            user_cookie=USER_COOKIE
+        )
+        console.log("Deployment result:")
+        print(
+            json.dumps(result,indent=3)
+        )
+    except Exception as e:
+        console.log(f"[red]Error when connecting to kalavai service: {str(e)}")
+
 
 @arguably.command
 def job__defaults(template_name, *others):

{kalavai_client-0.5.9 → kalavai_client-0.5.12}/kalavai_client/utils.py

@@ -284,8 +284,11 @@ def request_to_server(
         json=data,
         headers=headers
     )
-    result = response.json()
-    return result
+    try:
+        result = response.json()
+        return result
+    except Exception as e:
+        raise ValueError(f"Error with HTTP request: {response.text}\n{str(e)}")
 
 
 def generate_table(columns, rows, end_sections=None):
@@ -389,7 +392,7 @@ def resource_path(relative_path: str):
         last_slash = relative_path.rfind("/") 
         path = relative_path[:last_slash].replace("/", ".")
         filename = relative_path[last_slash+1:]
-        resource = importlib.resources.path(path, filename)
+        resource = str(importlib.resources.files(path).joinpath(filename))
     except Exception as e:
         return None
     return resource

{kalavai_client-0.5.9 → kalavai_client-0.5.12}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name            = "kalavai-client"
-version         = "0.5.9"
+version         = "0.5.12"
 authors = [
   {name = "Carlos Fernandez Musoles", email = "carlos@kalavai.net"}
 ]
@@ -12,7 +12,7 @@ license         = "Apache-2.0"
 license-files   = ["LICENSE"]
 keywords        = ["LLM", "platform"]
 readme          = {file = "README.md", content-type = "text/markdown"}
-requires-python = "<3.12"
+requires-python = ">=3.4"
 dependencies = [
     "requests>= 2.25",
     "psutil==5.9.8",

kalavai_client-0.5.9/kalavai_client/__init__.py

@@ -1,2 +0,0 @@
-
-__version__ = "0.5.9"