kalavai-client 0.5.8__tar.gz → 0.5.10__tar.gz

{kalavai_client-0.5.8 → kalavai_client-0.5.10}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: kalavai-client
-Version: 0.5.8
+Version: 0.5.10
 Summary: Client app for kalavai platform
 License: Apache-2.0
 Keywords: LLM,platform
@@ -8,10 +8,8 @@ Author: Carlos Fernandez Musoles
 Author-email: carlos@kalavai.net
 Maintainer: Carlos Fernandez Musoles
 Maintainer-email: carlos@kalavai.net
-Requires-Python: <3.12
+Requires-Python: >=3.4
 Classifier: License :: OSI Approved :: Apache Software License
-Classifier: Programming Language :: Python :: 2
-Classifier: Programming Language :: Python :: 2.7
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.4
 Classifier: Programming Language :: Python :: 3.5
@@ -21,6 +19,8 @@ Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Provides-Extra: dev
 Requires-Dist: Pillow (==10.3.0)
 Requires-Dist: anvil-uplink (==0.5.1)
@@ -89,6 +89,7 @@ https://github.com/user-attachments/assets/0d2316f3-79ea-46ac-b41e-8ef720f52672
 
 ### News updates
 
+- 31 January 2025: `kalavai-client` is now a [PyPI package](https://pypi.org/project/kalavai-client/), easier to install than ever!
 - 27 January 2025: Support for accessing pools from remote computers
 - 9 January 2025: Added support for [Aphrodite Engine](https://github.com/aphrodite-engine/aphrodite-engine) models
 - 8 January 2025: Release of [a free, public, shared pool](/docs/docs/public_llm_pool.md) for community LLM deployment
@@ -129,20 +130,24 @@ Not what you were looking for? [Tell us](https://github.com/kalavai-net/kalavai-
 
 ## Getting started
 
-The `kalavai` client is the main tool to interact with the Kalavai platform, to create and manage both local and public pools and also to interact with them (e.g. deploy models). Let's go over its installation. 
+The `kalavai-client` is the main tool to interact with the Kalavai platform, to create and manage both local and public pools and also to interact with them (e.g. deploy models). Let's go over its installation. 
 
-From release **v0.5.0, you can now install `kalavai` client in non-worker computers**. You can run a pool on a set of machines and have the client on a remote computer from which you access the LLM pool. Because the client only requires having python installed, this means more computers are now supported to run it.
+From release **v0.5.0, you can now install `kalavai-client` in non-worker computers**. You can run a pool on a set of machines and have the client on a remote computer from which you access the LLM pool. Because the client only requires having python installed, this means more computers are now supported to run it.
 
 
-### Requirements for a worker machine
+### Requirements
+
+For workers sharing resources with the pool:
 
 - A laptop, desktop or Virtual Machine
 - Docker engine installed (for [linux](https://docs.docker.com/engine/install/), [Windows and MacOS](https://docs.docker.com/desktop/)) with [privilege access](https://docs.docker.com/engine/containers/run/#runtime-privilege-and-linux-capabilities).
 
+> **Support for Windows and MacOS workers is experimental**: kalavai workers run on docker containers that require access to the host network interfaces, thus systems that do not support containers natively (Windows and MacOS) may have difficulties finding each other.
+
+Any system that runs python 3.6+ is able to run the `kalavai-client` and therefore connect and operate an LLM pool, [without sharing with the pool](). Your computer won't be adding its capacity to the pool, but it wil be able to deploy jobs and interact with models.
 
-### Requirements to run the client
 
-- Python 3.10+
+#### Common issues
 
 If you see the following error:
 
@@ -211,6 +216,17 @@ Copy the joining token. On the worker node, run:
 kalavai pool join <token>
 ```
 
+### 3. Attach more clients
+
+You can now connect to an existing pool from any computer -not just from worker nodes. To connect to a pool, run:
+
+```bash
+kalavai pool attach <token>
+```
+
+This won't add the machine as a worker, but you will be able to operate in the pool as if you were. This is ideal for remote access to the pool, and to use the pool from machines that cannot run workers (docker container limitations).
+
+
 ### Enough already, let's run stuff!
 
 Check our [examples](examples/) to put your new AI pool to good use!
@@ -275,7 +291,7 @@ Anything missing here? Give us a shout in the [discussion board](https://github.
 
 ### Requirements
 
-Python version <= 3.12.
+Python version >= 3.6.
 
 ```bash
 sudo add-apt-repository ppa:deadsnakes/ppa

{kalavai_client-0.5.8 → kalavai_client-0.5.10}/README.md

@@ -46,6 +46,7 @@ https://github.com/user-attachments/assets/0d2316f3-79ea-46ac-b41e-8ef720f52672
 
 ### News updates
 
+- 31 January 2025: `kalavai-client` is now a [PyPI package](https://pypi.org/project/kalavai-client/), easier to install than ever!
 - 27 January 2025: Support for accessing pools from remote computers
 - 9 January 2025: Added support for [Aphrodite Engine](https://github.com/aphrodite-engine/aphrodite-engine) models
 - 8 January 2025: Release of [a free, public, shared pool](/docs/docs/public_llm_pool.md) for community LLM deployment
@@ -86,20 +87,24 @@ Not what you were looking for? [Tell us](https://github.com/kalavai-net/kalavai-
 
 ## Getting started
 
-The `kalavai` client is the main tool to interact with the Kalavai platform, to create and manage both local and public pools and also to interact with them (e.g. deploy models). Let's go over its installation. 
+The `kalavai-client` is the main tool to interact with the Kalavai platform, to create and manage both local and public pools and also to interact with them (e.g. deploy models). Let's go over its installation. 
 
-From release **v0.5.0, you can now install `kalavai` client in non-worker computers**. You can run a pool on a set of machines and have the client on a remote computer from which you access the LLM pool. Because the client only requires having python installed, this means more computers are now supported to run it.
+From release **v0.5.0, you can now install `kalavai-client` in non-worker computers**. You can run a pool on a set of machines and have the client on a remote computer from which you access the LLM pool. Because the client only requires having python installed, this means more computers are now supported to run it.
 
 
-### Requirements for a worker machine
+### Requirements
+
+For workers sharing resources with the pool:
 
 - A laptop, desktop or Virtual Machine
 - Docker engine installed (for [linux](https://docs.docker.com/engine/install/), [Windows and MacOS](https://docs.docker.com/desktop/)) with [privilege access](https://docs.docker.com/engine/containers/run/#runtime-privilege-and-linux-capabilities).
 
+> **Support for Windows and MacOS workers is experimental**: kalavai workers run on docker containers that require access to the host network interfaces, thus systems that do not support containers natively (Windows and MacOS) may have difficulties finding each other.
+
+Any system that runs python 3.6+ is able to run the `kalavai-client` and therefore connect and operate an LLM pool, [without sharing with the pool](). Your computer won't be adding its capacity to the pool, but it wil be able to deploy jobs and interact with models.
 
-### Requirements to run the client
 
-- Python 3.10+
+#### Common issues
 
 If you see the following error:
 
@@ -168,6 +173,17 @@ Copy the joining token. On the worker node, run:
 kalavai pool join <token>
 ```
 
+### 3. Attach more clients
+
+You can now connect to an existing pool from any computer -not just from worker nodes. To connect to a pool, run:
+
+```bash
+kalavai pool attach <token>
+```
+
+This won't add the machine as a worker, but you will be able to operate in the pool as if you were. This is ideal for remote access to the pool, and to use the pool from machines that cannot run workers (docker container limitations).
+
+
 ### Enough already, let's run stuff!
 
 Check our [examples](examples/) to put your new AI pool to good use!
@@ -232,7 +248,7 @@ Anything missing here? Give us a shout in the [discussion board](https://github.
 
 ### Requirements
 
-Python version <= 3.12.
+Python version >= 3.6.
 
 ```bash
 sudo add-apt-repository ppa:deadsnakes/ppa

kalavai_client-0.5.10/kalavai_client/__init__.py

@@ -0,0 +1,2 @@
+
+__version__ = "0.5.10"

{kalavai_client-0.5.8 → kalavai_client-0.5.10}/kalavai_client/assets/apps.yaml

@@ -189,13 +189,13 @@ releases:
       value: "1"
     - name: devicePlugin.deviceSplitCount
       value: "1"
-    - name: scheduler.customWebhook.port
-      value: "30498"
-    - name: scheduler.service.schedulerPort
-      value: "30498"
-    - name: scheduler.service.monitorPort
-      value: "30493"
-    - name: devicePlugin.service.httpPort
-      value: "30492"
+    # - name: scheduler.customWebhook.port
+    #   value: "30498"
+    # - name: scheduler.service.schedulerPort
+    #   value: "30498"
+    # - name: scheduler.service.monitorPort
+    #   value: "30493"
+    # - name: devicePlugin.service.httpPort
+    #   value: "30492"
     
 

{kalavai_client-0.5.8 → kalavai_client-0.5.10}/kalavai_client/assets/docker-compose-template.yaml

@@ -14,14 +14,13 @@ services:
     # - "6443:6443" # kube server
     # - "10250:10250" # worker balancer
     # - "8472:8472/udp" # flannel vxlan
-    # - "51820:51820/udp" # flannel wireguard
+    # - "51820-51830:51820-51830" # flannel wireguard
     # {% if command == "server" %}
     # - "30000-30500:30000-30500"
     # {% endif %}
     environment:
       - HOST_NAME={{node_name}}
       - IFACE_NAME={{flannel_iface}}
-      - PORT=51820
       - TOKEN={{vpn_token}}
     volumes:
       - /dev/net/tun:/dev/net/tun
@@ -36,6 +35,9 @@ services:
 #     volumes:
 #       - {{nginx_path}}/nginx.conf:/etc/nginx/nginx.conf
 {% endif %}
+
+# run worker only if command is set
+{%if command %}
   {{service_name}}:
     image: docker.io/bundenth/kalavai-runner:gpu-latest
     container_name: {{service_name}}
@@ -44,24 +46,29 @@ services:
     - {{vpn_name}}
     network_mode: "service:{{vpn_name}}"
   {% else %}
-    hostname: {{node_name}}
-    networks:
-    - custom-network
-    ports:
-    - "6443:6443" # kube server
-    - "10250:10250" # worker balancer
-    - "8472:8472" # flannel vxlan
-    - "51820:51820" # flannel wireguard
-    {% if command == "server" %}
-    - "30000-30500:30000-30500"
-    {% endif %}
+    network_mode: host
+    # hostname: {{node_name}}
+    # networks:
+    # - custom-network
+    # ports:
+    # - "6443:6443" # kube server
+    # - "2379-2380:2379-2380" # etcd server
+    # - "10259:10259" # kube scheduler
+    # - "10257:10257" # kube controller manager
+    # - "10250:10250" # worker balancer
+    # - "8285:8285" # flannel
+    # - "8472:8472" # flannel vxlan
+    # - "51820:51820" # flannel wireguard
+    # {% if command == "server" %}
+    # - "30000-32767:30000-32767"
+    # {% endif %}
   {% endif %}
     privileged: true
     restart: unless-stopped
     command: >
       --command={{command}}
       {% if command == "server" %}
-      --port_range="30000-30500"
+      --port_range="30000-32767"
       {% else %}
       --server_ip={{pool_ip}}
       --token={{pool_token}}
@@ -82,7 +89,7 @@ services:
     - {{k3s_path}}:/var/lib/rancher/k3s # Persist data
     - {{etc_path}}:/etc/rancher/k3s # Config files
 
-{% if num_gpus and num_gpus > 0 %}
+  {% if num_gpus and num_gpus > 0 %}
     deploy:
       resources:
         reservations:
@@ -90,8 +97,10 @@ services:
           - driver: nvidia
             count: {{num_gpus}}
             capabilities: [gpu]
+  {% endif %}
 {% endif %}
 
 networks:
   custom-network:
-    driver: bridge
+    driver: bridge
+

{kalavai_client-0.5.8 → kalavai_client-0.5.10}/kalavai_client/cli.py

@@ -118,27 +118,6 @@ CLUSTER = dockerCluster(
 ######################
 ## HELPER FUNCTIONS ##
 ######################
-
-def check_vpn_compatibility():
-    """Check required packages to join VPN"""
-    logs = []
-    console.log("[white]Checking system requirements...")
-    # netclient
-    try:
-        run_cmd("sudo netclient version >/dev/null 2>&1")
-    except:
-        logs.append("[red]Netmaker not installed. Install instructions:\n")
-        logs.append("   Linux: https://docs.netmaker.io/docs/netclient#linux\n")
-        logs.append("   Windows: https://docs.netmaker.io/docs/netclient#windows\n")
-        logs.append("   MacOS: https://docs.netmaker.io/docs/netclient#mac\n")
-    
-    if len(logs) == 0:
-        console.log("[green]System is ready to join the vpn")
-        return True
-    else:
-        for log in logs:
-            console.log(log)
-        return False
     
 def check_seed_compatibility():
     """Check required packages to start pools"""
@@ -358,7 +337,7 @@ def select_token_type():
             break
     return {"admin": choice == 0, "user": choice == 1, "worker": choice == 2}
 
-def generate_compose_config(role, node_name, node_labels, is_public, pool_ip=None, vpn_token=None, pool_token=None):
+def generate_compose_config(role, node_name, is_public, node_labels=None, pool_ip=None, vpn_token=None, pool_token=None):
     num_gpus = 0
     try:
         has_gpus = check_gpu_drivers()
@@ -370,6 +349,8 @@ def generate_compose_config(role, node_name, node_labels, is_public, pool_ip=Non
             ) 
     except:
         console.log(f"[red]WARNING: error when fetching NVIDIA GPU info. GPUs will not be used on this local machine")
+    if node_labels is not None:
+        node_labels = " ".join([f"--node-label {key}={value}" for key, value in node_labels.items()])
     compose_values = {
         "user_path": user_path(""),
         "service_name": DEFAULT_CONTAINER_NAME,
@@ -384,7 +365,7 @@ def generate_compose_config(role, node_name, node_labels, is_public, pool_ip=Non
         "num_gpus": num_gpus,
         "k3s_path": f"{CONTAINER_HOST_PATH}/k3s",
         "etc_path": f"{CONTAINER_HOST_PATH}/etc",
-        "node_labels": " ".join([f"--node-label {key}={value}" for key, value in node_labels.items()]),
+        "node_labels": node_labels,
         "flannel_iface": DEFAULT_FLANNEL_IFACE if is_public else ""
     }
     # generate local config files
@@ -1121,6 +1102,7 @@ def pool__attach(token, *others, node_name=None):
     """
     Set creds in token on the local instance
     """
+    # check that is not attached to another instance
     if os.path.exists(USER_LOCAL_SERVER_FILE):
         option = user_confirm(
             question="You seem to be connected to an instance already. Are you sure you want to join a new one?",
@@ -1129,34 +1111,39 @@ def pool__attach(token, *others, node_name=None):
         if option == 0:
             console.log("[green]Nothing happened.")
             return
+    
+    # check token
+    if not pool__check_token(token):
+        return
+
     try:
         data = decode_dict(token)
         kalavai_seed_ip = data[CLUSTER_IP_KEY]
-        kalavai_token = data[CLUSTER_TOKEN_KEY]
         cluster_name = data[CLUSTER_NAME_KEY]
         auth_key = data[AUTH_KEY]
         watcher_service = data[WATCHER_SERVICE_KEY]
         public_location = data[PUBLIC_LOCATION_KEY]
-    except:
-        console.log("[red]Error when parsing token. Invalid token")
+        vpn = defaultdict(lambda: None)
+    except Exception as e:
+        console.log(str(e))
+        console.log("[red] Invalid token")
         return
-
+    
     user = defaultdict(lambda: None)
     if public_location is not None:
-        console.log("Joining private network")
+        user = user_login(user_cookie=USER_COOKIE)
+        if user is None:
+            console.log("[red]Must be logged in to join public pools. Run [yellow]kalavai login[red] to authenticate")
+            exit()
+        console.log("Fetching VPN credentials")
         try:
-            if not check_vpn_compatibility():
-                return
-            vpn = join_vpn(
+            vpn = get_vpn_details(
                 location=public_location,
                 user_cookie=USER_COOKIE)
-            user = user_login(user_cookie=USER_COOKIE)
-            time.sleep(5)
         except Exception as e:
             console.log(f"[red]Error when joining network: {str(e)}")
             console.log("Are you authenticated? Try [yellow]kalavai login")
             return
-        # validate public seed
         try:
             validate_join_public_seed(
                 cluster_name=cluster_name,
@@ -1165,9 +1152,19 @@ def pool__attach(token, *others, node_name=None):
             )
         except Exception as e:
             console.log(f"[red]Error when joining network: {str(e)}")
-            leave_vpn(container_name=DEFAULT_VPN_CONTAINER_NAME)
             return
-
+        
+    # local agent join
+    # 1. Generate local cache files
+    console.log("Generating config files...")
+    
+    # Generate docker compose recipe
+    generate_compose_config(
+        role="",
+        vpn_token=vpn["key"],
+        node_name=node_name,
+        is_public=public_location is not None)
+    
     store_server_info(
         server_ip=kalavai_seed_ip,
         auth_key=auth_key,
@@ -1178,7 +1175,26 @@ def pool__attach(token, *others, node_name=None):
         public_location=public_location,
         user_api_key=user["api_key"])
 
-    console.log(f"[green]You are now connected to {cluster_name} @ {kalavai_seed_ip}")
+    option = user_confirm(
+        question="Docker compose ready. Would you like Kalavai to deploy it?",
+        options=["no", "yes"]
+    )
+    if option == 0:
+        console.log("Manually deploy the worker with the following command:\n")
+        print(f"docker compose -f {USER_COMPOSE_FILE} up -d")
+        return
+    
+    console.log(f"[white] Connecting to {cluster_name} @ {kalavai_seed_ip} (this may take a few minutes)...")
+    run_cmd(f"docker compose -f {USER_COMPOSE_FILE} up -d")
+    # ensure we are connected
+    while True:
+        console.log("Waiting for core services to be ready, may take a few minutes...")
+        time.sleep(30)
+        if is_watcher_alive(server_creds=USER_LOCAL_SERVER_FILE, user_cookie=USER_COOKIE):
+            break
+    
+    # set status to schedulable
+    console.log(f"[green] You are connected to {cluster_name}")
 
 
 @arguably.command

{kalavai_client-0.5.8 → kalavai_client-0.5.10}/kalavai_client/utils.py

@@ -389,7 +389,7 @@ def resource_path(relative_path: str):
         last_slash = relative_path.rfind("/") 
         path = relative_path[:last_slash].replace("/", ".")
         filename = relative_path[last_slash+1:]
-        resource = importlib.resources.path(path, filename)
+        resource = str(importlib.resources.files(path).joinpath(filename))
     except Exception as e:
         return None
     return resource

{kalavai_client-0.5.8 → kalavai_client-0.5.10}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name            = "kalavai-client"
-version         = "0.5.8"
+version         = "0.5.10"
 authors = [
   {name = "Carlos Fernandez Musoles", email = "carlos@kalavai.net"}
 ]
@@ -12,7 +12,7 @@ license         = "Apache-2.0"
 license-files   = ["LICENSE"]
 keywords        = ["LLM", "platform"]
 readme          = {file = "README.md", content-type = "text/markdown"}
-requires-python = "<3.12"
+requires-python = ">=3.4"
 dependencies = [
     "requests>= 2.25",
     "psutil==5.9.8",

kalavai_client-0.5.8/kalavai_client/__init__.py

@@ -1,2 +0,0 @@
-
-__version__ = "0.5.8"