cluster-builder 0.3.1__tar.gz → 0.3.3__tar.gz

{cluster_builder-0.3.1 → cluster_builder-0.3.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cluster-builder
-Version: 0.3.1
+Version: 0.3.3
 Summary: Swarmchestrate cluster builder
 Author-email: Gunjan <G.Kotak@westminster.ac.uk>, Jay <J.Deslauriers@westminster.ac.uk>
 License: Apache2
@@ -16,7 +16,7 @@ Dynamic: license-file
 
 # Swarmchestrate - Cluster Builder
 
-This repository contains the codebase for **[cluster-builder]**, which builds K3s clusters for Swarmchestrate using OpenTofu.  
+This repository contains the codebase for **cluster-builder**, which builds K3s clusters for Swarmchestrate using OpenTofu.  
 
 Key features:
 - **Create**: Provisions infrastructure using OpenTofu and installs K3s.
@@ -33,11 +33,10 @@ Before proceeding, ensure the following prerequisites are installed:
 1. **Git**: For cloning the repository.
 2. **Python**: Version 3.9 or higher.
 3. **pip**: Python package manager.
-4. **OpenTofu**: Version 1.6 or higher for infrastructure provisioning.
-6. **Make**: To run the provided `Makefile`.
-7. **PostgreSQL**: For storing OpenTofu state.
-8. (Optional) **Docker**: To create a dev Postgres
-9. For detailed instructions on **edge device requirements**, refer to the [Edge Device Requirements](docs/edge-requirements.md) document.
+4. **Make**: To run the provided `Makefile`.
+5. **PostgreSQL**: For storing OpenTofu state.
+6. (Optional) **Docker**: To create a dev Postgres
+7. For detailed instructions on **edge device requirements**, refer to the [Edge Device Requirements](docs/edge-requirements.md) document.
 
 ---
 
@@ -142,16 +141,21 @@ orchestrator = Swarmchestrate(
 To create a new k3s cluster, use the **add_node** method with the **master** role:
 
 ```python
-# Configuration for a new cluster
+# Configuration for a new cluster using aws provider
 config = {
-    "cloud": "aws",  # Can be 'aws', 'openstack', or 'edge'
-    "k3s_role": "master",  # Role can be 'master', 'worker', or 'ha'
+    "cloud": "aws", 
+    "k3s_role": "master", 
     "ha": False,  # Set to True for high availability (HA) deployments
     "instance_type": "t2.small",  # AWS instance type
     "ssh_key_name": "g",  # SSH key name for AWS or OpenStack
     "ssh_user": "ec2-user",  # SSH user for the instance
     "ssh_private_key_path": "/workspaces/cluster-builder/scripts/g.pem",  # Path to SSH private key
     "ami": "ami-0c0493bbac867d427",  # AMI ID for AWS (specific to region)
+    # Optional parameters
+    # If existing SG is specified, it will be used directly with no port changes
+    "security_group_id": "sg-0123456789abcdef0",
+    # No security_group_id means a new SG will be created and these ports applied as rules
+    # These ports will be used ONLY if creating a new SG
     "tcp_ports": [10020],  # Optional list of TCP ports to open
     "udp_ports": [1003]  # Optional list of UDP ports to open
 }
@@ -161,27 +165,33 @@ cluster_name = orchestrator.add_node(config)
 print(f"Created cluster: {cluster_name}")
 ```
 
+Note: Fetch the outputs from the master node and use them when adding a worker node.
+
 ### Adding Nodes to an Existing Cluster
 
 To add worker or high-availability nodes to an existing cluster:
 
 ```python
-# Configuration for adding a worker node
+# Configuration for adding a worker node using aws provider
 worker_config = {
-    "cloud": "aws",  # Cloud provider (can be 'aws', 'openstack', or 'edge')
+    "cloud": "aws",
     "k3s_role": "worker",  # Role can be 'worker' or 'ha'
-    "ha": False,  # Set to True for high availability (HA) deployments
     "instance_type": "t2.small",  # AWS instance type
     "ssh_key_name": "g",  # SSH key name
     "ssh_user": "ec2-user",  # SSH user for the instance
     "ssh_private_key_path": "/workspaces/cluster-builder/scripts/g.pem",  # Path to SSH private key
     "ami": "ami-0c0493bbac867d427",  # AMI ID for AWS
-    # Optional parameters:
-    # "master_ip": "12.13.14.15",  # IP address of the master node (required for worker/HA roles)
-    # "cluster_name": "elastic_mcnulty",  # Name of the cluster
-    # "security_group_id": "sg-xxxxxxxxxxxxxxx",  # Security group ID for AWS or OpenStack
-    # "tcp_ports": [80, 443],  # List of TCP ports to open
-    # "udp_ports": [53]  # List of UDP ports to open
+    # Additional parameters obtained after deploying the master node:
+    "master_ip": "12.13.14.15",  # IP address of the master node (required for worker/HA roles)
+    "cluster_name": "elastic_mcnulty",  # Name of the cluster
+    "k3s_token": "G4lm7wEaFuCCygeU", # Token of the cluster
+    # Optional parameters
+    # If existing SG is specified, it will be used directly with no port changes
+    "security_group_id": "sg-0123456789abcdef0",
+    # No security_group_id means a new SG will be created and these ports applied as rules
+    # These ports will be used ONLY if creating a new SG
+    "tcp_ports": [10020],  # Optional list of TCP ports to open
+    "udp_ports": [1003]  # Optional list of UDP ports to open
 }
 
 # Add the worker node
@@ -226,17 +236,25 @@ Note for **Edge Devices**:
 Since the edge device is already provisioned, the `destroy` method will not remove K3s directly from the edge device. You will need to manually uninstall K3s from your edge device after the cluster is destroyed.
 
 ---
+### Deploying Manifests
 
-### Important Configuration Requirements
-#### High Availability Flag (ha):
+The deploy_manifests method copies Kubernetes manifests to the target cluster node.
 
-- For k3s_role="worker" or k3s_role="ha", you must specify a master_ip (the IP address of the master node).
+```python
+orchestrator.deploy_manifests(
+    manifest_folder="path/to/manifests",
+    master_ip="MASTER_NODE_IP",
+    ssh_key_path="path/to/key.pem",
+    ssh_user="USERNAME"
+)
+```
 
-- For k3s_role="master", you must not specify a master_ip.
+## Important Configuration Requirements
+### High Availability Flag (ha):
 
 - The ha flag should be set to True for high availability deployment (usually when adding a ha or worker node to an existing master).
 
-#### SSH Credentials:
+### SSH Credentials:
 
 - For all roles (k3s_role="master", k3s_role="worker", k3s_role="ha"), you must specify both ssh_user and ssh_private_key_path except for edge.
 
@@ -244,7 +262,7 @@ Since the edge device is already provisioned, the `destroy` method will not remo
 
 - The ssh_key_name and the ssh_private_key_path are different—ensure that your SSH key is placed correctly at the provided ssh_private_key_path.
 
-#### Ports:
+### Ports:
 You can specify custom ports for your nodes in the tcp_ports and udp_ports fields. However, certain ports are required for Kubernetes deployment (even if not specified explicitly):
 
 **TCP Ports:**
@@ -263,7 +281,7 @@ You can specify custom ports for your nodes in the tcp_ports and udp_ports field
 - 8472: VXLAN for Flannel
 - 53: DNS
 
-#### OpenStack:
+### OpenStack:
 When provisioning on OpenStack, you should provide the value for 'floating_ip_pool' from which floating IPs can be allocated for the instance. If not specified, OpenTofu will not assign floating IP.
 
 ---

{cluster_builder-0.3.1 → cluster_builder-0.3.3}/README.md

@@ -1,6 +1,6 @@
 # Swarmchestrate - Cluster Builder
 
-This repository contains the codebase for **[cluster-builder]**, which builds K3s clusters for Swarmchestrate using OpenTofu.  
+This repository contains the codebase for **cluster-builder**, which builds K3s clusters for Swarmchestrate using OpenTofu.  
 
 Key features:
 - **Create**: Provisions infrastructure using OpenTofu and installs K3s.
@@ -17,11 +17,10 @@ Before proceeding, ensure the following prerequisites are installed:
 1. **Git**: For cloning the repository.
 2. **Python**: Version 3.9 or higher.
 3. **pip**: Python package manager.
-4. **OpenTofu**: Version 1.6 or higher for infrastructure provisioning.
-6. **Make**: To run the provided `Makefile`.
-7. **PostgreSQL**: For storing OpenTofu state.
-8. (Optional) **Docker**: To create a dev Postgres
-9. For detailed instructions on **edge device requirements**, refer to the [Edge Device Requirements](docs/edge-requirements.md) document.
+4. **Make**: To run the provided `Makefile`.
+5. **PostgreSQL**: For storing OpenTofu state.
+6. (Optional) **Docker**: To create a dev Postgres
+7. For detailed instructions on **edge device requirements**, refer to the [Edge Device Requirements](docs/edge-requirements.md) document.
 
 ---
 
@@ -126,16 +125,21 @@ orchestrator = Swarmchestrate(
 To create a new k3s cluster, use the **add_node** method with the **master** role:
 
 ```python
-# Configuration for a new cluster
+# Configuration for a new cluster using aws provider
 config = {
-    "cloud": "aws",  # Can be 'aws', 'openstack', or 'edge'
-    "k3s_role": "master",  # Role can be 'master', 'worker', or 'ha'
+    "cloud": "aws", 
+    "k3s_role": "master", 
     "ha": False,  # Set to True for high availability (HA) deployments
     "instance_type": "t2.small",  # AWS instance type
     "ssh_key_name": "g",  # SSH key name for AWS or OpenStack
     "ssh_user": "ec2-user",  # SSH user for the instance
     "ssh_private_key_path": "/workspaces/cluster-builder/scripts/g.pem",  # Path to SSH private key
     "ami": "ami-0c0493bbac867d427",  # AMI ID for AWS (specific to region)
+    # Optional parameters
+    # If existing SG is specified, it will be used directly with no port changes
+    "security_group_id": "sg-0123456789abcdef0",
+    # No security_group_id means a new SG will be created and these ports applied as rules
+    # These ports will be used ONLY if creating a new SG
     "tcp_ports": [10020],  # Optional list of TCP ports to open
     "udp_ports": [1003]  # Optional list of UDP ports to open
 }
@@ -145,27 +149,33 @@ cluster_name = orchestrator.add_node(config)
 print(f"Created cluster: {cluster_name}")
 ```
 
+Note: Fetch the outputs from the master node and use them when adding a worker node.
+
 ### Adding Nodes to an Existing Cluster
 
 To add worker or high-availability nodes to an existing cluster:
 
 ```python
-# Configuration for adding a worker node
+# Configuration for adding a worker node using aws provider
 worker_config = {
-    "cloud": "aws",  # Cloud provider (can be 'aws', 'openstack', or 'edge')
+    "cloud": "aws",
     "k3s_role": "worker",  # Role can be 'worker' or 'ha'
-    "ha": False,  # Set to True for high availability (HA) deployments
     "instance_type": "t2.small",  # AWS instance type
     "ssh_key_name": "g",  # SSH key name
     "ssh_user": "ec2-user",  # SSH user for the instance
     "ssh_private_key_path": "/workspaces/cluster-builder/scripts/g.pem",  # Path to SSH private key
     "ami": "ami-0c0493bbac867d427",  # AMI ID for AWS
-    # Optional parameters:
-    # "master_ip": "12.13.14.15",  # IP address of the master node (required for worker/HA roles)
-    # "cluster_name": "elastic_mcnulty",  # Name of the cluster
-    # "security_group_id": "sg-xxxxxxxxxxxxxxx",  # Security group ID for AWS or OpenStack
-    # "tcp_ports": [80, 443],  # List of TCP ports to open
-    # "udp_ports": [53]  # List of UDP ports to open
+    # Additional parameters obtained after deploying the master node:
+    "master_ip": "12.13.14.15",  # IP address of the master node (required for worker/HA roles)
+    "cluster_name": "elastic_mcnulty",  # Name of the cluster
+    "k3s_token": "G4lm7wEaFuCCygeU", # Token of the cluster
+    # Optional parameters
+    # If existing SG is specified, it will be used directly with no port changes
+    "security_group_id": "sg-0123456789abcdef0",
+    # No security_group_id means a new SG will be created and these ports applied as rules
+    # These ports will be used ONLY if creating a new SG
+    "tcp_ports": [10020],  # Optional list of TCP ports to open
+    "udp_ports": [1003]  # Optional list of UDP ports to open
 }
 
 # Add the worker node
@@ -210,17 +220,25 @@ Note for **Edge Devices**:
 Since the edge device is already provisioned, the `destroy` method will not remove K3s directly from the edge device. You will need to manually uninstall K3s from your edge device after the cluster is destroyed.
 
 ---
+### Deploying Manifests
 
-### Important Configuration Requirements
-#### High Availability Flag (ha):
+The deploy_manifests method copies Kubernetes manifests to the target cluster node.
 
-- For k3s_role="worker" or k3s_role="ha", you must specify a master_ip (the IP address of the master node).
+```python
+orchestrator.deploy_manifests(
+    manifest_folder="path/to/manifests",
+    master_ip="MASTER_NODE_IP",
+    ssh_key_path="path/to/key.pem",
+    ssh_user="USERNAME"
+)
+```
 
-- For k3s_role="master", you must not specify a master_ip.
+## Important Configuration Requirements
+### High Availability Flag (ha):
 
 - The ha flag should be set to True for high availability deployment (usually when adding a ha or worker node to an existing master).
 
-#### SSH Credentials:
+### SSH Credentials:
 
 - For all roles (k3s_role="master", k3s_role="worker", k3s_role="ha"), you must specify both ssh_user and ssh_private_key_path except for edge.
 
@@ -228,7 +246,7 @@ Since the edge device is already provisioned, the `destroy` method will not remo
 
 - The ssh_key_name and the ssh_private_key_path are different—ensure that your SSH key is placed correctly at the provided ssh_private_key_path.
 
-#### Ports:
+### Ports:
 You can specify custom ports for your nodes in the tcp_ports and udp_ports fields. However, certain ports are required for Kubernetes deployment (even if not specified explicitly):
 
 **TCP Ports:**
@@ -247,7 +265,7 @@ You can specify custom ports for your nodes in the tcp_ports and udp_ports field
 - 8472: VXLAN for Flannel
 - 53: DNS
 
-#### OpenStack:
+### OpenStack:
 When provisioning on OpenStack, you should provide the value for 'floating_ip_pool' from which floating IPs can be allocated for the instance. If not specified, OpenTofu will not assign floating IP.
 
 ---

{cluster_builder-0.3.1 → cluster_builder-0.3.3}/cluster_builder/config/cluster.py

@@ -131,9 +131,12 @@ class ClusterConfig:
         logger.debug(f"Cluster directory: {cluster_dir}")
 
         # Generate a resource name
-        random_name = self.generate_random_name()
-        prepared_config["resource_name"] = f"{cloud}-{random_name}"
-        logger.debug(f"Resource name: {prepared_config['resource_name']}")
+        if "resource_name" not in prepared_config:
+            random_name = self.generate_random_name()
+            prepared_config["resource_name"] = f"{cloud}-{random_name}"
+            logger.debug(f"Resource name: {prepared_config['resource_name']}")
+        else:
+            logger.debug(f" USing provded Resource name: {prepared_config['resource_name']}")
 
         # Create the cluster directory
         try:

{cluster_builder-0.3.1 → cluster_builder-0.3.3}/cluster_builder/swarmchestrate.py

@@ -217,7 +217,7 @@ class Swarmchestrate:
         outputs_file = os.path.join(cluster_dir, "outputs.tf")
         
         # Define common output names
-        output_names = ["cluster_name", "master_ip", "worker_ip", "ha_ip", "k3s_token", "node_name"]
+        output_names = ["cluster_name", "master_ip", "worker_ip", "ha_ip", "k3s_token", "resource_name"]
         
         # Include additional outputs based on the cloud type
         if "aws" in cluster_dir:
@@ -234,9 +234,9 @@ class Swarmchestrate:
         try:
             self.deploy(cluster_dir, dryrun)
             cluster_name = prepared_config["cluster_name"]
-            node_name = prepared_config["resource_name"]
+            resource_name = prepared_config["resource_name"]
             logger.info(
-                f"✅ Successfully added '{node_name}' for cluster '{cluster_name}'"
+                f"✅ Successfully added '{resource_name}' for cluster '{cluster_name}'"
             )
             # Run 'tofu output -json' to get outputs
             result = subprocess.run(
@@ -256,7 +256,7 @@ class Swarmchestrate:
                 "k3s_token": outputs.get("k3s_token", {}).get("value"),
                 "worker_ip": outputs.get("worker_ip", {}).get("value"),
                 "ha_ip": outputs.get("ha_ip", {}).get("value"),
-                "node_name": outputs.get("node_name", {}).get("value")
+                "resource_name": outputs.get("resource_name", {}).get("value")
             }
             # Add cloud-specific output
             if "aws" in cluster_dir:
@@ -520,7 +520,7 @@ class Swarmchestrate:
                 if connection:
                     connection.close()
 
-    def run_copy_manifests_tf(
+    def deploy_manifests(
         self,
         manifest_folder: str,
         master_ip: str,
@@ -528,7 +528,7 @@ class Swarmchestrate:
         ssh_user: str,
     ):
         """
-        Copy and apply manifests to a cluster using copy_manifest.tf in a separate folder.
+        Copy and apply manifests to a cluster using copy_manifest.tf in a temporaryfolder.
 
         Args:
             manifest_folder: Path to local manifest folder
@@ -539,49 +539,55 @@ class Swarmchestrate:
         # Dedicated folder for copy-manifest operations
         copy_dir = Path(self.output_dir) / "copy-manifest"
         copy_dir.mkdir(parents=True, exist_ok=True)
-        logger.info(f"Using separate copy-manifest folder: {copy_dir}")
 
-        # Copy copy_manifest.tf from templates
-        tf_source_file = Path(self.template_manager.templates_dir) / "copy_manifest.tf"
-        if not tf_source_file.exists():
-            logger.error(f"copy_manifest.tf not found at: {tf_source_file}")
-            raise RuntimeError(f"copy_manifest.tf not found at: {tf_source_file}")
-        shutil.copy(tf_source_file, copy_dir)
-        logger.info(f"Copied copy_manifest.tf to {copy_dir}")
-
-        # Prepare environment for OpenTofu
-        env_vars = os.environ.copy()
-        env_vars["TF_LOG"] = os.getenv("TF_LOG", "INFO")
-        env_vars["TF_LOG_PATH"] = os.getenv("TF_LOG_PATH", "/tmp/opentofu.log")
+        logger.debug(f"Using copy-manifest folder: {copy_dir}")
 
         try:
-            # Initialize OpenTofu in the separate folder
-            logger.info(f"Initializing OpenTofu in {copy_dir}...")
-            subprocess.run(["tofu", "init"], cwd=copy_dir, check=True, env=env_vars)
-
-            # Apply the copy-manifest resource
-            logger.info(f"Applying copy-manifest resource in {copy_dir}...")
-            cmd = [
-                "tofu",
-                "apply",
-                "-auto-approve",
-                f"-var=manifest_folder={manifest_folder}",
-                f"-var=master_ip={master_ip}",
-                f"-var=ssh_private_key_path={ssh_key_path}",
-                f"-var=ssh_user={ssh_user}"
-            ]
-            result = subprocess.run(cmd, cwd=copy_dir, check=True, capture_output=True, text=True, env=env_vars)
-
-            # Log output
-            if result.stdout:
-                logger.info(result.stdout)
-            if result.stderr:
-                logger.warning(result.stderr)
-
-            logger.info("✅ Copy-manifest applied successfully on master node.")
+            # Copy copy_manifest.tf from templates
+            tf_source_file = Path(self.template_manager.templates_dir) / "deploy_manifest.tf"
+            if not tf_source_file.exists():
+                logger.debug(f"deploy_manifest.tf not found at: {tf_source_file}")
+                raise RuntimeError(f"deploy_manifest.tf not found at: {tf_source_file}")
+            shutil.copy(tf_source_file, copy_dir)
+            logger.debug(f"Copied copy_manifest.tf to {copy_dir}")
+
+            # Prepare environment for OpenTofu
+            env_vars = os.environ.copy()
+            env_vars["TF_LOG"] = os.getenv("TF_LOG", "INFO")
+            env_vars["TF_LOG_PATH"] = os.getenv("TF_LOG_PATH", "/tmp/opentofu.log")
+
+            logger.info(f"------------ Applying manifest on node: {master_ip} -------------------")
+
+            # Run tofu init with spinner
+            CommandExecutor.run_command(
+                ["tofu", "init"],
+                cwd=str(copy_dir),
+                description="OpenTofu init",
+                env=env_vars,
+            )
 
-        except subprocess.CalledProcessError as e:
-            error_msg = f"Error applying copy-manifest on master {master_ip}: {e.stderr or e.stdout}"
-            logger.error(error_msg)
-            raise RuntimeError(error_msg)
+            # Run tofu apply with spinner
+            CommandExecutor.run_command(
+                [
+                    "tofu",
+                    "apply",
+                    "-auto-approve",
+                    f"-var=manifest_folder={manifest_folder}",
+                    f"-var=master_ip={master_ip}",
+                    f"-var=ssh_private_key_path={ssh_key_path}",
+                    f"-var=ssh_user={ssh_user}"
+                ],
+                cwd=str(copy_dir),
+                description="OpenTofu apply",
+                env=env_vars,
+            )
+
+            logger.info("------------ Successfully applied manifests -------------------")
+
+        except RuntimeError as e:
+            print(f"\n---------- ERROR ----------\n{e}\n")
+            raise
 
+        finally:
+            if copy_dir.exists():
+                shutil.rmtree(copy_dir)

{cluster_builder-0.3.1 → cluster_builder-0.3.3}/cluster_builder/templates/aws/main.tf

@@ -91,7 +91,7 @@ resource "aws_instance" "k3s_node" {
   vpc_security_group_ids = var.security_group_id != "" ? [var.security_group_id] : [aws_security_group.k3s_sg[0].id]
 
   tags = {
-    Name        = "${var.cluster_name}-${var.resource_name}"
+    Name        = "${var.resource_name}"
     ClusterName = var.cluster_name
     Role        = var.k3s_role
   }
@@ -104,7 +104,7 @@ resource "aws_instance" "k3s_node" {
       master_ip    = var.master_ip,
       cluster_name = var.cluster_name,
       public_ip  = self.public_ip,
-      node_name = "${var.cluster_name}-${var.resource_name}"
+      resource_name = "${var.resource_name}"
     })
     destination = "/tmp/k3s_user_data.sh"
   }
@@ -151,6 +151,6 @@ output "instance_status" {
   value = aws_instance.k3s_node.id
 }
 
-output "node_name" {
+output "resource_name" {
   value = aws_instance.k3s_node.tags["Name"]
 }

cluster_builder-0.3.3/cluster_builder/templates/deploy_manifest.tf

@@ -0,0 +1,37 @@
+# main.tf
+
+variable "manifest_folder" {}
+variable "ssh_private_key_path" {}
+variable "master_ip" {}
+variable "ssh_user" {}
+
+resource "null_resource" "copy_manifests" {
+  connection {
+    type        = "ssh"
+    user        = var.ssh_user
+    private_key = file(var.ssh_private_key_path)
+    host        = var.master_ip
+  }
+
+  # Copy the manifest folder into /tmp
+  provisioner "file" {
+    source      = var.manifest_folder
+    destination = "/tmp/"
+  }
+
+  # Apply namespace.yaml first if exists
+  provisioner "remote-exec" {
+    inline = [
+      "folder_name=$(basename ${var.manifest_folder})",
+      "if [ -f /tmp/$folder_name/namespace.yaml ]; then sudo -E KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl apply -f /tmp/$folder_name/namespace.yaml; fi"
+    ]
+  }
+
+  # Apply the rest of the manifests
+  provisioner "remote-exec" {
+    inline = [
+      "folder_name=$(basename ${var.manifest_folder})",
+      "sudo -E KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl apply -R -f /tmp/$folder_name"
+    ]
+  }
+}

{cluster_builder-0.3.1 → cluster_builder-0.3.3}/cluster_builder/templates/edge/main.tf

@@ -30,7 +30,7 @@ data "template_file" "user_data" {
     ha        = var.ha
     public_ip = var.edge_device_ip
     master_ip = var.master_ip
-    node_name = "${var.cluster_name}-${var.resource_name}"
+    resource_name = "${var.resource_name}"
   }
 }
 
@@ -63,7 +63,7 @@ resource "null_resource" "deploy_k3s_edge" {
   }
 
   triggers = {
-    Name          = "K3s-${var.k3s_role}-${var.cluster_name}-${var.resource_name}"
+    Name          = "${var.resource_name}"
     cluster_name  = var.cluster_name
     role          = var.k3s_role
     resource_name = var.resource_name
@@ -93,6 +93,6 @@ output "k3s_token" {
   value = var.k3s_token
 }
 
-output "node_name" {
-  value = "${var.cluster_name}-${var.resource_name}"
+output "resource_name" {
+  value = "var.resource_name}"
 }

{cluster_builder-0.3.1 → cluster_builder-0.3.3}/cluster_builder/templates/ha_user_data.sh.tpl

@@ -21,7 +21,7 @@ log_message "Installing K3s HA Server and joining the cluster..."
 if ! curl -sfL https://get.k3s.io | K3S_TOKEN="${k3s_token}" sh -s - server \
     --server "https://${master_ip}:6443" \
     --node-external-ip="${public_ip}" \
-    --node-name="${node_name}" \
+    --node-name="${resource_name}" \
     --flannel-backend=wireguard-native \
     --flannel-external-ip; then
     log_message "ERROR: K3s server installation failed!"

{cluster_builder-0.3.1 → cluster_builder-0.3.3}/cluster_builder/templates/master_user_data.sh.tpl

@@ -25,10 +25,10 @@ else
     # Templated installation based on HA configuration
     if [[ "${ha}" == "true" ]]; then
         log_message "Installing in HA mode using cluster-init..."
-        curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--cluster-init --node-external-ip=${public_ip} --node-name="${node_name}" --flannel-backend=wireguard-native --flannel-external-ip" K3S_TOKEN="${k3s_token}" sh -s - server
+        curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--cluster-init --node-external-ip=${public_ip} --node-name="${resource_name}" --flannel-backend=wireguard-native --flannel-external-ip" K3S_TOKEN="${k3s_token}" sh -s - server
     else
         log_message "Installing in single-server mode..."
-        curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--node-external-ip=${public_ip} --node-name="${node_name}" --flannel-backend=wireguard-native --flannel-external-ip" K3S_TOKEN="${k3s_token}" sh -s - server
+        curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--node-external-ip=${public_ip} --node-name="${resource_name}" --flannel-backend=wireguard-native --flannel-external-ip" K3S_TOKEN="${k3s_token}" sh -s - server
     fi
 
     log_message "K3s installation completed successfully."

{cluster_builder-0.3.1 → cluster_builder-0.3.3}/cluster_builder/templates/openstack/main.tf

@@ -113,7 +113,7 @@ resource "openstack_networking_port_secgroup_associate_v2" "port_2" {
 resource "openstack_compute_instance_v2" "k3s_node" {
   depends_on = [openstack_networking_port_v2.port_1] 
 
-  name             = "${var.cluster_name}-${var.resource_name}"
+  name             = "${var.resource_name}"
   flavor_name      = var.openstack_flavor_id
   key_pair         = var.ssh_key_name
  # Only add the image_id if block device is NOT used
@@ -136,7 +136,7 @@ resource "openstack_compute_instance_v2" "k3s_node" {
   }
 
   tags = [
-    "${var.cluster_name}-${var.resource_name}",
+    "${var.resource_name}",
     "ClusterName=${var.cluster_name}",
     "Role=${var.k3s_role}"
   ]
@@ -166,7 +166,7 @@ resource "null_resource" "k3s_provision" {
       master_ip    = var.master_ip,
       cluster_name = var.cluster_name,
       public_ip    = openstack_networking_floatingip_v2.floatip_1.address,
-      node_name    = "${var.cluster_name}-${var.resource_name}"
+      resource_name    = "${var.resource_name}"
     })
     destination = "/tmp/k3s_user_data.sh"
   }
@@ -213,6 +213,6 @@ output "instance_power_state" {
   value = openstack_compute_instance_v2.k3s_node.power_state
 }
 
-output "node_name" {
+output "resource_name" {
   value = openstack_compute_instance_v2.k3s_node.name
 }

{cluster_builder-0.3.1 → cluster_builder-0.3.3}/cluster_builder/templates/worker_user_data.sh.tpl

@@ -23,7 +23,7 @@ else
     export K3S_TOKEN="${k3s_token}"
 
     # Install the K3s agent and join the cluster
-    if ! curl -sfL https://get.k3s.io | sh -s - agent --node-external-ip="${public_ip}" --node-name="${node_name}"; then
+    if ! curl -sfL https://get.k3s.io | sh -s - agent --node-external-ip="${public_ip}" --node-name="${resource_name}"; then
         log_message "ERROR: K3s agent installation failed!"
         exit 1
     else

{cluster_builder-0.3.1 → cluster_builder-0.3.3}/cluster_builder.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cluster-builder
-Version: 0.3.1
+Version: 0.3.3
 Summary: Swarmchestrate cluster builder
 Author-email: Gunjan <G.Kotak@westminster.ac.uk>, Jay <J.Deslauriers@westminster.ac.uk>
 License: Apache2
@@ -16,7 +16,7 @@ Dynamic: license-file
 
 # Swarmchestrate - Cluster Builder
 
-This repository contains the codebase for **[cluster-builder]**, which builds K3s clusters for Swarmchestrate using OpenTofu.  
+This repository contains the codebase for **cluster-builder**, which builds K3s clusters for Swarmchestrate using OpenTofu.  
 
 Key features:
 - **Create**: Provisions infrastructure using OpenTofu and installs K3s.
@@ -33,11 +33,10 @@ Before proceeding, ensure the following prerequisites are installed:
 1. **Git**: For cloning the repository.
 2. **Python**: Version 3.9 or higher.
 3. **pip**: Python package manager.
-4. **OpenTofu**: Version 1.6 or higher for infrastructure provisioning.
-6. **Make**: To run the provided `Makefile`.
-7. **PostgreSQL**: For storing OpenTofu state.
-8. (Optional) **Docker**: To create a dev Postgres
-9. For detailed instructions on **edge device requirements**, refer to the [Edge Device Requirements](docs/edge-requirements.md) document.
+4. **Make**: To run the provided `Makefile`.
+5. **PostgreSQL**: For storing OpenTofu state.
+6. (Optional) **Docker**: To create a dev Postgres
+7. For detailed instructions on **edge device requirements**, refer to the [Edge Device Requirements](docs/edge-requirements.md) document.
 
 ---
 
@@ -142,16 +141,21 @@ orchestrator = Swarmchestrate(
 To create a new k3s cluster, use the **add_node** method with the **master** role:
 
 ```python
-# Configuration for a new cluster
+# Configuration for a new cluster using aws provider
 config = {
-    "cloud": "aws",  # Can be 'aws', 'openstack', or 'edge'
-    "k3s_role": "master",  # Role can be 'master', 'worker', or 'ha'
+    "cloud": "aws", 
+    "k3s_role": "master", 
     "ha": False,  # Set to True for high availability (HA) deployments
     "instance_type": "t2.small",  # AWS instance type
     "ssh_key_name": "g",  # SSH key name for AWS or OpenStack
     "ssh_user": "ec2-user",  # SSH user for the instance
     "ssh_private_key_path": "/workspaces/cluster-builder/scripts/g.pem",  # Path to SSH private key
     "ami": "ami-0c0493bbac867d427",  # AMI ID for AWS (specific to region)
+    # Optional parameters
+    # If existing SG is specified, it will be used directly with no port changes
+    "security_group_id": "sg-0123456789abcdef0",
+    # No security_group_id means a new SG will be created and these ports applied as rules
+    # These ports will be used ONLY if creating a new SG
     "tcp_ports": [10020],  # Optional list of TCP ports to open
     "udp_ports": [1003]  # Optional list of UDP ports to open
 }
@@ -161,27 +165,33 @@ cluster_name = orchestrator.add_node(config)
 print(f"Created cluster: {cluster_name}")
 ```
 
+Note: Fetch the outputs from the master node and use them when adding a worker node.
+
 ### Adding Nodes to an Existing Cluster
 
 To add worker or high-availability nodes to an existing cluster:
 
 ```python
-# Configuration for adding a worker node
+# Configuration for adding a worker node using aws provider
 worker_config = {
-    "cloud": "aws",  # Cloud provider (can be 'aws', 'openstack', or 'edge')
+    "cloud": "aws",
     "k3s_role": "worker",  # Role can be 'worker' or 'ha'
-    "ha": False,  # Set to True for high availability (HA) deployments
     "instance_type": "t2.small",  # AWS instance type
     "ssh_key_name": "g",  # SSH key name
     "ssh_user": "ec2-user",  # SSH user for the instance
     "ssh_private_key_path": "/workspaces/cluster-builder/scripts/g.pem",  # Path to SSH private key
     "ami": "ami-0c0493bbac867d427",  # AMI ID for AWS
-    # Optional parameters:
-    # "master_ip": "12.13.14.15",  # IP address of the master node (required for worker/HA roles)
-    # "cluster_name": "elastic_mcnulty",  # Name of the cluster
-    # "security_group_id": "sg-xxxxxxxxxxxxxxx",  # Security group ID for AWS or OpenStack
-    # "tcp_ports": [80, 443],  # List of TCP ports to open
-    # "udp_ports": [53]  # List of UDP ports to open
+    # Additional parameters obtained after deploying the master node:
+    "master_ip": "12.13.14.15",  # IP address of the master node (required for worker/HA roles)
+    "cluster_name": "elastic_mcnulty",  # Name of the cluster
+    "k3s_token": "G4lm7wEaFuCCygeU", # Token of the cluster
+    # Optional parameters
+    # If existing SG is specified, it will be used directly with no port changes
+    "security_group_id": "sg-0123456789abcdef0",
+    # No security_group_id means a new SG will be created and these ports applied as rules
+    # These ports will be used ONLY if creating a new SG
+    "tcp_ports": [10020],  # Optional list of TCP ports to open
+    "udp_ports": [1003]  # Optional list of UDP ports to open
 }
 
 # Add the worker node
@@ -226,17 +236,25 @@ Note for **Edge Devices**:
 Since the edge device is already provisioned, the `destroy` method will not remove K3s directly from the edge device. You will need to manually uninstall K3s from your edge device after the cluster is destroyed.
 
 ---
+### Deploying Manifests
 
-### Important Configuration Requirements
-#### High Availability Flag (ha):
+The deploy_manifests method copies Kubernetes manifests to the target cluster node.
 
-- For k3s_role="worker" or k3s_role="ha", you must specify a master_ip (the IP address of the master node).
+```python
+orchestrator.deploy_manifests(
+    manifest_folder="path/to/manifests",
+    master_ip="MASTER_NODE_IP",
+    ssh_key_path="path/to/key.pem",
+    ssh_user="USERNAME"
+)
+```
 
-- For k3s_role="master", you must not specify a master_ip.
+## Important Configuration Requirements
+### High Availability Flag (ha):
 
 - The ha flag should be set to True for high availability deployment (usually when adding a ha or worker node to an existing master).
 
-#### SSH Credentials:
+### SSH Credentials:
 
 - For all roles (k3s_role="master", k3s_role="worker", k3s_role="ha"), you must specify both ssh_user and ssh_private_key_path except for edge.
 
@@ -244,7 +262,7 @@ Since the edge device is already provisioned, the `destroy` method will not remo
 
 - The ssh_key_name and the ssh_private_key_path are different—ensure that your SSH key is placed correctly at the provided ssh_private_key_path.
 
-#### Ports:
+### Ports:
 You can specify custom ports for your nodes in the tcp_ports and udp_ports fields. However, certain ports are required for Kubernetes deployment (even if not specified explicitly):
 
 **TCP Ports:**
@@ -263,7 +281,7 @@ You can specify custom ports for your nodes in the tcp_ports and udp_ports field
 - 8472: VXLAN for Flannel
 - 53: DNS
 
-#### OpenStack:
+### OpenStack:
 When provisioning on OpenStack, you should provide the value for 'floating_ip_pool' from which floating IPs can be allocated for the instance. If not specified, OpenTofu will not assign floating IP.
 
 ---

{cluster_builder-0.3.1 → cluster_builder-0.3.3}/cluster_builder.egg-info/SOURCES.txt

@@ -15,7 +15,7 @@ cluster_builder/infrastructure/__init__.py
 cluster_builder/infrastructure/executor.py
 cluster_builder/infrastructure/templates.py
 cluster_builder/templates/aws_provider.tf
-cluster_builder/templates/copy_manifest.tf
+cluster_builder/templates/deploy_manifest.tf
 cluster_builder/templates/ha_user_data.sh.tpl
 cluster_builder/templates/master_user_data.sh.tpl
 cluster_builder/templates/openstack_provider.tf

{cluster_builder-0.3.1 → cluster_builder-0.3.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "cluster-builder"
-version = "0.3.1"
+version = "0.3.3"
 description = "Swarmchestrate cluster builder"
 readme = "README.md"
 authors = [

cluster_builder-0.3.1/cluster_builder/templates/copy_manifest.tf

@@ -1,36 +0,0 @@
-# main.tf
-
-variable "manifest_folder" {}
-variable "ssh_private_key_path" {}
-variable "master_ip" {}
-variable "ssh_user" {}
-
-resource "null_resource" "copy_manifests" {
-  connection {
-    type        = "ssh"
-    user        = var.ssh_user
-    private_key = file(var.ssh_private_key_path)
-    host        = var.master_ip
-  }
-
-  # Ensure the manifests folder exists on the remote host
-  provisioner "remote-exec" {
-    inline = [
-      "mkdir -p /home/${var.ssh_user}/manifests",
-      "sudo chmod 755 /home/${var.ssh_user}/manifests"
-    ]
-  }
-
-  # Copy the manifests
-  provisioner "file" {
-    source      = var.manifest_folder
-    destination = "/home/${var.ssh_user}"
-  }
-
-  # Apply manifests using K3s kubeconfig
-  provisioner "remote-exec" {
-    inline = [
-    "sudo -E KUBECONFIG=/etc/rancher/k3s/k3s.yaml kubectl apply -R -f /home/ubuntu/manifests/"
-  ]
-  }
-}