proxlb 2.2.0__tar.gz → 2.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. proxlb-2.3.0/PKG-INFO +659 -0
  2. {proxlb-2.2.0 → proxlb-2.3.0}/README.md +15 -2
  3. {proxlb-2.2.0 → proxlb-2.3.0}/config/proxlb_example.yaml +2 -0
  4. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/__main__.py +30 -13
  5. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/models/guests.py +99 -62
  6. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/models/nodes.py +72 -37
  7. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/models/tags.py +17 -7
  8. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/utils/config_parser.py +2 -0
  9. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/utils/helper.py +11 -10
  10. proxlb-2.3.0/proxlb/utils/rrd.py +39 -0
  11. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/utils/version.py +1 -1
  12. proxlb-2.3.0/proxlb.egg-info/PKG-INFO +659 -0
  13. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb.egg-info/SOURCES.txt +3 -1
  14. {proxlb-2.2.0 → proxlb-2.3.0}/setup.py +6 -2
  15. proxlb-2.3.0/tests/test_solver_fallback_to_greedy.py +11 -0
  16. proxlb-2.2.0/PKG-INFO +0 -28
  17. proxlb-2.2.0/proxlb.egg-info/PKG-INFO +0 -28
  18. {proxlb-2.2.0 → proxlb-2.3.0}/LICENSE +0 -0
  19. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/__init__.py +0 -0
  20. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/models/__init__.py +0 -0
  21. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/models/balancing.py +0 -0
  22. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/models/calculations.py +0 -0
  23. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/models/features.py +0 -0
  24. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/models/groups.py +0 -0
  25. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/models/ha_rules.py +0 -0
  26. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/models/ha_status.py +0 -0
  27. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/models/pools.py +0 -0
  28. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/py.typed +0 -0
  29. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/utils/__init__.py +0 -0
  30. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/utils/cli_parser.py +0 -0
  31. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/utils/logger.py +0 -0
  32. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/utils/proxlb_data.py +0 -0
  33. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb/utils/proxmox_api.py +0 -0
  34. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb.egg-info/dependency_links.txt +0 -0
  35. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb.egg-info/requires.txt +0 -0
  36. {proxlb-2.2.0 → proxlb-2.3.0}/proxlb.egg-info/top_level.txt +0 -0
  37. {proxlb-2.2.0 → proxlb-2.3.0}/service/proxlb.service +0 -0
  38. {proxlb-2.2.0 → proxlb-2.3.0}/setup.cfg +0 -0
  39. {proxlb-2.2.0 → proxlb-2.3.0}/tests/test_calculations_get_most_free_node.py +0 -0
  40. {proxlb-2.2.0 → proxlb-2.3.0}/tests/test_calculations_relocate_guests.py +0 -0
  41. {proxlb-2.2.0 → proxlb-2.3.0}/tests/test_calculations_update_node_resources.py +0 -0
  42. {proxlb-2.2.0 → proxlb-2.3.0}/tests/test_maintenance_nodes_schedule.py +0 -0
  43. {proxlb-2.2.0 → proxlb-2.3.0}/tests/test_predictive_check.py +0 -0
proxlb-2.3.0/PKG-INFO ADDED
@@ -0,0 +1,659 @@
1
+ Metadata-Version: 2.4
2
+ Name: proxlb
3
+ Version: 2.3.0
4
+ Summary: An advanced resource scheduler and load balancer for Proxmox clusters.
5
+ Home-page: https://github.com/credativ/ProxLB
6
+ Author: Florian Paul Azim Hoberg
7
+ Author-email: gyptazy@gyptazy.com
8
+ Maintainer: credativ GmbH
9
+ Maintainer-email: support@credativ.de
10
+ Description-Content-Type: text/markdown
11
+ License-File: LICENSE
12
+ Requires-Dist: packaging
13
+ Requires-Dist: proxlb-solver>=0.1.1
14
+ Requires-Dist: proxmoxer
15
+ Requires-Dist: pydantic
16
+ Requires-Dist: pyyaml
17
+ Requires-Dist: requests
18
+ Requires-Dist: urllib3
19
+ Dynamic: author
20
+ Dynamic: author-email
21
+ Dynamic: description
22
+ Dynamic: description-content-type
23
+ Dynamic: home-page
24
+ Dynamic: license-file
25
+ Dynamic: maintainer
26
+ Dynamic: maintainer-email
27
+ Dynamic: requires-dist
28
+ Dynamic: summary
29
+
30
+ # ProxLB - (Re)Balance VM Workloads in Proxmox Clusters
31
+ <img align="left" src="https://cdn.gyptazy.com/img/ProxLB.jpg"/>
32
+ <br>
33
+
34
+ <p float="center"><img src="https://img.shields.io/github/license/gyptazy/ProxLB"/><img src="https://img.shields.io/github/contributors/gyptazy/ProxLB"/><img src="https://img.shields.io/github/last-commit/gyptazy/ProxLB/main"/><img src="https://img.shields.io/github/issues-raw/gyptazy/ProxLB"/><img src="https://img.shields.io/github/issues-pr/gyptazy/ProxLB"/></p>
35
+
36
+ ## Table of Contents
37
+ 1. [Introduction](#introduction)
38
+ 2. [Features](#features)
39
+ 3. [How does it work?](#how-does-it-work)
40
+ 4. [ProxLB vs Proxmox Dynamic Load Balancing](#ProxLB-vs-Proxmox-Dynamic-Load-Balancing)
41
+ 5. [Documentation](#documentation)
42
+ 6. [Installation](#installation)
43
+ 1. [Requirements / Dependencies](#requirements--dependencies)
44
+ 2. [Debian Package](#debian-package)
45
+ 4. [Container / Docker](#container--docker)
46
+ 5. [Source](#source)
47
+ 6. [CP-SAT Solver (Optional)](#cp-sat-solver-optional)
48
+ 7. [Usage / Configuration](#usage--configuration)
49
+ 1. [Proxmox HA Integration](#proxmox-ha-integration)
50
+ 2. [Options](#options)
51
+ 8. [Affinity & Anti-Affinity Rules](#affinity--anti-affinity-rules)
52
+ 1. [Affinity Rules](#affinity-rules)
53
+ 2. [Anti-Affinity Rules](#anti-affinity-rules)
54
+ 3. [Ignore VMs](#ignore-vms)
55
+ 4. [Pin VMs to Hypervisor Nodes](#pin-vms-to-hypervisor-nodes)
56
+ 9. [Maintenance](#maintenance)
57
+ 10. [Misc](#misc)
58
+ 1. [Bugs](#bugs)
59
+ 2. [Contributing](#contributing)
60
+ 3. [Support](#support)
61
+ 11. [Enterprise-Support](#enterprise-support)
62
+ 12. [Prox-Tools Collection](#prox-tools-collection)
63
+ 13. [Author(s)](#authors)
64
+
65
+
66
+ ## Introduction
67
+ ProxLB is an advanced load balancing solution (initially written by [@gyptazy](https://gyptazy.com/proxlb/)) specifically designed for Proxmox clusters, addressing the absence of an intelligent and more advanced resource scheduler. As a third-party solution, ProxLB enhances the management and efficiency of Proxmox clusters by intelligently distributing workloads across available nodes. Workloads can be balanced by different times like the guest's memory, CPU or disk usage or their assignment to avoid overprovisioning and ensuring resources.
68
+
69
+ One of the key advantages of ProxLB is that it is fully open-source and free, making it accessible for anyone to use, modify, and contribute to. This ensures transparency and fosters community-driven improvements. ProxLB supports filtering and ignoring specific nodes and guests through configuration files and API calls, providing administrators with the flexibility to tailor the load balancing behavior to their specific needs.
70
+
71
+ A standout feature of ProxLB is its maintenance mode. When enabled, all guest workloads are automatically moved to other nodes within the cluster, ensuring that a node can be safely updated, rebooted, or undergo hardware maintenance without disrupting the overall cluster operation. Additionally, ProxLB supports both affinity and anti-affinity rules, allowing operators to group multiple guests to run together on the same node or ensure that certain guests do not run on the same node, depending on the cluster's node count. This feature is crucial for optimizing performance and maintaining high availability.
72
+
73
+ ProxLB can also return the best next node for guest placement, which can be integrated into CI/CD pipelines using tools like Ansible or Terraform. This capability streamlines the deployment process and ensures efficient resource utilization. Furthermore, ProxLB leverages the Proxmox API, including the entire ACL (Access Control List) system, for secure and efficient operation. Unlike some solutions, it does not require SSH access, enhancing security and simplifying configuration.
74
+
75
+ Overall, ProxLB significantly enhances resource management by intelligently distributing workloads, reducing downtime through its maintenance mode, and providing improved flexibility with affinity and anti-affinity rules. Its seamless integration with CI/CD tools and reliance on the Proxmox API make it a robust and secure solution for optimizing Proxmox cluster performance.
76
+
77
+ ### Video of Migration
78
+ <img src="https://cdn.gyptazy.com/img/proxlb-rebalancing-demo.gif"/>
79
+
80
+ ## Features
81
+ ProxLB's key features are by enabling automatic rebalancing of VMs and CTs across a Proxmox cluster based on memory, CPU, and local disk usage while identifying optimal nodes for automation. It supports maintenance mode, affinity rules, and seamless Proxmox API integration with ACL support, offering flexible usage as a one-time operation, a daemon, or through the Proxmox Web GUI.
82
+
83
+ **Features**
84
+ * Rebalance VMs/CTs in the cluster by:
85
+ * Memory
86
+ * Disk (only local storage)
87
+ * CPU
88
+ * Rebalance by different modes:
89
+ * Used resources
90
+ * Assigned resources
91
+ * PSI (Pressure) of resources
92
+ * Get best nodes for further automation
93
+ * Supported Guest Types
94
+ * VMs
95
+ * CTs
96
+ * Maintenance Mode
97
+ * Set node(s) into maintenance
98
+ * Move all workloads to different nodes
99
+ * Affinity / Anti-Affinity Rules
100
+ * Fully based on Proxmox API
101
+ * Fully integrated into the Proxmox ACL
102
+ * No SSH required
103
+ * Usage
104
+ * One-Time
105
+ * Daemon
106
+ * Proxmox Web GUI Integration
107
+
108
+ ## How does it work?
109
+ ProxLB is a load-balancing system designed to optimize the distribution of virtual machines (VMs) and containers (CTs) across a cluster. It works by first gathering resource usage metrics from all nodes in the cluster through the Proxmox API. This includes detailed resource metrics for each VM and CT on every node. ProxLB then evaluates the difference between the maximum and minimum resource usage of the nodes, referred to as "Balanciness." If this difference exceeds a predefined threshold (which is configurable), the system initiates the rebalancing process.
110
+
111
+ Before starting any migrations, ProxLB validates that rebalancing actions are necessary and beneficial. Depending on the selected balancing mode — such as CPU, memory, or disk — it creates a balancing matrix. This matrix sorts the VMs by their maximum used or assigned resources, identifying the VM with the highest usage. ProxLB then places this VM on the node with the most free resources in the selected balancing type. This process runs recursively until the operator-defined Balanciness is achieved. Balancing can be defined for the used or max. assigned resources of VMs/CTs.
112
+
113
+ ## ProxLB vs Proxmox Dynamic Load Balancing
114
+ With the introduction of Dynamic Load Balancing (DLB) in Proxmox VE 9.2, Proxmox now includes a native solution for automatically balancing HA workloads across cluster nodes. This naturally raises the question of how ProxLB compares to the built-in scheduler and whether an external balancing solution is still necessary.
115
+
116
+ While both solutions aim to optimize resource utilization and automate VM placement, they follow different design philosophies. Proxmox DLB focuses on native integration, simplicity, and HA-aware balancing directly within the Proxmox cluster stack, whereas ProxLB provides a more flexible and customizable orchestration layer with advanced placement logic, affinity handling, and policy-based scheduling for any kind of workloads and is **not** limited to only HA workloads.
117
+
118
+ The following comparison highlights the key differences, strengths, and use cases of both approaches.
119
+
120
+ | Feature | ProxLB | PVE Native DLB |
121
+ |---|---|---|
122
+ | Integration level | External third-party scheduler | Built directly into Proxmox HA/CRS |
123
+ | Works for guests with HA stack | Yes | Yes|
124
+ | Works for guests without HA stack | Yes | No |
125
+ | Real-time balancing | Yes | Yes |
126
+ | VM migration automation | Yes | Yes |
127
+ | Affinity / anti-affinity rules | Advanced | More basic / HA-rule focused |
128
+ | Node pinning | Yes, incl. groups | Yes |
129
+ | ProxPatch Support | Yes | No |
130
+ | Custom scheduling logic | Extensive | Limited to Proxmox parameters |
131
+ | Metrics considered | CPU, RAM, disk, overprovisioning, assignment logic | Primarily node + guest runtime utilization |
132
+ | GUI integration | Partial / custom | Fully native |
133
+ | Maintenance burden | You manage updates/config | Supported by Proxmox |
134
+ | Stability / support | Community project | Officially supported |
135
+ | Enterprise readiness | Powerful but external | Much better for standardized environments |
136
+ | Complexity | Higher | Lower |
137
+ | Best suited for | Advanced custom orchestration | Native enterprise balancing |
138
+ | Dependency footprint | Additional daemon/service | Built-in functionality |
139
+ | Upgrade handling | Manual compatibility validation | Included in Proxmox upgrades |
140
+ | Scheduling aggressiveness | Highly customizable | Conservative by design |
141
+ | Corosync related | No (good) | Yes (bad) |
142
+
143
+ > [!TIP]
144
+ > Still using PVE 8? The only solution is ProxLB! If you encounter issues due to more Corosync traffic, you might also want to switch to ProxLB (for large-scaled environemnts).
145
+
146
+ ## Documentation
147
+ This `README.md` doesn't contain all information and only highlights the most important facts. Extended information, such like API permissions, creating dedicated user, best-practices in running ProxLB and much more can be found in the [docs/](https://github.com/gyptazy/ProxLB/tree/main/docs) directory. Please consult the documentation before creating issues.
148
+
149
+ ## Installation
150
+ ProxLB can be installed as a Debian package or by a container image. With the provided Helm charts it can also easily be deployed into Kubernetes stacks.
151
+
152
+ *Note: The Debian repository and container image registry are currently operated by @gyptazy and will be migrated to a new location in the future.*
153
+
154
+
155
+ ### Requirements / Dependencies
156
+ * Proxmox
157
+ * Proxmox 7.x
158
+ * Proxmox 8.x
159
+ * Proxmox 9.x
160
+ * Python3.x
161
+ * proxmoxer
162
+ * requests
163
+ * urllib3
164
+ * pyyaml
165
+
166
+ The dependencies can simply be installed with `pip` by running the following command:
167
+ ```
168
+ pip install -r requirements.txt
169
+ ```
170
+
171
+ *Note: Distribution packages, such like the provided `.deb` package will automatically resolve and install all required dependencies by using already packaged version from the distribution's repository. By using the Docker (container) image or Debian packages, you do not need to take any care of the requirements listed here.*
172
+
173
+ ### Debian Package
174
+ ProxLB is a powerful and flexible load balancer designed to work across various architectures, including `amd64`, `arm64`, `rv64` and many other ones that support Python. It runs independently of the underlying hardware, making it a versatile choice for different environments. This chapter covers the step-by-step process to install ProxLB on Debian-based systems, including Debian clones like Ubuntu.
175
+
176
+ #### Quick-Start
177
+ You can simply use this snippet to install the repository and to install ProxLB on your system.
178
+
179
+ ```bash
180
+ # Add GPG key
181
+ curl -fsSL https://packages.credativ.com/public/proxtools/public.key \
182
+ | sudo gpg --dearmor -o /etc/apt/keyrings/proxtools-archive-keyring.gpg
183
+
184
+ # Add repository
185
+ echo "deb [signed-by=/etc/apt/keyrings/proxtools-archive-keyring.gpg] \
186
+ https://packages.credativ.com/public/proxtools stable main" \
187
+ | sudo tee /etc/apt/sources.list.d/proxlb.list
188
+
189
+ # Update & install
190
+ sudo apt-get update
191
+ sudo apt-get -y install proxlb
192
+
193
+ # Copy example config
194
+ sudo cp /etc/proxlb/proxlb_example.yaml /etc/proxlb/proxlb.yaml
195
+
196
+ # Adjust the config to your needs
197
+ sudo vi /etc/proxlb/proxlb.yaml
198
+
199
+ # Start service
200
+ sudo systemctl start proxlb
201
+
202
+ # Adjust the config to your needs
203
+ sudo vi /etc/proxlb/proxlb.yaml
204
+ sudo systemctl start proxlb
205
+ ```
206
+
207
+ Afterwards, ProxLB is running in the background and balances your cluster by your defined balancing method (default: memory).
208
+
209
+ #### Details
210
+ ProxLB provides two different repositories:
211
+ * https://packages.credativ.com/public/proxtools stable main
212
+ * https://packages.credativ.com/public/proxtools snapshots main
213
+
214
+ The repository is signed and the GPG key can be found at:
215
+ * https://packages.credativ.com/public/proxtools/archive-keyring.gpg
216
+
217
+ You can also simply import it by running:
218
+
219
+ ```
220
+ # KeyID: 34C5B9642CD591E5D090A03B062A8A3A410B831D
221
+ # UID: Proxtools Repository Signer <info@credativ.de>
222
+ # SHA256: 4cb4a74b25f775616709eb0596eeeac61d8d28717f4872fef2d68fb558434ed3 public.key
223
+
224
+ wget -O /etc/apt/keyrings/proxtools-archive-keyring.gpg https://packages.credativ.com/public/proxtools/public.key
225
+ ```
226
+
227
+ ### Container Images / Docker
228
+ Using the ProxLB container images is straight forward and only requires you to mount the config file.
229
+
230
+ Available images can be found at the GitHub [packages page](https://github.com/credativ/ProxLB/pkgs/container/proxlb) or [Docker Hub](https://hub.docker.com/r/credativ/proxlb).
231
+
232
+ ```bash
233
+ # Pull the image from GHCR
234
+ docker pull ghcr.io/credativ/proxlb:latest
235
+ # or Docker Hub
236
+ docker pull credativ/proxlb:latest
237
+ # Download the config
238
+ wget -O proxlb.yaml https://raw.githubusercontent.com/gyptazy/ProxLB/refs/heads/main/config/proxlb_example.yaml
239
+ # Adjust the config to your needs
240
+ vi proxlb.yaml
241
+ # Start the ProxLB container image with the ProxLB config
242
+ docker run -it --rm -v $(pwd)/proxlb.yaml:/etc/proxlb/proxlb.yaml proxlb
243
+ ```
244
+
245
+ ### Docker Compose
246
+
247
+ ```bash
248
+ services:
249
+ proxlb:
250
+ image: ghcr.io/credativ/proxlb:latest
251
+ container_name: proxlb
252
+ restart: unless-stopped
253
+ volumes:
254
+ - ./proxlb.yaml:/etc/proxlb/proxlb.yaml:ro
255
+ ```
256
+
257
+ *Note: ProxLB container images are officially only available at ghcr.io/credativ/proxlb or docker.io/credativ/proxlb*
258
+
259
+ ### Source
260
+ ProxLB can also easily be used from the provided sources - for traditional systems but also as a Docker/Podman container image.
261
+
262
+ #### Traditional System
263
+ Setting up and running ProxLB from the sources is simple and requires just a few commands. Ensure Python 3 and the Python dependencies are installed on your system, then run ProxLB using the following command:
264
+ ```bash
265
+ git clone https://github.com/credativ/ProxLB.git
266
+ cd ProxLB
267
+ ```
268
+
269
+ Afterwards simply adjust the config file to your needs:
270
+ ```bash
271
+ vi config/proxlb.yaml
272
+ ```
273
+
274
+ Start ProxLB by Python3 on the system:
275
+ ```bash
276
+ python3 -m proxlb -c config/proxlb.yaml
277
+ ```
278
+
279
+ #### Container Image
280
+ Creating a container image of ProxLB is straightforward using the provided Dockerfile. The Dockerfile simplifies the process by automating the setup and configuration required to get ProxLB running in an Alpine container. Simply follow the steps in the Dockerfile to build the image, ensuring all dependencies and configurations are correctly applied. For those looking for an even quicker setup, a ready-to-use ProxLB container image is also available, eliminating the need for manual building and allowing for immediate deployment.
281
+
282
+ ```bash
283
+ git clone https://github.com/credativ/ProxLB.git
284
+ cd ProxLB
285
+ docker build -t proxlb .
286
+ ```
287
+
288
+ Afterwards simply adjust the config file to your needs:
289
+ ```bash
290
+ vi config/proxlb.yaml
291
+ ```
292
+
293
+ Finally, start the created container.
294
+ ```bash
295
+ docker run -it --rm -v $(pwd)/proxlb.yaml:/etc/proxlb/proxlb.yaml proxlb
296
+ ```
297
+
298
+ ### CP-SAT Solver (Optional)
299
+ ProxLB optionally integrates a CP-SAT-based placement solver (Google OR-Tools) that replaces the built-in greedy balancer with a mathematically optimal assignment engine. It runs in two modes:
300
+
301
+ * **Shadow mode** *(default)* — solver computes an optimal plan alongside ProxLB for comparison; no migrations are changed.
302
+ * **Active mode** — solver drives all migrations with automatic per-step verification and re-solve on failure.
303
+
304
+ Both modes produce a structured JSONL log and an HTML report showing the solver plan, node load before/after, and (in active mode) the execution result of every migration.
305
+
306
+ Add a `solver:` block to your `proxlb.yaml`:
307
+
308
+ ```yaml
309
+ solver:
310
+ enable: True
311
+ mode: shadow # 'shadow' (observe only) or 'active' (solver drives migrations)
312
+ log_dir: /var/log/proxlb/solver
313
+ timeout_seconds: 30
314
+ use_reservations: True
315
+ active_step_retries: 3
316
+ fallback_to_greedy: True
317
+ ```
318
+
319
+ See [docs/04_solver.md](docs/04_solver.md) for the full configuration reference, a description of the feedback loop in active mode, and instructions for generating HTML reports.
320
+
321
+ ## Usage / Configuration
322
+ Running ProxLB is straightforward and versatile, as it only requires `Python3` and the `proxmoxer` library. This means ProxLB can be executed directly on a Proxmox node or on dedicated systems such as Debian, RedHat, or even FreeBSD, provided that the Proxmox API is accessible from the client running ProxLB. ProxLB can also run inside a Container - Docker or LXC - and is simply up to you.
323
+
324
+ ### Proxmox HA Integration
325
+ Proxmox HA (High Availability) groups are designed to ensure that virtual machines (VMs) remain running within a Proxmox cluster. HA groups define specific rules for where VMs should be started or migrated in case of node failures, ensuring minimal downtime and automatic recovery.
326
+
327
+ However, when used in conjunction with ProxLB, the built-in load balancer for Proxmox, conflicts can arise. ProxLB operates with its own logic for workload distribution, taking into account affinity and anti-affinity rules. While it effectively balances guest workloads, it may re-shift and redistribute VMs in a way that does not align with HA group constraints, potentially leading to unsuitable placements.
328
+
329
+ Due to these conflicts, it is currently not recommended to use both HA groups and ProxLB simultaneously. The interaction between the two mechanisms can lead to unexpected behavior, where VMs might not adhere to HA group rules after being moved by ProxLB.
330
+
331
+ A solution to improve compatibility between HA groups and ProxLB is under evaluation, aiming to ensure that both features can work together without disrupting VM placement strategies.
332
+
333
+ See also: [#65: Host groups: Honour HA groups](https://github.com/gyptazy/ProxLB/issues/65).
334
+
335
+ ### Options
336
+ The following options can be set in the configuration file `proxlb.yaml`:
337
+
338
+ | Section | Option | Sub Option | Example | Type | Description |
339
+ |---------|:------:|:----------:|:-------:|:----:|:-----------:|
340
+ | `proxmox_api` | | | | | |
341
+ | | hosts | | ['virt01.example.com', '10.10.10.10', 'fe01:bad:code::cafe', 'virt01.example.com:443', '[fc00::1]', '[fc00::1]:443', 'fc00::1:8006'] | `List` | List of Proxmox nodes. Can be IPv4, IPv6 or mixed. You can specify custom ports. In case of IPv6 without brackets the port is considered after the last colon |
342
+ | | user | | root@pam | `Str` | Username for the API. |
343
+ | | pass | | FooBar | `Str` | Password for the API. (Recommended: Use API token authorization!) |
344
+ | | token_id | | proxlb | `Str` | Token ID of the user for the API. |
345
+ | | token_secret | | 430e308f-1337-1337-beef-1337beefcafe | `Str` | Secret of the token ID for the API. |
346
+ | | ssl_verification | | True | `Bool` | Validate SSL certificates (1) or ignore (0). [values: `1` (default), `0`] |
347
+ | | timeout | | 10 | `Int` | Timeout for the Proxmox API in sec. |
348
+ | | retries | | 1 | `Int` | How often a connection attempt to the defined API host should be performed. |
349
+ | | wait_time | | 1 | `Int` | How many seconds should be waited before performing another connection attempt to the API host. |
350
+ | `proxmox_cluster` | | | | | |
351
+ | | maintenance_nodes | | ['virt66.example.com'] | `List` | A list of Proxmox nodes that are defined to be in a maintenance. (must be the same node names as used within the cluster) |
352
+ | | maintenance_nodes_schedule | | | `Dict` | A weekly schedule that temporarily adds nodes to `maintenance_nodes` during runtime. |
353
+ | | ignore_nodes | | [] | `List` | A list of Proxmox nodes that are defined to be ignored. |
354
+ | | overprovisioning | | False | `Bool` | Avoids balancing when nodes would become overprovisioned. |
355
+ | `balancing` | | | | | |
356
+ | | enable | | True | `Bool` | Enables the guest balancing.|
357
+ | | enforce_affinity | | False | `Bool` | Enforcing affinity/anti-affinity rules but balancing might become worse. |
358
+ | | enforce_pinning | | False | `Bool` | Enforcing pinning rules but balancing might become worse. |
359
+ | | parallel | | False | `Bool` | If guests should be moved in parallel or sequentially.|
360
+ | | parallel_jobs | | 5 | `Int` | The amount if parallel jobs when migrating guests. (default: `5`)|
361
+ | | live | | True | `Bool` | If guests should be moved live or shutdown.|
362
+ | | with_local_disks | | True | `Bool` | If balancing of guests should include local disks.|
363
+ | | with_conntrack_state | | True | `Bool` | If balancing of guests should including the conntrack state.|
364
+ | | balance_types | | ['vm', 'ct'] | `List` | Defined the types of guests that should be honored. [values: `vm`, `ct`]|
365
+ | | max_job_validation | | 1800 | `Int` | How long a job validation may take in seconds. (default: 1800) |
366
+ | | balanciness | | 10 | `Int` | The maximum delta of resource usage between node with highest and lowest usage. |
367
+ | | memory_threshold | | 75 | `Int` | The maximum threshold (in percent) that needs to be hit to perform balancing actions. (Optional) |
368
+ | | cpu_threshold | | 75 | `Int` | The maximum threshold (in percent) that needs to be hit to perform balancing actions. (Optional) |
369
+ | | method | | memory | `Str` | The balancing method that should be used. [values: `memory` (default), `cpu`, `disk`]|
370
+ | | mode | | used | `Str` | The balancing mode that should be used. [values: `used` (default), `assigned`, `psi` (pressure)] |
371
+ | | balance_larger_guests_first | | False | `Bool` | Option to prefer larger/smaller guests first |
372
+ | | node_resource_reserve | | { default: { memory: 4 }, { node01: { memory: 6 }} } | `Dict` | A dict of pool names and their type for creating affinity/anti-affinity rules |
373
+ | | psi | | { nodes: { memory: { pressure_full: 0.20, pressure_some: 0.20, pressure_spikes: 1.00 }}} | `Dict` | A dict of PSI based thresholds for nodes and guests |
374
+ | | pools | | pools: { dev: { type: affinity }, de-nbg01-db: { type: anti-affinity }} | `Dict` | A dict of pool names and their type for creating affinity/anti-affinity rules |
375
+ | `service` | | | | | |
376
+ | | daemon | | True | `Bool` | If daemon mode should be activated. |
377
+ | | enable_ha | | False | `Bool` | Enables HA mode of ProxLB. |
378
+ | | `schedule` | | | `Dict` | Schedule config block for rebalancing. |
379
+ | | | interval | 12 | `Int` | How often rebalancing should occur in daemon mode.|
380
+ | | | format | hours | `Str` | Sets the time format. [values: `hours` (default), `minutes`]|
381
+ | | `delay` | | | `Dict` | Schedule config block for an optional delay until the service starts. |
382
+ | | | enable | False | `Bool` | If a delay time should be validated.|
383
+ | | | time | 1 | `Int` | Delay time until the service starts after the initial execution.|
384
+ | | | format | hours | `Str` | Sets the time format. [values: `hours` (default), `minutes`]|
385
+ | | log_level | | INFO | `Str` | Defines the default log level that should be logged. [values: `INFO` (default), `WARNING`, `CRITICAL`, `DEBUG`] |
386
+
387
+
388
+ An example of the configuration file looks like:
389
+ ```
390
+ proxmox_api:
391
+ hosts: ['virt01.example.com', '10.10.10.10', 'fe01:bad:code::cafe']
392
+ user: root@pam
393
+ pass: crazyPassw0rd!
394
+ # API Token method
395
+ # token_id: proxlb
396
+ # token_secret: 430e308f-1337-1337-beef-1337beefcafe
397
+ ssl_verification: True
398
+ timeout: 10
399
+ # API Connection retries
400
+ # retries: 1
401
+ # wait_time: 1
402
+
403
+ proxmox_cluster:
404
+ maintenance_nodes: ['virt66.example.com']
405
+ maintenance_nodes_schedule:
406
+ duration: 3
407
+ pre-migration: 10
408
+ schedules:
409
+ virt77.example.com:
410
+ - 'Monday, 8:00'
411
+ ignore_nodes: []
412
+ overprovisioning: True
413
+
414
+ balancing:
415
+ enable: True
416
+ enforce_affinity: False
417
+ enforce_pinning: False
418
+ parallel: False
419
+ live: True
420
+ with_local_disks: True
421
+ with_conntrack_state: True
422
+ balance_types: ['vm', 'ct']
423
+ max_job_validation: 1800
424
+ memory_threshold: 75
425
+ #cpu_threshold: 75
426
+ balanciness: 5
427
+ method: memory
428
+ mode: used
429
+ balance_larger_guests_first: False
430
+ node_resource_reserve:
431
+ defaults:
432
+ memory: 4
433
+ node01:
434
+ memory: 6
435
+ # # PSI thresholds only apply when using mode 'psi'
436
+ # # PSI based balancing is currently in beta and req. PVE >= 9
437
+ # psi:
438
+ # nodes:
439
+ # memory:
440
+ # pressure_full: 0.20
441
+ # pressure_some: 0.20
442
+ # pressure_spikes: 1.00
443
+ # cpu:
444
+ # pressure_full: 0.20
445
+ # pressure_some: 0.20
446
+ # pressure_spikes: 1.00
447
+ # disk:
448
+ # pressure_full: 0.20
449
+ # pressure_some: 0.20
450
+ # pressure_spikes: 1.00
451
+ # guests:
452
+ # memory:
453
+ # pressure_full: 0.20
454
+ # pressure_some: 0.20
455
+ # pressure_spikes: 1.00
456
+ # cpu:
457
+ # pressure_full: 0.20
458
+ # pressure_some: 0.20
459
+ # pressure_spikes: 1.00
460
+ # disk:
461
+ # pressure_full: 0.20
462
+ # pressure_some: 0.20
463
+ # pressure_spikes: 1.00
464
+ pools:
465
+ dev:
466
+ type: affinity
467
+ de-nbg01-db
468
+ type: anti-affinity
469
+ pin:
470
+ - virt66
471
+ - virt77
472
+ strict: False
473
+
474
+ service:
475
+ daemon: True
476
+ enable_ha: False
477
+ schedule:
478
+ interval: 12
479
+ format: hours
480
+ delay:
481
+ enable: False
482
+ time: 1
483
+ format: hours
484
+ log_level: INFO
485
+ ```
486
+
487
+ ### Parameters
488
+ The following options and parameters are currently supported:
489
+
490
+ | Option | Long Option | Description | Default |
491
+ |------|:------:|------:|------:|
492
+ | -c | --config | Path to a config file. | /etc/proxlb/proxlb.yaml (default) |
493
+ | -d | --dry-run | Performs a dry-run without doing any actions. | False |
494
+ | -j | --json | Returns a JSON of the VM movement. | False |
495
+ | -b | --best-node | Returns the best next node for a VM/CT placement (useful for further usage with Terraform/Ansible). | False |
496
+ | -v | --version | Returns the ProxLB version on stdout. | False |
497
+
498
+ ## Affinity & Anti-Affinity Rules
499
+ ProxLB provides an advanced mechanism to define affinity and anti-affinity rules, enabling precise control over virtual machine (VM) placement. These rules help manage resource distribution, improve high availability configurations, and optimize performance within a Proxmox Virtual Environment (PVE) cluster. By leveraging Proxmox’s integrated access management, ProxLB ensures that users can only define and manage rules for guests they have permission to access.
500
+
501
+ ProxLB implements affinity and anti-affinity rules through a tag-based system within the Proxmox web interface. Each guest (virtual machine or container) can be assigned specific tags, which then dictate its placement behavior. This method maintains a streamlined and secure approach to managing VM relationships while preserving Proxmox’s inherent permission model.
502
+
503
+ ### Affinity Rules
504
+ <img align="left" src="https://cdn.gyptazy.com/img/proxlb-affinity-rules.jpg"/> Affinity rules are used to group certain VMs together, ensuring that they run on the same host whenever possible. This can be beneficial for workloads requiring low-latency communication, such as clustered databases or application servers that frequently exchange data. In general, there're two ways to manage affinity rules:
505
+
506
+ #### Affinity Rules by Tags
507
+ To define an affinity rule which keeps all guests assigned to this tag together on a node, users assign a tag with the prefix `plb_affinity_$TAG`:
508
+
509
+ #### Example for Screenshot
510
+ ```
511
+ plb_affinity_talos
512
+ ```
513
+ As a result, ProxLB will attempt to place all VMs with the `plb_affinity_web` tag on the same host (see also the attached screenshot with the same node).
514
+
515
+ #### Affinity Rules by Pools
516
+ Antoher approach is by using pools in Proxmox. This way, it can easily also combined with other resources like backup jobs. However, in this approach you need to modify the ProxLB config file to your needs. Within the `balancing` section you can create a dict of pools, including the pool name and the affinity type. Please see the example for further details:
517
+
518
+ **Example Config**
519
+ ```
520
+ balancing:
521
+ [...]
522
+ pools: # Optional: Define affinity/anti-affinity rules per pool
523
+ dev: # Pool name: dev
524
+ type: affinity # Type: affinity (keeping VMs together)
525
+ pin: # Pin VMs to Nodes
526
+ - virt77 # Pinning to 'virt77' which is maybe an older system for dev labs
527
+ ```
528
+
529
+ ### Anti-Affinity Rules by Tags
530
+ <img align="left" src="https://cdn.gyptazy.com/img/proxlb-anti-affinity-rules.jpg"/> Conversely, anti-affinity rules ensure that designated VMs do not run on the same physical host. This is particularly useful for high-availability setups, where redundancy is crucial. Ensuring that critical services are distributed across multiple hosts reduces the risk of a single point of failure. In general, there're two ways to manage anti-affinity rules:
531
+
532
+ To define an anti-affinity rule that ensures to not move systems within this group to the same node, users assign a tag with the prefix:
533
+
534
+ #### Example for Screenshot
535
+ ```
536
+ plb_anti_affinity_ntp
537
+ ```
538
+
539
+ As a result, ProxLB will try to place the VMs with the `plb_anti_affinity_ntp` tag on different hosts (see also the attached screenshot with the different nodes).
540
+
541
+ #### Anti-Affinity Rules by Pools
542
+ Antoher approach is by using pools in Proxmox. This way, it can easily also combined with other resources like backup jobs. However, in this approach you need to modify the ProxLB config file to your needs. Within the `balancing` section you can create a dict of pools, including the pool name and the affinity type. Please see the example for further details:
543
+
544
+ **Example Config**
545
+ ```
546
+ balancing:
547
+ [...]
548
+ pools: # Optional: Define affinity/anti-affinity rules per pool
549
+ de-nbg01-db: # Pool name: de-nbg01-db
550
+ type: anti-affinity # Type: anti-affinity (spreading VMs apart)
551
+ ```
552
+
553
+
554
+ **Note:** While this ensures that ProxLB tries distribute these VMs across different physical hosts within the Proxmox cluster this may not always work. If you have more guests attached to the group than nodes in the cluster, we still need to run them anywhere. If this case occurs, the next one with the most free resources will be selected.
555
+
556
+ ### Ignore VMs
557
+ <img align="left" src="https://cdn.gyptazy.com/img/proxlb-ignore-vm-movement.jpg"/> Guests, such as VMs or CTs, can also be completely ignored. This means, they won't be affected by any migration (even when (anti-)affinity rules are enforced). To ensure a proper resource evaluation, these guests are still collected and evaluated but simply skipped for balancing actions. Another thing is the implementation. While ProxLB might have a very restricted configuration file including the file permissions, this file is only read- and writeable by the Proxmox administrators. However, we might have user and groups who want to define on their own that their systems shouldn't be moved. Therefore, these users can simpy set a specific tag to the guest object - just like the (anti)affinity rules.
558
+
559
+ **Note:** Ignored guests are really ignored. Even by enforcing affinity rules this guest will be ignored.
560
+
561
+ #### Ignoring VMs via Tags
562
+ To define a guest to be ignored from the balancing, users assign a tag with the prefix `plb_ignore_$TAG`:
563
+
564
+ ##### Example for Screenshot
565
+ ```
566
+ plb_ignore_dev
567
+ ```
568
+
569
+ As a result, ProxLB will not migrate this guest with the `plb_ignore_dev` tag to any other node.
570
+
571
+ #### Ignoring VMs by Name via Configuration
572
+ Guests can also be ignored by name directly in the ProxLB configuration file. This is useful for automatically provisioned VMs or CTs that cannot have tags assigned themselves.
573
+
574
+ **Example Config**
575
+ ```yaml
576
+ balancing:
577
+ ignore_guests: ['my-auto-vm-01', 'my-auto-vm-02']
578
+ ```
579
+
580
+ As a result, ProxLB will not migrate any guest whose name appears in the `ignore_guests` list.
581
+
582
+ ### Pin VMs to Specific Hypervisor Nodes
583
+ <img align="left" src="https://cdn.gyptazy.com/img/proxlb-tag-node-pinning.jpg"/> Guests, such as VMs or CTs, can also be pinned to specific (and multiple) nodes in the cluster. This might be usefull when running applications with some special licensing requirements that are only fulfilled on certain nodes. It might also be interesting, when some physical hardware is attached to a node, that is not available in general within the cluster.
584
+
585
+ #### Pinning VMs to (a) specific Hypervisor Node(s) by Tag
586
+ To pin a guest to a specific cluster node, users assign a tag with the prefix `plb_pin_$nodename` to the desired guest:
587
+
588
+ #### Example for Screenshot
589
+ ```
590
+ plb_pin_node03
591
+ ```
592
+
593
+ As a result, ProxLB will pin the guest `dev-vm01` to the node `virt03`.
594
+
595
+
596
+ #### Pinning VMs to (a) specific Hypervisor Node(s) by Pools
597
+ Beside the tag approach, you can also pin a resource group to a specific hypervisor or groups of hypervisors by defining a `pin` key of type list.
598
+
599
+ **Example Config**
600
+ ```
601
+ balancing:
602
+ [...]
603
+ pools: # Optional: Define affinity/anti-affinity rules per pool
604
+ dev: # Pool name: dev
605
+ type: affinity # Type: affinity (keeping VMs together)
606
+ pin: # Pin VMs to Nodes
607
+ - virt77 # Pinning to 'virt77' which is maybe an older system for dev labs
608
+ ```
609
+
610
+
611
+ You can also repeat this step multiple times for different node names to create a potential group of allowed hosts where a the guest may be served on. In this case, ProxLB takes the node with the lowest used resources according to the defined balancing values from this group.
612
+
613
+ **Note:** The given node names from the tag are validated. This means, ProxLB validated if the given node name is really part of the cluster. In case of a wrongly defined or unavailable node name it continous to use the regular processes to make sure the guest keeps running.
614
+
615
+ ## Maintenance
616
+ The `maintenance_nodes` option allows operators to designate one or more Proxmox nodes for maintenance mode. When a node is set to maintenance, no new guest workloads will be assigned to it, and all existing workloads will be migrated to other available nodes within the cluster. This process ensures that (anti)-affinity rules and resource availability are respected, preventing disruptions while maintaining optimal performance across the infrastructure.
617
+
618
+ ### Adding / Removing Nodes from Maintenance
619
+ Within the section `proxmox_cluster` you can define the key `maintenance_nodes` as a list object. Simply add/remove one or more nodes with their equal name in the cluster and restart the daemon.
620
+ ```
621
+ proxmox_cluster:
622
+ maintenance_nodes: ['virt66.example.com']
623
+ ```
624
+ Afterwards, all guest objects will be moved to other nodes in the cluster by ensuring the best balancing.
625
+
626
+ Maintenance mode can also be scheduled. Scheduled nodes are added during each runtime cycle and removed automatically when the active window ends. `duration` is counted in hours from the configured start time, while `pre-migration` starts maintenance mode that many minutes earlier.
627
+ ```yaml
628
+ proxmox_cluster:
629
+ maintenance_nodes_schedule:
630
+ duration: 3
631
+ pre-migration: 10
632
+ schedules:
633
+ virt77.example.com:
634
+ - 'Monday, 8:00'
635
+ ```
636
+
637
+ ## Misc
638
+ ### Bugs
639
+ Bugs can be reported via the GitHub issue tracker [here](https://github.com/gyptazy/ProxLB/issues). You may also report bugs via email or deliver PRs to fix them on your own. Therefore, you might also see the contributing chapter.
640
+
641
+ ### Contributing
642
+ Feel free to add further documentation, to adjust already existing one or to contribute with code. Please take care about the style guide and naming conventions. You can find more in our [CONTRIBUTING.md](https://github.com/gyptazy/ProxLB/blob/main/CONTRIBUTING.md) file.
643
+
644
+ ### Enterprise-Support
645
+ Running critical infrastructure in an enterprise environment often comes with requirements that go far beyond functionality alone. Enterprises typically expect predictable service levels, defined escalation paths, and guaranteed response times. In many cases, organizations also demand 24x7 support availability to ensure that their systems remain stable and resilient, even under unexpected circumstances.
646
+
647
+ [credativ.de GmbH](https://www.credativ.de/en/portfolio/support/proxmox-virtualization/) is also happy to provide enterprise support for ProxLB, including 24x7 support.
648
+
649
+ ## Prox-Tools Collection
650
+ The [Prox-Tools collection](https://gyptazy.com/projects/) is a set of (mostly) open-source solutions crafted by our employee [@gyptazy](https://gyptazy.com/) with a clear mission: to close functional gaps in Proxmox VE environments that are especially relevant for enterprise use cases. While Proxmox VE already provides a powerful and flexible virtualization platform, real world production deployments often require additional tooling around scheduling, compatibility, automation, networking, and operational safety. Prox-Tools addresses exactly these needs.
651
+ All tools in this collection are designed with the same principles in mind: practical relevance, minimal complexity, transparent behavior, and full openness. They are built to integrate seamlessly into existing Proxmox VE clusters, without vendor lock-in, and with a strong focus on reliability and operational clarity. The current tools are listed below:
652
+ * [ProxLB](https://gyptazy.com/proxlb/): An advanced open-source load balancing and resource scheduling solution for Proxmox clusters.
653
+ * [ProxCLMC](https://gyptazy.com/proxclmc/): A CPU compatibility validator across Proxmox VE nodes for safe live migrations.
654
+ * [ProxSnap](https://gyptazy.com/proxsnap/): Snapshot management and cleanup tool for Proxmox VE clusters.
655
+ * [ProxWall](https://gyptazy.com/proxwall/): Micro-segmentation and advanced firewalling and networking for Proxmox VE clusters. It is designed to give enterprises fine-grained control over east-west and north-south traffic.
656
+ * [ProxWire](https://cdn.gyptazy.com/talks/ProxWire_Connecting_Proxmox_Clusters_Around_the_World_Without_Corosync.pdf): A forward-looking concept aimed at securely connecting Proxmox VE nodes across larger geographic distances.
657
+
658
+ ## Author(s)
659
+ * Florian Paul Azim Hoberg [@gyptazy](https://gyptazy.com/proxlb/)