CFLA 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- cfla-1.0.0/CFLA.egg-info/PKG-INFO +245 -0
- cfla-1.0.0/CFLA.egg-info/SOURCES.txt +55 -0
- cfla-1.0.0/CFLA.egg-info/dependency_links.txt +1 -0
- cfla-1.0.0/CFLA.egg-info/requires.txt +9 -0
- cfla-1.0.0/CFLA.egg-info/top_level.txt +20 -0
- cfla-1.0.0/MANIFEST.in +20 -0
- cfla-1.0.0/PKG-INFO +245 -0
- cfla-1.0.0/README.md +224 -0
- cfla-1.0.0/datasets/__init__.py +3 -0
- cfla-1.0.0/datasets/ag_news.py +198 -0
- cfla-1.0.0/datasets/dataset_utils.py +222 -0
- cfla-1.0.0/datasets/emnist.py +30 -0
- cfla-1.0.0/datasets/femnist.py +181 -0
- cfla-1.0.0/experiments/scripts/cgpfl.py +74 -0
- cfla-1.0.0/experiments/scripts/fedgroup.py +76 -0
- cfla-1.0.0/experiments/scripts/fedper.py +72 -0
- cfla-1.0.0/experiments/scripts/fesem.py +73 -0
- cfla-1.0.0/experiments/scripts/flhc.py +75 -0
- cfla-1.0.0/experiments/scripts/hcfl.py +93 -0
- cfla-1.0.0/experiments/scripts/ifca.py +73 -0
- cfla-1.0.0/experiments/scripts/lcfed.py +82 -0
- cfla-1.0.0/experiments/scripts/plot_results.py +396 -0
- cfla-1.0.0/experiments/scripts/run_all_agnews.py +305 -0
- cfla-1.0.0/experiments/scripts/run_all_cifar-10.py +164 -0
- cfla-1.0.0/experiments/scripts/run_all_femnist.py +341 -0
- cfla-1.0.0/experiments/scripts/run_all_mnist.py +164 -0
- cfla-1.0.0/framework/client/__init__.py +1 -0
- cfla-1.0.0/framework/client/client_cgpfl.py +166 -0
- cfla-1.0.0/framework/client/client_fedgroup.py +149 -0
- cfla-1.0.0/framework/client/client_fedper.py +168 -0
- cfla-1.0.0/framework/client/client_fesem.py +153 -0
- cfla-1.0.0/framework/client/client_flhc.py +154 -0
- cfla-1.0.0/framework/client/client_hcfl.py +241 -0
- cfla-1.0.0/framework/client/client_ifca.py +173 -0
- cfla-1.0.0/framework/client/client_lcfed.py +277 -0
- cfla-1.0.0/framework/client/client_madmtop.py +94 -0
- cfla-1.0.0/framework/client/clientbase.py +108 -0
- cfla-1.0.0/framework/common/__init__.py +1 -0
- cfla-1.0.0/framework/common/parameter_tree.py +89 -0
- cfla-1.0.0/framework/common/utils.py +55 -0
- cfla-1.0.0/framework/models/computer_vision.py +167 -0
- cfla-1.0.0/framework/models/large_language_models.py +42 -0
- cfla-1.0.0/framework/models/nlp_models.py +71 -0
- cfla-1.0.0/framework/server/__init__.py +1 -0
- cfla-1.0.0/framework/server/server_cgpfl.py +184 -0
- cfla-1.0.0/framework/server/server_fedavg.py +132 -0
- cfla-1.0.0/framework/server/server_fedgroup.py +229 -0
- cfla-1.0.0/framework/server/server_fedper.py +145 -0
- cfla-1.0.0/framework/server/server_fesem.py +188 -0
- cfla-1.0.0/framework/server/server_flhc.py +261 -0
- cfla-1.0.0/framework/server/server_hcfl.py +356 -0
- cfla-1.0.0/framework/server/server_ifca.py +184 -0
- cfla-1.0.0/framework/server/server_madmtop.py +228 -0
- cfla-1.0.0/framework/server/serverbase.py +28 -0
- cfla-1.0.0/framework/server/serveur_lcfed.py +319 -0
- cfla-1.0.0/pyproject.toml +52 -0
- cfla-1.0.0/setup.cfg +4 -0
|
@@ -0,0 +1,245 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: CFLA
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: CFLA - a framework for clustered federated learning algorithms
|
|
5
|
+
Author-email: HOUNSI Antoine <antoinehounsi3@gmail.com>
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/Totorino02/CFLA
|
|
8
|
+
Classifier: Programming Language :: Python :: 3
|
|
9
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
10
|
+
Classifier: Operating System :: OS Independent
|
|
11
|
+
Requires-Python: >=3.9
|
|
12
|
+
Description-Content-Type: text/markdown
|
|
13
|
+
Requires-Dist: torch
|
|
14
|
+
Requires-Dist: numpy
|
|
15
|
+
Requires-Dist: tqdm
|
|
16
|
+
Requires-Dist: declearn
|
|
17
|
+
Requires-Dist: scikit-learn
|
|
18
|
+
Requires-Dist: pandas
|
|
19
|
+
Requires-Dist: matplotlib
|
|
20
|
+
Provides-Extra: energy
|
|
21
|
+
|
|
22
|
+
# CFLA — Clustered Federated Learning Algorithms
|
|
23
|
+
|
|
24
|
+
A Python library implementing and benchmarking **Clustered Federated Learning (CFL)** algorithms. It includes reproductions of state-of-the-art methods from the literature alongside **HCFL**, an original algorithm that automatically discovers the number of clusters without requiring it as a hyperparameter.
|
|
25
|
+
|
|
26
|
+
> **Paper:** *HCFL: Hierarchical Clustered Federated Learning with Automatic Cluster Discovery* — Antoine Hounsi.
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Overview
|
|
31
|
+
|
|
32
|
+
**Federated Learning (FL)** enables training machine learning models on decentralized data without sharing it. Each client trains locally and only sends model updates to a central server.
|
|
33
|
+
|
|
34
|
+
**Clustered Federated Learning** extends FL by grouping similar clients and training one specialized model per cluster. This improves personalization when client data is heterogeneous (non-IID), which is the common case in practice.
|
|
35
|
+
|
|
36
|
+
CFLA provides a unified framework to implement, run, and compare CFL algorithms on standard benchmarks, with built-in support for energy consumption monitoring.
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## Algorithms
|
|
41
|
+
|
|
42
|
+
### Baselines from the literature
|
|
43
|
+
|
|
44
|
+
| Algorithm | Clustering strategy | Regularization | Reference |
|
|
45
|
+
|-----------|-------------------|----------------|-----------|
|
|
46
|
+
| **FedAvg** | None — single global model | None | McMahan et al., AISTATS 2017 |
|
|
47
|
+
| **FLHC** | Offline — agglomerative on gradient update vectors | None | Briggs et al., 2020 |
|
|
48
|
+
| **FedGroup** | Offline — agglomerative on model params (cosine) | None | Tan et al., 2022 |
|
|
49
|
+
| **FeSEM** | Online — L2 distance to cluster centers (EM) | None | Li et al., 2021 |
|
|
50
|
+
| **CGPFL** | Online — cosine similarity to cluster centers | `(μ/2)‖ω−Ω_k‖²` | Liu et al. |
|
|
51
|
+
| **IFCA** | Online — empirical risk (loss) per cluster | None | Ghosh et al., NeurIPS 2020 |
|
|
52
|
+
| **LCFed** | Online — PCA low-rank projection + cosine | `(μ/2)‖ω−Ω_k‖² + (λ/2)‖φ−Φ‖²` | Zhang et al., ICASSP 2025 |
|
|
53
|
+
|
|
54
|
+
### HCFL (original contribution)
|
|
55
|
+
|
|
56
|
+
HCFL automatically discovers the number of clusters $K^*$ via agglomerative hierarchical clustering on client embedding update vectors, then trains cluster-specific models using a FedProx-style objective toward the cluster center:
|
|
57
|
+
|
|
58
|
+
$$
|
|
59
|
+
\mathcal{L}_i(\omega) = \mathcal{L}_{\sup}(\omega; D_i) + \frac{\mu}{2} \|\omega - \Omega_k\|^2
|
|
60
|
+
$$
|
|
61
|
+
|
|
62
|
+
Inter-cluster knowledge sharing is handled **server-side** via a scheduled blending of cluster models toward the global embedding Φ:
|
|
63
|
+
|
|
64
|
+
$$
|
|
65
|
+
\Omega_k^{(t)} \leftarrow (1 - \lambda^{(t)}) \cdot \text{Avg}\left(\{\omega_i : i \in S_t \cap C_k\}\right) + \lambda^{(t)} \cdot \Phi^{(t)}
|
|
66
|
+
$$
|
|
67
|
+
$$
|
|
68
|
+
\lambda^{(t)} = \frac{\lambda_0}{(1 + \alpha \cdot t)^p}
|
|
69
|
+
$$
|
|
70
|
+
|
|
71
|
+
This decouples two roles: **μ** controls client-to-cluster fidelity (local), **λ(t)** controls inter-cluster sharing (server-side, decaying).
|
|
72
|
+
|
|
73
|
+
**Training procedure:**
|
|
74
|
+
1. **Pre-training** (`R_pre` rounds) — FedAvg warm-up to build a meaningful global representation
|
|
75
|
+
2. **Cluster discovery** — each client computes δᵢ = φᵢ - φ⁽⁰⁾ after one local step; agglomerative clustering on {δᵢ} determines K* automatically
|
|
76
|
+
3. **CFL phase** (`T` rounds) — per-cluster training with server-side embedding blending; clusters specialize progressively as λ(t) → 0
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
## Results
|
|
81
|
+
|
|
82
|
+
Evaluated on MNIST, FEMNIST, and AG News with N=50 clients.
|
|
83
|
+
|
|
84
|
+
| Method | MNIST Acc ± Std | FEMNIST Acc ± Std | AG News Acc ± Std |
|
|
85
|
+
|--------|----------------|-------------------|-------------------|
|
|
86
|
+
| IFCA† | 97.3 ± 0.5 | 11.4 ± 15.5 | 81.5 ± 19.2 |
|
|
87
|
+
| FLHC† | 97.1 ± 0.4 | **35.2 ± 15.1** | 79.1 ± 22.6 |
|
|
88
|
+
| LCFed† | 82.4 ± 10.5 | 6.9 ± 16.3 | 27.4 ± 30.5 |
|
|
89
|
+
| FeSEM | 90.4 ± 12.2 | 12.8 ± 19.2 | 79.4 ± 22.3 |
|
|
90
|
+
| FedGroup | 91.5 ± 2.1 | 5.2 ± 11.2 | 23.0 ± 25.6 |
|
|
91
|
+
| CGPFL | 50.7 ± 7.9 | 12.7 ± 20.5 | 27.9 ± 26.9 |
|
|
92
|
+
| **HCFL (ours)** | **94.3 ± 0.7** | **32.3 ± 16.7** | **82.0 ± 18.3** |
|
|
93
|
+
|
|
94
|
+
†: requires K a priori. **Bold**: best no-K method. **Bold + best overall**: AG News.
|
|
95
|
+
|
|
96
|
+
HCFL is the best no-K method on all three benchmarks, and achieves the best overall accuracy on AG News — surpassing oracle-K methods without any prior knowledge of K.
|
|
97
|
+
|
|
98
|
+
---
|
|
99
|
+
|
|
100
|
+
## Project Structure
|
|
101
|
+
|
|
102
|
+
```
|
|
103
|
+
CFLA/
|
|
104
|
+
├── framework/
|
|
105
|
+
│ ├── client/
|
|
106
|
+
│ │ ├── clientbase.py # Abstract Client base class
|
|
107
|
+
│ │ ├── client_hcfl.py # HCFL client
|
|
108
|
+
│ │ ├── client_flhc.py # FLHC client
|
|
109
|
+
│ │ ├── client_lcfed.py # LCFed client
|
|
110
|
+
│ │ ├── client_fesem.py # FeSEM client
|
|
111
|
+
│ │ ├── client_cgpfl.py # CGPFL client
|
|
112
|
+
│ │ └── client_ifca.py # IFCA client
|
|
113
|
+
│ ├── server/
|
|
114
|
+
│ │ ├── serverbase.py # Abstract Server base class
|
|
115
|
+
│ │ ├── server_hcfl.py # HCFL server
|
|
116
|
+
│ │ ├── server_flhc.py # FLHC server
|
|
117
|
+
│ │ ├── serveur_lcfed.py # LCFed server
|
|
118
|
+
│ │ ├── server_fesem.py # FeSEM server
|
|
119
|
+
│ │ ├── server_cgpfl.py # CGPFL server
|
|
120
|
+
│ │ └── server_ifca.py # IFCA server
|
|
121
|
+
│ ├── models/
|
|
122
|
+
│ │ ├── computer_vision.py # LeNet-5, SplitLeNet5, CNN variants
|
|
123
|
+
│ │ └── nlp_models.py # DistilBERT-based text encoder
|
|
124
|
+
│ └── common/
|
|
125
|
+
│ └── utils.py # flatten_params, average_state_dict, cosine_sim
|
|
126
|
+
├── datasets/
|
|
127
|
+
│ ├── femnist.py # FEMNIST loader + Dirichlet partitioning
|
|
128
|
+
│ └── ag_news.py # AG News loader + Dirichlet partitioning
|
|
129
|
+
├── experiments/
|
|
130
|
+
│ └── scripts/
|
|
131
|
+
│ ├── run_all_mnist.py # Run all algorithms on MNIST
|
|
132
|
+
│ ├── run_all_cifar-10.py # Run all algorithms on CIFAR-10
|
|
133
|
+
│ ├── run_all_femnist.py # Run all algorithms on FEMNIST
|
|
134
|
+
│ ├── run_all_agnews.py # Run all algorithms on AG News
|
|
135
|
+
│ ├── hcfl.py
|
|
136
|
+
│ ├── lcfed.py
|
|
137
|
+
│ ├── flhc.py
|
|
138
|
+
│ ├── fedgroup.py
|
|
139
|
+
│ ├── fesem.py
|
|
140
|
+
│ ├── cgpfl.py
|
|
141
|
+
│ ├── ifca.py
|
|
142
|
+
│ └── plot_results.py # Performance plots + comparison table
|
|
143
|
+
├── pyproject.toml
|
|
144
|
+
└── requirements.dev.txt
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
## Installation
|
|
150
|
+
|
|
151
|
+
```bash
|
|
152
|
+
git clone https://github.com/Totorino02/CFLA.git
|
|
153
|
+
cd CFLA
|
|
154
|
+
pip install -e .
|
|
155
|
+
pip install -r requirements.dev.txt
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
### Energy monitoring (Linux only)
|
|
159
|
+
|
|
160
|
+
Energy tracking via RAPL (CPU) and NVML (GPU) is disabled by default (`monitor_energy: False`).
|
|
161
|
+
To enable it on Linux:
|
|
162
|
+
|
|
163
|
+
```bash
|
|
164
|
+
sudo chmod 444 /sys/class/powercap/intel-rapl:*/energy_uj
|
|
165
|
+
sudo chmod 444 /sys/class/powercap/intel-rapl:*:*/energy_uj
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
Then set `"monitor_energy": True` in the client args of your experiment script.
|
|
169
|
+
|
|
170
|
+
---
|
|
171
|
+
|
|
172
|
+
## Quick Start
|
|
173
|
+
|
|
174
|
+
Run HCFL on MNIST:
|
|
175
|
+
|
|
176
|
+
```bash
|
|
177
|
+
python -m experiments.scripts.hcfl
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
Run all algorithms on a specific dataset:
|
|
181
|
+
|
|
182
|
+
```bash
|
|
183
|
+
python -m experiments.scripts.run_all_mnist
|
|
184
|
+
python -m experiments.scripts.run_all_femnist
|
|
185
|
+
python -m experiments.scripts.run_all_cifar-10
|
|
186
|
+
python -m experiments.scripts.run_all_agnews
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
### Generate plots and comparison table
|
|
190
|
+
|
|
191
|
+
```bash
|
|
192
|
+
python -m experiments.scripts.plot_results --results_dir ./RESULTS/my_run --output_dir ./PLOTS/my_run
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Produces:
|
|
196
|
+
- `accuracy_curves.png` — mean accuracy per round with ±1σ band
|
|
197
|
+
- `loss_curves.png` — mean loss per round
|
|
198
|
+
- `accuracy_boxplot.png` — per-client accuracy distribution at the final round
|
|
199
|
+
- `convergence_speed.png` — rounds needed to reach a target accuracy
|
|
200
|
+
- `comparison_table.csv` — final metrics summary
|
|
201
|
+
|
|
202
|
+
---
|
|
203
|
+
|
|
204
|
+
## Supported Datasets
|
|
205
|
+
|
|
206
|
+
| Dataset | Classes | Input | Partitioning |
|
|
207
|
+
|---------|---------|-------|--------------|
|
|
208
|
+
| **MNIST** | 10 | 28×28 grayscale | Structured non-IID (disjoint class groups) |
|
|
209
|
+
| **CIFAR-10** | 10 | 32×32 RGB | Structured non-IID |
|
|
210
|
+
| **FEMNIST** | 62 | 28×28 grayscale | Dirichlet(α) |
|
|
211
|
+
| **AG News** | 4 | Text | Dirichlet(α) |
|
|
212
|
+
|
|
213
|
+
---
|
|
214
|
+
|
|
215
|
+
## Output Format
|
|
216
|
+
|
|
217
|
+
Each experiment writes results to a local `RESULTS/` directory (not tracked in git):
|
|
218
|
+
|
|
219
|
+
```
|
|
220
|
+
result_{algo}_{dataset}_{timestamp}/
|
|
221
|
+
├── server_metrics.csv # round, mean_acc, std_acc, mean_loss
|
|
222
|
+
└── client_{id}/
|
|
223
|
+
└── metrics.csv # round, loss, accuracy_before, accuracy_after, energy_consumed, energy_ratio
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
---
|
|
227
|
+
|
|
228
|
+
## References
|
|
229
|
+
|
|
230
|
+
- McMahan, B. et al. (2017). Communication-efficient learning of deep networks from decentralized data. *AISTATS*. [arXiv:1602.05629](https://arxiv.org/abs/1602.05629)
|
|
231
|
+
- Briggs, C., Fan, Z., & Andras, P. (2020). Federated learning with hierarchical clustering of local updates to improve training on non-IID data. [arXiv:2004.11791](https://arxiv.org/abs/2004.11791)
|
|
232
|
+
- Ghosh, A. et al. (2020). An efficient framework for clustered federated learning. *NeurIPS*. [arXiv:2006.04088](https://arxiv.org/abs/2006.04088)
|
|
233
|
+
- Tan, Y. et al. (2022). Towards personalized federated learning. *IEEE TNNLS*. [arXiv:2103.00710](https://arxiv.org/abs/2103.00710)
|
|
234
|
+
- Li, X. et al. (2021). FeSEM: Federated learning via expectation maximization. *IEEE TPAMI*.
|
|
235
|
+
- Zhang, Y. et al. (2025). LCFed: An efficient clustered federated learning framework for heterogeneous data. *ICASSP*. [arXiv:2501.01850](https://arxiv.org/abs/2501.01850)
|
|
236
|
+
- Zhang, Y. et al. (2015). Character-level convolutional networks for text classification. *NeurIPS*. [arXiv:1509.01626](https://arxiv.org/abs/1509.01626)
|
|
237
|
+
- Caldas, S. et al. (2019). LEAF: A benchmark for federated settings. [arXiv:1812.01097](https://arxiv.org/abs/1812.01097)
|
|
238
|
+
|
|
239
|
+
---
|
|
240
|
+
|
|
241
|
+
## Author
|
|
242
|
+
|
|
243
|
+
Antoine Hounsi — [antoinehounsi3@gmail.com](mailto:antoinehounsi3@gmail.com)
|
|
244
|
+
|
|
245
|
+
Master's research project, Université de Lille.
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
MANIFEST.in
|
|
2
|
+
README.md
|
|
3
|
+
pyproject.toml
|
|
4
|
+
CFLA.egg-info/PKG-INFO
|
|
5
|
+
CFLA.egg-info/SOURCES.txt
|
|
6
|
+
CFLA.egg-info/dependency_links.txt
|
|
7
|
+
CFLA.egg-info/requires.txt
|
|
8
|
+
CFLA.egg-info/top_level.txt
|
|
9
|
+
datasets/__init__.py
|
|
10
|
+
datasets/ag_news.py
|
|
11
|
+
datasets/dataset_utils.py
|
|
12
|
+
datasets/emnist.py
|
|
13
|
+
datasets/femnist.py
|
|
14
|
+
experiments/scripts/cgpfl.py
|
|
15
|
+
experiments/scripts/fedgroup.py
|
|
16
|
+
experiments/scripts/fedper.py
|
|
17
|
+
experiments/scripts/fesem.py
|
|
18
|
+
experiments/scripts/flhc.py
|
|
19
|
+
experiments/scripts/hcfl.py
|
|
20
|
+
experiments/scripts/ifca.py
|
|
21
|
+
experiments/scripts/lcfed.py
|
|
22
|
+
experiments/scripts/plot_results.py
|
|
23
|
+
experiments/scripts/run_all_agnews.py
|
|
24
|
+
experiments/scripts/run_all_cifar-10.py
|
|
25
|
+
experiments/scripts/run_all_femnist.py
|
|
26
|
+
experiments/scripts/run_all_mnist.py
|
|
27
|
+
framework/client/__init__.py
|
|
28
|
+
framework/client/client_cgpfl.py
|
|
29
|
+
framework/client/client_fedgroup.py
|
|
30
|
+
framework/client/client_fedper.py
|
|
31
|
+
framework/client/client_fesem.py
|
|
32
|
+
framework/client/client_flhc.py
|
|
33
|
+
framework/client/client_hcfl.py
|
|
34
|
+
framework/client/client_ifca.py
|
|
35
|
+
framework/client/client_lcfed.py
|
|
36
|
+
framework/client/client_madmtop.py
|
|
37
|
+
framework/client/clientbase.py
|
|
38
|
+
framework/common/__init__.py
|
|
39
|
+
framework/common/parameter_tree.py
|
|
40
|
+
framework/common/utils.py
|
|
41
|
+
framework/models/computer_vision.py
|
|
42
|
+
framework/models/large_language_models.py
|
|
43
|
+
framework/models/nlp_models.py
|
|
44
|
+
framework/server/__init__.py
|
|
45
|
+
framework/server/server_cgpfl.py
|
|
46
|
+
framework/server/server_fedavg.py
|
|
47
|
+
framework/server/server_fedgroup.py
|
|
48
|
+
framework/server/server_fedper.py
|
|
49
|
+
framework/server/server_fesem.py
|
|
50
|
+
framework/server/server_flhc.py
|
|
51
|
+
framework/server/server_hcfl.py
|
|
52
|
+
framework/server/server_ifca.py
|
|
53
|
+
framework/server/server_madmtop.py
|
|
54
|
+
framework/server/serverbase.py
|
|
55
|
+
framework/server/serveur_lcfed.py
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
PLOTS
|
|
2
|
+
PLOTS_
|
|
3
|
+
PLOTS_0_NOISE
|
|
4
|
+
PLOTS__
|
|
5
|
+
RESULT
|
|
6
|
+
RESULTS
|
|
7
|
+
RESULTSS
|
|
8
|
+
RESULTS_
|
|
9
|
+
RESULTS_0_NOISE
|
|
10
|
+
RESULTS_FEDPER
|
|
11
|
+
RESULTS_PATHO
|
|
12
|
+
RESULTS__
|
|
13
|
+
RESULTS_covariate
|
|
14
|
+
data
|
|
15
|
+
datasets
|
|
16
|
+
dist
|
|
17
|
+
experiments
|
|
18
|
+
framework
|
|
19
|
+
paper
|
|
20
|
+
result_hcfl-c_mnist_26-03-29_01-54
|
cfla-1.0.0/MANIFEST.in
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
prune .idea
|
|
2
|
+
prune data
|
|
3
|
+
prune results
|
|
4
|
+
prune .vscode
|
|
5
|
+
prune .claude
|
|
6
|
+
prune paper
|
|
7
|
+
prune datasets/FEMNIST
|
|
8
|
+
prune tmp
|
|
9
|
+
exclude tmp.py
|
|
10
|
+
exclude **/__pycache__/*
|
|
11
|
+
exclude test*
|
|
12
|
+
exclude *RESULT*
|
|
13
|
+
exclude *images*
|
|
14
|
+
exclude *.log
|
|
15
|
+
exclude *.egg-info
|
|
16
|
+
exclude result_*
|
|
17
|
+
exclude PLOTS*
|
|
18
|
+
exclude experiments/scripts/main_*.py
|
|
19
|
+
exclude experiments/scripts/ablation_*.py
|
|
20
|
+
exclude experiments/scripts/plot_sharing*.py
|
cfla-1.0.0/PKG-INFO
ADDED
|
@@ -0,0 +1,245 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: CFLA
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: CFLA - a framework for clustered federated learning algorithms
|
|
5
|
+
Author-email: HOUNSI Antoine <antoinehounsi3@gmail.com>
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/Totorino02/CFLA
|
|
8
|
+
Classifier: Programming Language :: Python :: 3
|
|
9
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
10
|
+
Classifier: Operating System :: OS Independent
|
|
11
|
+
Requires-Python: >=3.9
|
|
12
|
+
Description-Content-Type: text/markdown
|
|
13
|
+
Requires-Dist: torch
|
|
14
|
+
Requires-Dist: numpy
|
|
15
|
+
Requires-Dist: tqdm
|
|
16
|
+
Requires-Dist: declearn
|
|
17
|
+
Requires-Dist: scikit-learn
|
|
18
|
+
Requires-Dist: pandas
|
|
19
|
+
Requires-Dist: matplotlib
|
|
20
|
+
Provides-Extra: energy
|
|
21
|
+
|
|
22
|
+
# CFLA — Clustered Federated Learning Algorithms
|
|
23
|
+
|
|
24
|
+
A Python library implementing and benchmarking **Clustered Federated Learning (CFL)** algorithms. It includes reproductions of state-of-the-art methods from the literature alongside **HCFL**, an original algorithm that automatically discovers the number of clusters without requiring it as a hyperparameter.
|
|
25
|
+
|
|
26
|
+
> **Paper:** *HCFL: Hierarchical Clustered Federated Learning with Automatic Cluster Discovery* — Antoine Hounsi.
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Overview
|
|
31
|
+
|
|
32
|
+
**Federated Learning (FL)** enables training machine learning models on decentralized data without sharing it. Each client trains locally and only sends model updates to a central server.
|
|
33
|
+
|
|
34
|
+
**Clustered Federated Learning** extends FL by grouping similar clients and training one specialized model per cluster. This improves personalization when client data is heterogeneous (non-IID), which is the common case in practice.
|
|
35
|
+
|
|
36
|
+
CFLA provides a unified framework to implement, run, and compare CFL algorithms on standard benchmarks, with built-in support for energy consumption monitoring.
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## Algorithms
|
|
41
|
+
|
|
42
|
+
### Baselines from the literature
|
|
43
|
+
|
|
44
|
+
| Algorithm | Clustering strategy | Regularization | Reference |
|
|
45
|
+
|-----------|-------------------|----------------|-----------|
|
|
46
|
+
| **FedAvg** | None — single global model | None | McMahan et al., AISTATS 2017 |
|
|
47
|
+
| **FLHC** | Offline — agglomerative on gradient update vectors | None | Briggs et al., 2020 |
|
|
48
|
+
| **FedGroup** | Offline — agglomerative on model params (cosine) | None | Tan et al., 2022 |
|
|
49
|
+
| **FeSEM** | Online — L2 distance to cluster centers (EM) | None | Li et al., 2021 |
|
|
50
|
+
| **CGPFL** | Online — cosine similarity to cluster centers | `(μ/2)‖ω−Ω_k‖²` | Liu et al. |
|
|
51
|
+
| **IFCA** | Online — empirical risk (loss) per cluster | None | Ghosh et al., NeurIPS 2020 |
|
|
52
|
+
| **LCFed** | Online — PCA low-rank projection + cosine | `(μ/2)‖ω−Ω_k‖² + (λ/2)‖φ−Φ‖²` | Zhang et al., ICASSP 2025 |
|
|
53
|
+
|
|
54
|
+
### HCFL (original contribution)
|
|
55
|
+
|
|
56
|
+
HCFL automatically discovers the number of clusters $K^*$ via agglomerative hierarchical clustering on client embedding update vectors, then trains cluster-specific models using a FedProx-style objective toward the cluster center:
|
|
57
|
+
|
|
58
|
+
$$
|
|
59
|
+
\mathcal{L}_i(\omega) = \mathcal{L}_{\sup}(\omega; D_i) + \frac{\mu}{2} \|\omega - \Omega_k\|^2
|
|
60
|
+
$$
|
|
61
|
+
|
|
62
|
+
Inter-cluster knowledge sharing is handled **server-side** via a scheduled blending of cluster models toward the global embedding Φ:
|
|
63
|
+
|
|
64
|
+
$$
|
|
65
|
+
\Omega_k^{(t)} \leftarrow (1 - \lambda^{(t)}) \cdot \text{Avg}\left(\{\omega_i : i \in S_t \cap C_k\}\right) + \lambda^{(t)} \cdot \Phi^{(t)}
|
|
66
|
+
$$
|
|
67
|
+
$$
|
|
68
|
+
\lambda^{(t)} = \frac{\lambda_0}{(1 + \alpha \cdot t)^p}
|
|
69
|
+
$$
|
|
70
|
+
|
|
71
|
+
This decouples two roles: **μ** controls client-to-cluster fidelity (local), **λ(t)** controls inter-cluster sharing (server-side, decaying).
|
|
72
|
+
|
|
73
|
+
**Training procedure:**
|
|
74
|
+
1. **Pre-training** (`R_pre` rounds) — FedAvg warm-up to build a meaningful global representation
|
|
75
|
+
2. **Cluster discovery** — each client computes δᵢ = φᵢ - φ⁽⁰⁾ after one local step; agglomerative clustering on {δᵢ} determines K* automatically
|
|
76
|
+
3. **CFL phase** (`T` rounds) — per-cluster training with server-side embedding blending; clusters specialize progressively as λ(t) → 0
|
|
77
|
+
|
|
78
|
+
---
|
|
79
|
+
|
|
80
|
+
## Results
|
|
81
|
+
|
|
82
|
+
Evaluated on MNIST, FEMNIST, and AG News with N=50 clients.
|
|
83
|
+
|
|
84
|
+
| Method | MNIST Acc ± Std | FEMNIST Acc ± Std | AG News Acc ± Std |
|
|
85
|
+
|--------|----------------|-------------------|-------------------|
|
|
86
|
+
| IFCA† | 97.3 ± 0.5 | 11.4 ± 15.5 | 81.5 ± 19.2 |
|
|
87
|
+
| FLHC† | 97.1 ± 0.4 | **35.2 ± 15.1** | 79.1 ± 22.6 |
|
|
88
|
+
| LCFed† | 82.4 ± 10.5 | 6.9 ± 16.3 | 27.4 ± 30.5 |
|
|
89
|
+
| FeSEM | 90.4 ± 12.2 | 12.8 ± 19.2 | 79.4 ± 22.3 |
|
|
90
|
+
| FedGroup | 91.5 ± 2.1 | 5.2 ± 11.2 | 23.0 ± 25.6 |
|
|
91
|
+
| CGPFL | 50.7 ± 7.9 | 12.7 ± 20.5 | 27.9 ± 26.9 |
|
|
92
|
+
| **HCFL (ours)** | **94.3 ± 0.7** | **32.3 ± 16.7** | **82.0 ± 18.3** |
|
|
93
|
+
|
|
94
|
+
†: requires K a priori. **Bold**: best no-K method. **Bold + best overall**: AG News.
|
|
95
|
+
|
|
96
|
+
HCFL is the best no-K method on all three benchmarks, and achieves the best overall accuracy on AG News — surpassing oracle-K methods without any prior knowledge of K.
|
|
97
|
+
|
|
98
|
+
---
|
|
99
|
+
|
|
100
|
+
## Project Structure
|
|
101
|
+
|
|
102
|
+
```
|
|
103
|
+
CFLA/
|
|
104
|
+
├── framework/
|
|
105
|
+
│ ├── client/
|
|
106
|
+
│ │ ├── clientbase.py # Abstract Client base class
|
|
107
|
+
│ │ ├── client_hcfl.py # HCFL client
|
|
108
|
+
│ │ ├── client_flhc.py # FLHC client
|
|
109
|
+
│ │ ├── client_lcfed.py # LCFed client
|
|
110
|
+
│ │ ├── client_fesem.py # FeSEM client
|
|
111
|
+
│ │ ├── client_cgpfl.py # CGPFL client
|
|
112
|
+
│ │ └── client_ifca.py # IFCA client
|
|
113
|
+
│ ├── server/
|
|
114
|
+
│ │ ├── serverbase.py # Abstract Server base class
|
|
115
|
+
│ │ ├── server_hcfl.py # HCFL server
|
|
116
|
+
│ │ ├── server_flhc.py # FLHC server
|
|
117
|
+
│ │ ├── serveur_lcfed.py # LCFed server
|
|
118
|
+
│ │ ├── server_fesem.py # FeSEM server
|
|
119
|
+
│ │ ├── server_cgpfl.py # CGPFL server
|
|
120
|
+
│ │ └── server_ifca.py # IFCA server
|
|
121
|
+
│ ├── models/
|
|
122
|
+
│ │ ├── computer_vision.py # LeNet-5, SplitLeNet5, CNN variants
|
|
123
|
+
│ │ └── nlp_models.py # DistilBERT-based text encoder
|
|
124
|
+
│ └── common/
|
|
125
|
+
│ └── utils.py # flatten_params, average_state_dict, cosine_sim
|
|
126
|
+
├── datasets/
|
|
127
|
+
│ ├── femnist.py # FEMNIST loader + Dirichlet partitioning
|
|
128
|
+
│ └── ag_news.py # AG News loader + Dirichlet partitioning
|
|
129
|
+
├── experiments/
|
|
130
|
+
│ └── scripts/
|
|
131
|
+
│ ├── run_all_mnist.py # Run all algorithms on MNIST
|
|
132
|
+
│ ├── run_all_cifar-10.py # Run all algorithms on CIFAR-10
|
|
133
|
+
│ ├── run_all_femnist.py # Run all algorithms on FEMNIST
|
|
134
|
+
│ ├── run_all_agnews.py # Run all algorithms on AG News
|
|
135
|
+
│ ├── hcfl.py
|
|
136
|
+
│ ├── lcfed.py
|
|
137
|
+
│ ├── flhc.py
|
|
138
|
+
│ ├── fedgroup.py
|
|
139
|
+
│ ├── fesem.py
|
|
140
|
+
│ ├── cgpfl.py
|
|
141
|
+
│ ├── ifca.py
|
|
142
|
+
│ └── plot_results.py # Performance plots + comparison table
|
|
143
|
+
├── pyproject.toml
|
|
144
|
+
└── requirements.dev.txt
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
## Installation
|
|
150
|
+
|
|
151
|
+
```bash
|
|
152
|
+
git clone https://github.com/Totorino02/CFLA.git
|
|
153
|
+
cd CFLA
|
|
154
|
+
pip install -e .
|
|
155
|
+
pip install -r requirements.dev.txt
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
### Energy monitoring (Linux only)
|
|
159
|
+
|
|
160
|
+
Energy tracking via RAPL (CPU) and NVML (GPU) is disabled by default (`monitor_energy: False`).
|
|
161
|
+
To enable it on Linux:
|
|
162
|
+
|
|
163
|
+
```bash
|
|
164
|
+
sudo chmod 444 /sys/class/powercap/intel-rapl:*/energy_uj
|
|
165
|
+
sudo chmod 444 /sys/class/powercap/intel-rapl:*:*/energy_uj
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
Then set `"monitor_energy": True` in the client args of your experiment script.
|
|
169
|
+
|
|
170
|
+
---
|
|
171
|
+
|
|
172
|
+
## Quick Start
|
|
173
|
+
|
|
174
|
+
Run HCFL on MNIST:
|
|
175
|
+
|
|
176
|
+
```bash
|
|
177
|
+
python -m experiments.scripts.hcfl
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
Run all algorithms on a specific dataset:
|
|
181
|
+
|
|
182
|
+
```bash
|
|
183
|
+
python -m experiments.scripts.run_all_mnist
|
|
184
|
+
python -m experiments.scripts.run_all_femnist
|
|
185
|
+
python -m experiments.scripts.run_all_cifar-10
|
|
186
|
+
python -m experiments.scripts.run_all_agnews
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
### Generate plots and comparison table
|
|
190
|
+
|
|
191
|
+
```bash
|
|
192
|
+
python -m experiments.scripts.plot_results --results_dir ./RESULTS/my_run --output_dir ./PLOTS/my_run
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Produces:
|
|
196
|
+
- `accuracy_curves.png` — mean accuracy per round with ±1σ band
|
|
197
|
+
- `loss_curves.png` — mean loss per round
|
|
198
|
+
- `accuracy_boxplot.png` — per-client accuracy distribution at the final round
|
|
199
|
+
- `convergence_speed.png` — rounds needed to reach a target accuracy
|
|
200
|
+
- `comparison_table.csv` — final metrics summary
|
|
201
|
+
|
|
202
|
+
---
|
|
203
|
+
|
|
204
|
+
## Supported Datasets
|
|
205
|
+
|
|
206
|
+
| Dataset | Classes | Input | Partitioning |
|
|
207
|
+
|---------|---------|-------|--------------|
|
|
208
|
+
| **MNIST** | 10 | 28×28 grayscale | Structured non-IID (disjoint class groups) |
|
|
209
|
+
| **CIFAR-10** | 10 | 32×32 RGB | Structured non-IID |
|
|
210
|
+
| **FEMNIST** | 62 | 28×28 grayscale | Dirichlet(α) |
|
|
211
|
+
| **AG News** | 4 | Text | Dirichlet(α) |
|
|
212
|
+
|
|
213
|
+
---
|
|
214
|
+
|
|
215
|
+
## Output Format
|
|
216
|
+
|
|
217
|
+
Each experiment writes results to a local `RESULTS/` directory (not tracked in git):
|
|
218
|
+
|
|
219
|
+
```
|
|
220
|
+
result_{algo}_{dataset}_{timestamp}/
|
|
221
|
+
├── server_metrics.csv # round, mean_acc, std_acc, mean_loss
|
|
222
|
+
└── client_{id}/
|
|
223
|
+
└── metrics.csv # round, loss, accuracy_before, accuracy_after, energy_consumed, energy_ratio
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
---
|
|
227
|
+
|
|
228
|
+
## References
|
|
229
|
+
|
|
230
|
+
- McMahan, B. et al. (2017). Communication-efficient learning of deep networks from decentralized data. *AISTATS*. [arXiv:1602.05629](https://arxiv.org/abs/1602.05629)
|
|
231
|
+
- Briggs, C., Fan, Z., & Andras, P. (2020). Federated learning with hierarchical clustering of local updates to improve training on non-IID data. [arXiv:2004.11791](https://arxiv.org/abs/2004.11791)
|
|
232
|
+
- Ghosh, A. et al. (2020). An efficient framework for clustered federated learning. *NeurIPS*. [arXiv:2006.04088](https://arxiv.org/abs/2006.04088)
|
|
233
|
+
- Tan, Y. et al. (2022). Towards personalized federated learning. *IEEE TNNLS*. [arXiv:2103.00710](https://arxiv.org/abs/2103.00710)
|
|
234
|
+
- Li, X. et al. (2021). FeSEM: Federated learning via expectation maximization. *IEEE TPAMI*.
|
|
235
|
+
- Zhang, Y. et al. (2025). LCFed: An efficient clustered federated learning framework for heterogeneous data. *ICASSP*. [arXiv:2501.01850](https://arxiv.org/abs/2501.01850)
|
|
236
|
+
- Zhang, Y. et al. (2015). Character-level convolutional networks for text classification. *NeurIPS*. [arXiv:1509.01626](https://arxiv.org/abs/1509.01626)
|
|
237
|
+
- Caldas, S. et al. (2019). LEAF: A benchmark for federated settings. [arXiv:1812.01097](https://arxiv.org/abs/1812.01097)
|
|
238
|
+
|
|
239
|
+
---
|
|
240
|
+
|
|
241
|
+
## Author
|
|
242
|
+
|
|
243
|
+
Antoine Hounsi — [antoinehounsi3@gmail.com](mailto:antoinehounsi3@gmail.com)
|
|
244
|
+
|
|
245
|
+
Master's research project, Université de Lille.
|