One repo, fifty federated learning methods, zero reimplementation

What it does

FL-bench is a benchmarking framework for federated learning research. It ships unified PyTorch implementations of a large catalog of methods — classic ones like FedAvg, FedProx, SCAFFOLD and MOON, plus a long tail of personalized FL (Ditto, FedRep, pFedMe, kNN-Per…), domain generalization variants, and a differentially private DP-FedAvg. Everything runs through the same dataset generation, config, and experiment pipeline, so comparisons are at least apples-to-apples-ish.

The interesting bit

The architecture is deliberately boring in the good way: every method inherits from FedAvgServer and FedAvgClient, so adding your own algorithm means subclassing two well-understood classes rather than wiring up a new distributed system. Hyperparameters resolve in a sane priority order — CLI over config file over defaults — which is exactly the kind of unglamorous detail that makes benchmarking tolerable.

Key highlights

• 50+ FL methods spanning traditional, personalized, domain generalization, and differential privacy flavors, each linked to its paper.
• Dataset partitioning tools (e.g., Dirichlet-based non-IID splits) for building federated scenarios.
• Optional parallel training via Ray to speed up experiments.
• Monitoring through Visdom or TensorBoard.
• The maintainer’s own pFedSim is included — marked with a star, which is at least honest self-promotion.

Caveats

• The README doesn’t state which datasets or models are supported out of the box beyond an MNIST example, so scope beyond that is unclear.
• Visdom/TensorBoard servers must be launched by the user; monitoring isn’t fully turnkey.

Verdict

If you’re an FL researcher tired of reimplementing FedAvg as a baseline for the tenth time, this is worth a look. If you just want to deploy federated learning in production, this is a benchmark, not a framework — look elsewhere.