Enabling Federated Learning with Small, Low Power, and Computationally Constrained Devices: A Decentralized-Client-Server Approach
Abstract
Federated Learning (FL) has recently emerged as an important distributed learning paradigm that fosters collaboration among multiple edge devices with private data sources. To facilitate distributed learning in FL, there are two major types of system architectures: i) the client-server architecture and ii) the fully decentralized architecture. However, despite the significant advances that have been made since the inception of FL, both architectures are inherently unsuitable for deploying efficient FL with small, low power, and computationally constrained devices: 1) Under the client-server architecture, the fact that each client has to reach and exchange information with the central server in one hop requires having sufficiently strong communication power, which could be challenging for low-power devices; 2) While a fully decentralized FL system architecture allows the use of small and low-power devices and short-range local communication, the lack of a central server results in a much longer period for the FL system to reach consensus and converge, which is also unfriendly for low-power and computationally constrained devices.In light of the gap between the rapidly growing demand for FL and the practical constraints of power and computation capabilities of small-edge network devices, we propose building FL systems based on a hybrid decentralized-client-server networking-computing architecture, which enjoys the best of both client-server and fully decentralized architectures, while avoiding their pitfalls. Our proposed decentralized client-server federatedlearning (DCS-FL) architecture is inherently cluster-based and hierarchical, where the cluster heads form a decentralized peer-to-peer network at the upper layer and nodes in a cluster at the lower layer follows a local client-server architecture. More importantly, our new DCS-FL enables three solution approaches to enable federated learning with small, low-power, and computationally constrained devices, which also constitute three objectives in this project: 1) Adaptive Clustering for Two-Tier Data Heterogeneity Mitigation in DCS-FL;2) Anarchic Federated Learning at the Lower Layer of DCS-FL; and 3) Decentralized FL Convergence Acceleration at the Upper Layer of DCS-FL. Collectively, the outcomes of this research will have a far-reaching impact on advancing communication- and computation-efficient FL algorithm development with small, low-power, and computationally constrained devices. Moreover, our proposed research is envisioned to serve a timely and critical need in the broadly defined machine learning, networking, signal processing, and artificial intelligence research communities by exploring a fundamental understanding of decentralized-client-server FL, and developing efficient learning algorithms. This project addresses TA 2: Computationally Efficient, Low Power, Federated Learning.This abstract is approved for public release by the applicant.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Nov 09, 2024
- Source ID
- N000142412729
Entities
People
- Jia Liu
Organizations
- Office of Naval Research
- Ohio State University
- United States Navy