Optimization for Distributed Machine Learning
Abstract
The primary objective of this research is to design efficient and resilient algorithms for machine learning performed by a large number of heterogeneous agents distributed over a network. Army s operations include large heterogeneous teams of intelligent agents including airfighters, autonomous vehicles and Soldiers etc. Today s Army is already facing a deluge of distributed data collected by these agents that cannot be effectively processed in real time. It is likely that this problem will only get worse as more sensors and autonomous platforms are introduced and more knowledge bases can be conveniently accessed in the cloud. During the last few years, significant progress has been made in the area of machine learning which helps to transform data of sheer size into knowledge to support operations. However, existing machine learning algorithms are not capable of dealing with a few significant challenges, such as communication efficiency, asynchronous optimization and heterogeneous infrastructure, and differential privacy under the aforementioned distributed setting. In this research, we propose to study algorithms for solving two different types of distributed machine learning problems: decentralized learning and federated learning. The first one does not have a central node and each agent can only communicate with its own neighbors. We focus on the development of asynchronous and communication-efficient stochastic optimization algorithms for distributed learning under such decentralized settings. In the second problem, i.e., federated learning, there exists a central node in the network to maintain the updating of decision variables, even though the data set is still distributed over different agents. We propose to study novel asynchronous randomized gradient methods for solving these federated learning problems. Moreover, we will study new differential privacy techniques for general distributed convex optimization and the incorporation of these techniques into our proposed algorithmic framework for the both decentralized and federated learning. If successful, the proposed research will provide a novel set of algorithmic tools that can significantly advance the state of the art for distributed machine learning, which will ultimately help to support Army s future operations through extracting useful knowledge from distributed raw information.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Feb 14, 2019
- Source ID
- W911NF1810223
Entities
People
- Guanghui Lan
Organizations
- Army Contracting Command
- Georgia Tech Research Corporation
- United States Army