Collectives for Multiple Resource Job Scheduling Across Heterogeneous Servers
Abstract
Efficient management of large-scale, distributed data storage and processing systems is a major challenge for many computational applications. Many of these systems are characterized by multi resource tasks processed across a heterogeneous network. Conventional approaches, such as load balancing, work well for centralized, single resource problems, but breakdown in the more general case. In addition, most approaches are often based on heuristics which do not directly attempt to optimize the world utility. In this paper, we propose an agent based control system using the theory of collectives. We configure the servers of our network with agents who make local job scheduling decisions. These decisions are based on local goals which are constructed to be aligned with the objective of optimizing the overall efficiency of the system. We demonstrate that agents configured using collectives outperform both team games and load balancing, by up to four times for the latter.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jan 14, 2005
- Accession Number
- ADP020801
Entities
People
- John Lawson
- Kagan Tumer
Organizations
- National Aeronautics and Space Administration