Dynamic Function Placement in Active Storage Clusters

Abstract

Optimally partitioning application and file system functionality within a cluster of clients and servers is a difficult problem due to dynamic variations in application behavior, resource availability and workload mixes. This paper presents ABACUS, a run-time system that monitors and dynamically changes function placement for applications that manipulate large data sets. Several examples of data-intensive workloads are used to show the importance of proper function placement and its dependence on dynamic runtime characteristics, with performance differences frequently reaching 2-10X. We evaluate how well the ABACUS prototype adapts to run-time system behavior, including both long-term variation (e.g., filter selectivity) and short-term variation (e.g.,multi-phase applications and inter-application resource contention). Our experiments with ABACUS indicate that it is possible to adapt in all of these situations and that the adaptation converges most quickly in those cases where the performance impact is most significant.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Jun 01, 1999
Accession Number
ADA461222

Entities

People

  • David Petrou
  • Garth Gibson
  • Gregory Ganger
  • Khalil Amiri

Organizations

  • Carnegie Mellon University

Tags

Communities of Interest

  • Energy and Power Technologies

DTIC Thesaurus Topics

  • Algorithms
  • Bandwidth
  • Computations
  • Computer Programming
  • Computer Science
  • Computers
  • Data Sets
  • Directories
  • Equations
  • Information Science
  • Instructions
  • Local Area Networks
  • Monitoring
  • Network Topology
  • Networks
  • Operating Systems
  • Statistics

Fields of Study

  • Computer science

Readers

  • Distributed Systems and Data Platform Development
  • Parallel and Distributed Computing.
  • Vision Science/Vision Psychology/Cognitive Neuroscience.