Scaling to Multiple Graphics Processing Units (GPUs) in TensorFlow

Abstract

Although accuracies of neural networks are surpassing human performance, training a deep neural network is a time-consuming task due to its increasing high-dimensional parameters. It is not uncommon for the training of deep neural networks to run for a week. Accordingly, the size of neural networks has doubled every 2.4 years, exhibiting an exponential growth from 1958 to 2014. The increasing size of neural network architectures will likely lead to higher computational complexity that will need scalable solutions. To mitigate the computational requirement and maximize throughput, this work focuses on multi-graphics-processing-unit scalability.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Nov 01, 2018
Accession Number
AD1063956

Entities

People

  • Song J. Park

Organizations

  • United States Army Research Laboratory

Tags

Communities of Interest

  • Autonomy

DTIC Thesaurus Topics

  • Artificial Intelligence
  • Artificial Intelligence Computing
  • Artificial Intelligence Software
  • Bayesian Networks
  • Computer Programming
  • Computers
  • Computing System Architectures
  • Deep Learning
  • Dimensionality Reduction
  • Graphics
  • Graphics Processing Unit
  • Information Science
  • Machine Learning
  • Networks
  • Neural Networks
  • Probabilistic Models
  • Signal Processing

Fields of Study

  • Computer science

Readers

  • Neural Network Machine Learning.
  • Parallel and Distributed Computing.

Technology Areas

  • AI & ML
  • AI & ML - Neural Networks