HAMILTON-JACOBI PDE METHODS TO ACCELERATE TRAINING IN DEEP LEARNING

Abstract

A major challenge for the deep learning research community is to better understand generalization. Generalization means that the trained DNN performs effectively on relevant data outside the training set. Generalization as currently understood, is not mathematically tractable, since it involves a combination of data, network architecture, and regularization, along with training methods. Our approach facilitates generalization by widening the local minima at various scales via PDE methods, in particular the Hamilton-Jacobi (HJ) equation. This reduces classification error (trapping in non-global minima) and speeds-up convergence.

Document Details

Document Type
DoD Grant Award
Publication Date
May 30, 2018
Source ID
FA95501810167

Entities

People

  • Stanley Osher

Organizations

  • Air Force Office of Scientific Research
  • United States Air Force
  • University of California, Los Angeles

Tags

Fields of Study

  • Computer science

Readers

  • Calculus or Mathematical Analysis
  • Neural Network Machine Learning.
  • Operations Research

Technology Areas

  • AI & ML
  • AI & ML - Machine Learning Algorithms
  • AI & ML - Neural Networks