Hybrid Encoding for Singed Expressions (HESE) and Direct HESE Analog-to-Digital Converters
Abstract
The goal of the research is to develop hardware-efficient algorithms and ADCs suitable for use in analog neural networks (ANNs). While analog processing can for MAC (multiply and accumulate) be more power and area efficient compared with digital processing, the results must be converted to digital values for further processing such as activation, and interface with the next level of neural networks.This research exploits bit-level sparsity, present in both model weight and activation data values, for efficient convolutional neural network (CNN) inference. A value with bit-level sparsity, expressed as a sum of decreasing power-of-two terms, has gaps between consecutive nonzero terms that do not contribute to the CNN inference computation. We propose to study three novel architecture optimizations at the term level to leverage the presence of such sparsity,without significantly impacting accuracy performance.To fully utilize a large number of parallel analog processing elements (APEs), parallel ADCs are strongly desired. Besides, to keep up with the processing speed of APEs, the sampling rate of the ADCs must be high. In this research, we propose an ADC architecture that provides 12-bit resolution at 100MS/s, while occupying a small area that can be pitch matched with APEs.Since HESE is much more hardware efficient in processing MAC, we propose a novel ADC architecture that produces a direct HESE digital output, thus avoiding the need for a separate binary-to-HESE encoder for each ADC. Moreover, the direct HESE ADC algorithm pre-detects and skips consecutive 1s, shortening the bit cycle time and lowering the conversion energy.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Aug 31, 2020
- Source ID
- N000142014005
Entities
People
- Hae-seung Lee
Organizations
- Massachusetts Institute of Technology
- Office of Naval Research
- United States Navy