Next-Generation All Flash Big Data Parallel Processing Engine for Mobile Computing
Abstract
Publically Releasable With the explosive growth in mobile application data volume, real time big data analytical frameworks on the myriad wireless network input data have been trending. Data analysis jobs for mobile computing are often performed at the application level by selecting data based on semantic meanings without any knowledge of the lower level data placement. For todayÕs widely deployed storage server Ñ All Flash Solid State Disk Arrays, the I/O performance is highly dependent on the data access pattern. Random workloads will result in severe performance degradation for SSDs. One of the main reasons for this is SSDsÕ internal bandwidth contention. In this work, we propose Non-Volatile MapReduce (NVMR), which is a framework that leverages the tolerance of data selection and task scheduler in many applications to perform data-layout aware parallel data analysis on the SSDs. Aiming to minimize the read latency, NVMR not only uses data-layout aware task scheduler to balance workloads on open-channel SSDs but also utilizes delay reflection to avoid occasional contentions. We have developed a prototype system for the NVMR in Scala. Preliminary evaluation results show that the data sampling application can achieve up to 2.7 speedups compared to Spark by maintaining high output accuracy alongside. Our research will rely on the massive storage infrastructure enhanced by the requested equipment, will make significant contributions to our existing projects and foster future Army, Navy and Airforce related projects relating to mobile computing and big data. We already implemented a RAM disk-based emulator to evaluate NVMR on a small-scale testbed (4-node) with simple applications. The requested equipment will enable a larger testbed (16-node) for more complex applications. Besides, the testbed gives us a unique opportunity to prototype the SSD with correlation aware FTL.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Sep 11, 2018
- Source ID
- W911NF1710208
Entities
People
- Jun Wang
Organizations
- Army Contracting Command
- United States Army
- University of Central Florida