Application of a Big Data Platform for Binary Data
Abstract
Using big data is essential for survival in any organization. While most data used in the workplace in the past were structured, the use of unstructured data is dramatically increasing and accounts for most of data growth. Handling unstructured big data is now significant for success in organizations. Sonar files are an example of unstructured big data used in the U.S. Navy. The Navy stores sonar data on separate data storage devices such as the hard disks of laptops and CDs. When users need to make a decision by analyzing the data, they must manually search for the relevant file across the storage system and physically transfer it to the proper applications to be read, which delays decision making. This thesis suggests a way to handle this challenge by using a big data platform. In this thesis, the performance of a big data platform is compared to that of the standalone system. Specifically, the binary data of sonar files were extracted and converted into metadata, which can be used for a query in the Oracle relational database and Application Express (APEX). As an example of big data analytics, a Multilayer Perceptron Regression algorithm was implemented with the metadata. The same environment was established in a Hadoop ecosystem like MapReduce, Hive, and Spark. Finally, this research suggests that a big data platform would be economically beneficial.
Document Details
- Document Type
- Technical Report
- Publication Date
- Jun 01, 2019
- Accession Number
- AD1080308
Entities
People
- Pyojeong Kim
Organizations
- Naval Postgraduate School