Real-Time Information Extraction from Big Data

Abstract

We are drowning under the 3 Vs (volume, velocity and variety) of big data. Real-time information extraction from big data,at global as well as local levels, is critical for making rapid decisions in many important DoD scenarios. However, such real-time information from big data is enormously complex and extremely challenging. We argue that data movement is ofcrucial importance in big data analytics, and show that recently developed photonic communication technologies has the potential to extract real-time information, and improve performance/Watt by a hundredfold with corresponding cost savings at the local, rack-size level, provided the entire system, consisting of communications hardware and network, computer hardware, algorithms, software and architecture, is holistically optimized. We provide examples of some applications of interest to DoD for which such holistic system optimization could be a game changer.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Oct 01, 2015
Accession Number
AD1000187

Entities

People

  • Francisco L. Loaiza-lemos
  • Jagdeep Shah
  • Robert M. Rolfe

Organizations

  • Institute for Defense Analyses

Tags

Communities of Interest

  • Advanced Electronics
  • C4I
  • Cyber
  • Energy and Power Technologies
  • Human Systems

DTIC Thesaurus Topics

  • Algorithms
  • Big Data
  • Command And Control
  • Computer Networks
  • Computers
  • Computing System Architectures
  • Cyber Warfare
  • Cyberattacks
  • Data Analysis
  • Data Centers
  • Department Of Defense
  • Digital Communications
  • Internet Of Things
  • Network Architecture
  • Parallel Computing
  • Parallel Processing
  • Silicon Photonics

Fields of Study

  • Computer science

Readers

  • Distributed Systems and Data Platform Development

Technology Areas

  • AI & ML
  • AI & ML - DoD AI Strategy
  • AI & ML - Neural Networks