Detection And Classification Of Web Robots With Honeypots

Abstract

Web robots are automated programs that systematically browse the Web, collecting information. Although Web robots are valuable tools for indexing content on the Web, they can also be maliciousthrough phishing, spamming, or performing targeted attacks. In this thesis, we study an approach to Web-robot detection that uses honeypots in the form of hidden resources on Web pages. Our detection model isbased upon the observation that malicious Web robots do not consider a resources visibility whengathering information. We performed a test on an academic website and analyzed the honeypots performance using Web logs from the sites server. Our results did detect Web robots, but did not adequately detect the more sophisticated robots, such as those using deep-crawling algorithms with query generation.

Open PDF

Document Details

Document Type
Technical Report
Publication Date
Mar 01, 2016
Accession Number
AD1027491

Entities

People

  • Sean F Mckenna

Organizations

  • Naval Postgraduate School

Tags

Communities of Interest

  • Autonomy

DTIC Thesaurus Topics

  • Algorithms
  • Computer Programming
  • Computer Programs
  • Computer Science
  • Computers
  • Data Mining
  • Department Of Homeland Security
  • Detection
  • Electronic Mail
  • Internet
  • Network Computing
  • Network Protocols
  • Network Science
  • Operating Systems
  • Social Media
  • Web Browsers
  • Websites

Fields of Study

  • Computer science

Readers

  • Cybersecurity.
  • Geospatial Intelligence and Artificial Intelligence Analytics

Technology Areas

  • AI & ML
  • AI & ML - Autonomous Systems
  • AI & ML - Information Retrieval
  • Autonomy