SaTC: CORE: Small: Scalable and Meaningful Threat Intelligence Generation

New York University, New York NY

Investigators

Abstract

Threat intelligence is used by organizations to protect systems and end-users by detecting and blocking communications with known attackers' systems. Quality threat intelligence can also provide methods of detecting attackers' tools, which are often less ephemeral than their attack infrastructure. Unfortunately, producing quality threat intelligence is often a highly manual and inefficient process. This has resulted in limited amounts of useful threat intelligence which is available only to those companies that can afford it. This research develops new data-analytics methods to identify an attacker's infrastructure and attack tools. Our methods leverage the ability to efficiently collect large amounts of raw attacker data, process it, and build artificial intelligence techniques to discover attack patterns. This project improves the efficiency of generating high quality threat intelligence data, and makes it more affordable to a large range of companies.   Achieving this goal of improving the efficiency of generating useful threat intelligence requires progress on several key challenges. The project (i) investigates supervised machine learning based methods for efficiently collected large-scale amounts of data from attackers, (ii) improves methods for storing this data and other freely available raw threat intelligence data such that it can be easily joined, (iii) identifies robust features that can be extracted from this raw data which can be used for training supervised machine learning detection techniques, and (iv) enables high performance and efficient generation of large-scale useful threat intelligence data. Consequently, this research has the potential to transform the way in which threat intelligence data is produced and improve the security of organizations by making threat intelligence more accessible. This work also creates many educational opportunities for undergraduate and graduate students to gain experience using data-analytics techniques to efficiently detect emerging threats and improve the security of organizations.