UNSUPERVISED ALGORITHMS TO DETECT ZERO-DAY.ATTACKS STRATEGY AND APPLICATION
Abstract : ? In the last decade, researchers, practitioners and companies struggled for devising mechanisms to detect cyber-security threats. Among others, those efforts originated rule-based, signature-based or supervised Machine Learning (ML) algorithms that were proven effective for detecting those intrusions that have already been encountered and characterized. Instead, new unknown threats, often referred to as zero-day attacks or zero-days, likely go undetected as they are often misclassified by those techniques. In recent years, unsupervised anomaly detection algorithms showed potential to detect zero-days However, dedicated support for quantitative analyses of unsupervised anomaly detection algorithms is still scarce and often does not promote meta-learning, which has potential to improve classification performance. To such extent, this paper introduces the problem of zero-days and reviews unsupervised algorithms for their detection.
? Then, the paper applies a question-answer approach to identify typical issues in conducting quantitative analyses for zero-days detection, and shows how to setup and exercise unsupervised algorithms with appropriate tooling. Using a very recent attack dataset
we debate on
? i) the impact of features on the detection performance of unsupervised algorithms,
? ii) the relevant metrics to evaluate intrusion detectors,
? iii) means to compare multiple unsupervised algorithms,
? iv) the application of meta-learning to reduce misclassifications. Ultimately,
? v) we measure detection performance of unsupervised anomaly detection algorithms with respect to zero-days.
Overall, the paper exemplifies how to practically orchestrate and apply an appropriate methodology, process and tool, providing even non-experts with means to select appropriate strategies to deal with zero-days.
INDEX TERMS Zero-day attacks, intrusion detection, machine learning, anomaly detection, RELOAD, security, unsupervised learning, cyber-attacks
? Different unsupervised anomaly detectors have been existing throughout years and grouped into families . We describe them with the support . First, depicts various normal data points and four anomalous data points (supposedly, corresponding to attacks).
? The successive to graphically describe the different families, reviewed below. Clustering algorithms partition a dataset by grouping data points in the same cluster if they share similar characteristics. Data points that cannot be assigned to any of the existing clusters, or that do not meet specific inclusion criteria, are anomalous.
? To such extent, this paper introduces the problem of zero-days and reviews unsupervised algorithms for their detection. Then, the paper applies a question-answer approach to identify typical issues in conducting quantitative analyses for zero-days detection, and shows how to setup and exercise unsupervised algorithms with appropriate tooling.
? presents anomaly detection algorithms for unsupervised intrusion detection. Describes best practices and introduces some research questions that drive quantitative analyses of unsupervised algorithms. Reports on tooling tailored to perform such analyses.
? Data points that cannot be assigned to any of the proposed clusters, or that do not meet specific inclusion criteria, are anomalous. An example of such behavior is shown in which identifies 3 separate clusters.
In the example, the clustering algorithm identifies two true positives (green tick marks) and two false negatives (red crosses). Similarly, density-based algorithms in estimate the density of a region: data points lying in dense regions of the input space are considered normal, while anomalies are expected in sparse areas
? when used as the sole or main instrument for intrusion detection. In particular, they are likely to generate a high amount of False Positives (the detector raises a security alert but no attacks are happening) and False Negatives (attacks going undetected), thus lowering correct classifications as True Positives or True Negatives.
? On the other hand, they have shown an discussed superiority in detecting zerodays, therefore a sensible strategy appears to create a synergy between supervised ML algorithms and unsupervised ones to build effective IDSs that deal with both known and zero-day attacks
|