ARMOUR: An Application-Level Threat Detection Framework

Conventional security mechanisms at network, host, and source code levels are no longer sufficient in detecting and responding to increasingly dynamic and sophisticated cyber threats today. Detecting malicious behavior at the application level can help better understand the intent of the threat and strengthen overall system security posture. To that end, we have developed an innovative, use case-driven framework called ARMOUR (which standards for Association Rules Mining Of Undesired behavioR) that involves mining software component interactions from system execution history and applying an adaptive detection algorithm to identify potential malicious behavior. The framework uses unsupervised learning; can perform fast, "inline" detection in near real time; and can quickly adapt to system load fluctuations and other concept drifts. Our evaluation of the approach against a real Emergency Deployment System has demonstrated very promising results.

As depicted in the diagram above, the ARMOUR framework consists of two major phases:

The first phase is mining the usage model. In this phase, inter-component messages are captured by sensors on the target system in near real-time, stored in the event log, and then preprocessed to produce itemsets based on pre-identified use case types. Afterwards, the Apriori algorithm for association rules mining is applied on the itemsets to generate the rule base. Finally, architecture-based heuristics are leveraged to effectively prune the rule base from extraneous rules and keep the rule base within a manageable size. The second phase is applying the model on the current system event stream to detect anomalous behavior. We developed an efficient detection algorithm for this purpose, which produces a quantitative anomaly measure for each event. When the anomaly likelihood exceeds a certain threshold, the event is marked as suspicious and its details are recorded. As mentioned earlier, the number of simultaneously running use cases may fluctuate at runtime and adversely impact the detection accuracy. We therefore take an extra step to monitor the degree of execution concurrency and adapt the detection threshold accordingly. Once detected, the anomalous events trigger alerts to the system's security administrator or an autonomous agent that would examine them as part of the ongoing risk assessment process and deploy countermeasures to the target system such as isolating the compromised components. This last step is beyond the scope of this paper and left as an item for future research.

The contributions of our architecture-based framework, dubbed Association Rules Mining Of Undesired behavioR (ARMOUR), consist of three key elements: use case-informed monitoring of system execution history at the component level, the novel use of Association Rules mining to build a normal-use behavior model of the target system, and an efficient and adaptive algorithm that applies the model to anomaly detection. Our approach is different from many existing approaches in the following aspects:

• Most related mining efforts look at low-level datasets such as system API calls and network traffic (e.g. tcpdump) as opposed to "macro-level" system traits, such as component interactions in a large-scale software system. The former usually require specialized skills to analyze and interpret, whereas anomalous patterns at the component level can be much stronger clues for a human administrator or an autonomous agent to understand the intent of the malicious attack and deploy appropriate countermeasures.
• Many threat detection efforts rely on domain-specific knowledge (such as network protocols or OS commands) and are therefore difficult to generalize. Our approach, however, leverages architecture level abstractions such as component models and high-level use cases to inform the data mining process, which makes our approach broadly applicable to many target systems.
• Our use of the associations mining technique sets this effort apart from most other threat detection research that uses outlier detection algorithms based on distance, density, or other statistical measures.

The source code for simulating the Emergency Deployment System (EDS), a real component-based software system, can be found here.

More detailed evaluation results are also available.

Publications

More details about ARMOUR can be found in our publication:

• Mining Software Component Interactions to Detect Security Threats at the Architectural Level
  Eric Yuan and Sam Malek
  13th Working IEEE/IFIP Conference on Software Architecture (WICSA 2016), Venice, Italy, April 2016.
  [PDF]