In October 2021, while writing an article about EDR/XDR solutions, I read an article from The Journal of CyberSecurity and Privacy entitled: “An Empirical Assessment of Endpoint Detection and Response Systems against Advanced Persistent Threats Attack Vectors”. I think now is a good time to revisit that research paper. The study tested state of the art EPP and EDR platforms against simulated APT attacks. They key contribution of this paper is that it reveals what type of TTPs are still able to circumvent top of the line EDR solutions. The products tested in the study are a who’s who of leading endpoint security vendors? Seeing a blog post from Recorded Future discussing the same paper reminded me about it, and wanted to contribute my take on it.
The full FINAL paper is available and the published version is available by searching Google for the article title “An Empirical Assessment of Endpoint Detection and Response Systems against Advanced Persistent Threats Attack Vectors”.
The ultimate take-away from this mid-2021 research paper is that EDR/XDR solutions are not 100% effective. There is still significant work to be done with respect to protecting the attack surface of a MicroSoft Windows 10 (MSW10) installation. These attack surfaces including legacy DLLs, processes hijacking, and many other capabilities such as CPL, and HTA which are not used in under the normal circumstances of an office desktop workstation computer. However, these rarely (if ever) capabilities increase the complexity of defending the attack surface, while creating a complex environment for malware to “live off the land binaries” (LOLBIN).
LOLBIN allows malware to avoid downloading its own tools to conduct an attack. Instead, the malware can use tools already built into the Windows operating system. This greatly increases the malware’s ability to remain stealthy, since it is not creating suspicious network traffic to remote servers.
The installers for Linux distributions often include the option for minimal installations where includes features and tools are greatly reduced. From a security perspective this is good because those tools represent attack surface you need to defends against. But, you don’t have to defend against the hijacking of a software application that isn’t installed. Instead you would have to monitor network traffic to ensure it is not being imported into the fight. At this point, shouldn’t MSW10 or future versions of MS Windows allow custom installation options to remove legacy DLL support and other features that only represent risk?
Let’s take a look at the research approach and findings.
The researchers sought to answer the following questions:
- Can state-of-the-art EDR detect common APT attack methods?
- Which are the blind spots of state-of-the-art EDRs?
- What information is reported by EDRs and which is their significance?
- How can one decrease the significance of reported events or even prevent the reporting?
Four real world attack scenarios were used that were modelled from the MITRE ATT&CK framework. The goal was to minimize the threat score reported by the various EDRs tested thereby either not triggering a security alert, or triggering one with the lowest score possible. 4 attacks were used:
- A .cpl file: A DLL file which can be executed by double-clicking under the context of the rundll32 LOLBINS which can execute code maliciously under its context. The file was crafted using CPLResourceRunner (https://github.com/rvrsh3ll/CPLResourceRunner accessed on 8 July 2021). To this end, the researchers used a shellcode storage technique using Memory-mapped files (MMF) and then trigger it using delegates.
- A legitimate Microsoft (MS) Teams installation that will load a malicious DLL. In this regard, DLL side-loading (https://attack.mitre.org/techniques/T1574/002/accessed on 8 July 2021) will lead to a self-injection, thus allowing attackers to “live” under a signed binary. To achieve this, we used the AQUARMOURY-Brownie (https://github.com/slaeryan/AQUARMOURY accessed on 8 July 2021).
- An unsigned PE executable file; from now on referred to as EXE, that will execute process injection using the “Early Bird” technique of AQUARMOURY into werfault.exe. For this, researchers spoofed the parent of explorer.exe using the PROC_THREAD_ATTRIBUTE_MITIGATION_POLICY flag to protect their malware from an unsigned by Microsoft DLL event that is commonly used by EDRs for processes monitoring.
- An HTA file. Once the user visits a harmless HTML page containing an IFrame, he will be redirected and prompted to run an HTML file infused with executable VBS code that will load the .NET code provided in Listing 2 and perform self-injection under the context of mshta.exe.
The researchers simulated APT attacks using an experimental network setup against 20 Endpoint Security products, and the results are shown in the table below:
Table 1. Aggregated results of the attacks for each EDR. Notation: : ✓ Successful attack,<> Successful attack, raised medium alert, •: Successful attack, raised minor alert, ⋆: Successful attack, alert was raised ◦:Unsuccessful attack, no alert raised, ✗: failed attack, alerts were raised. † In two experiments supplied by the vendor, in the first it was detected after five hours, in the second it was detected after 25 minutes. ⊙ Initial test was blocked due to file signature, second one was successful with another application.
EDR Product Test Results
|Product||Attack 1||Attack 2||Attack 3||Attack 4||Score|
|BitDefender GravityZone Plus||✗||✗||✓||✗||300|
|Carbon Black Response||•||✗||✓||✓||125|
|Check Point Harmony||✗||<>||✗||✓||250|
|F-Secure Elements EDR||<>||†||✓||✗||250|
|MicroSoft Defender for Endpoints||*||✗||✗||✓||275|
|Panda Adaptive Defense 360||✗||✓||*||✓||175|
|SentinelOne w/ tests features||✓||✓||✓||✗||100|
|SentinelOne w/o test features||✗||✗||✗||✗||400|
|Sophos Intercept X with EDR||✗||✗||✓||-||250|
|TrendMicro Apex One||•||•||✓||✓||50|
|ESET Protect Enterprise||✗||✗||✓||✓||200|
|F-Secure Elements Endpoint w/o EDR||✓||✓||✓||✓||0|
|Kaspersky Endpoint Security||✗||✗||✗||✓||300|
|McAfee Endpoint Protection||✗||✗||✓||✓||200|
|Symantec Endpoint Protection||✓||✗||✓||✓||100|