© 2020 Trend Micro Inc.2
APT29
In this testing, MITRE took on the persona of APT29, a threat group that has been attributed to the Russian government and has operated since at least 2008.
Other aliases: Cozy Bear, The Dukes, YTTRIUM
© 2020 Trend Micro Inc.3
APT29
They attacked participant environments using two scenarios. These scenarios were publicly reported to match the tradecraft and operational flows followed by the group APT29.
© 2020 Trend Micro Inc.4
How this evaluation differs
MITRE Evaluation tests how solutions detect an adversary performing a targeted attack.It does not test a product’s ability to block/prevent malware.
Adversary-centric vs. Malware-centric
© 2020 Trend Micro Inc.5
APT29Participants included:
FireEye, Bitdefender, Cybereason, Cycraft, Elastic, F-Secure, HanSight, Malwarebytes, ReaQta, and Secureworks
© 2020 Trend Micro Inc.6
APT29
All detections (not prevention) were recorded across every step of the attack.
Results
Detection enriched usingTrend Micro’s Managed XDR team
DetectedAutomatically
Not Detected
© 2020 Trend Micro Inc.7
APT29
However the challenge is, no clear scoring is given.
We show the detections we observed without providing a ‘winner.’ There are no scores, rankings, or ratings. Instead, we show how each vendor approaches threat defense within the context of ATT&CK.
""
© 2020 Trend Micro Inc.9
#1 in Initial Overall Detection
LeaderDetection Rate
With Initial Configuration(Out of a 78% average)
#1 #2 #3
Wait… What about…
#786%
#1476%
#884%
91% 90% 89.5%
© 2020 Trend Micro Inc.10
Detections without configuration changes
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
© 2020 Trend Micro Inc.11
Detections after configuration changes
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
© 2020 Trend Micro Inc.12
• Lowest number of missed detections among all vendors – initial configuration
Key Highlights – Fewest Missed Detections
0
20
40
60
80
100
120
140
Tren
dMic
roSe
ntin
elO
neM
icro
soft
Palo
Alto
Net
wor
ksF-
Secu
reFi
reEy
eEl
astic
Crow
dStr
ike
Sym
ante
cSe
cure
wor
ksHa
nSig
htCy
craf
tCy
lanc
eVm
war
eCy
bere
ason
Kasp
ersk
yGo
Secu
reBi
tDef
ende
rRe
aQta
McA
fee
Mal
war
ebyt
es
© 2020 Trend Micro Inc.13
Key Highlights – Strong Technique Detections• Detected very well on individual attack techniques, which are higher
confidence detections.
0
5
10
15
20
25
30
35
40
45
0
41TREND MICRO
34
Range of Vendor Detections on Technique – Initial Configuration
CrowdStrike (26)
Microsoft (13)
© 2020 Trend Micro Inc.14
• Managed alert volumes to avoid alert fatigue.
• Lower level of alerts combined with high detection rate means we reduced the noise of all detections into a minimal amount of meaningful/actionable alerts.
Key Highlights – Actionable Alerts
Detection Rate AlertsTrendMicro 91.04% 24SentinelOne 90.30% 51Microsoft 89.55% 33PaloAltoNetworks 89.55% 50F-Secure 88.06% 38FireEye 88.06% 54Elastic 87.31% 46CrowdStrike 85.82% 22Symantec 84.33% 21Secureworks 82.09% 34…Cycraft 80.60% 90
Top 10 Vendors for detection rate vs. alerts – Initial Configuration
© 2020 Trend Micro Inc.15
0
20
40
60
80
100
120
1
113107 - TREND MICRO
• Telemetry = Visibility. We give security analysts access to the type and depth of visibility they need when looking into detailed attacker activity
Key Highlights – Strong Telemetry
Range of Vendors’ Collected Telemetry
© 2020 Trend Micro Inc.16
• Our detection coverage results would have remained strong without MDR service, though the service was able to add more valuable context.
Key Highlights – MDR Enriched Detections
Only 6 detections were exclusive to
MSSP (MDR)
© 2020 Trend Micro Inc.17
References• MITRE Evaluation results (APT29):
– https://attackevals.mitre.org/evaluations.html?round=APT29
• Blog posts:– https://blog.trendmicro.com/mitre-evaluation2020/– https://blog.trendmicro.com/top-ten-mitre/
• Forrester assessment:
– https://github.com/joshzelonis/EnterpriseAPT29Eval