On November 9, 2022, MITRE Engenuity™ released the results from their first-ever ATT&CK® Evaluation for Security Services Providers. The evaluations highlighted results across 15 security services providers, assessing their capabilities in detecting, analyzing, and describing adversary behavior.

Sophos Managed Detection and Response (MDR) successfully reported malicious activity across all 10 MITRE ATT&CK® steps, excelling in its ability to detect and respond to sophisticated threat actors with speed and precision.

Why MITRE Engenuity introduced an ATT&CK Evaluation for managed services

For nearly 5 years, MITRE Engenuity has conducted independent evaluations of cybersecurity products using an open methodology based on the ATT&CK knowledge base. These evaluations are predicated on real-world attack emulations that simulate the tactics, techniques, and procedures (TTPs) of relevant advanced persistent threats (APTs) and task vendor participants with demonstrating their ability to detect, analyze, and describe those activities.

The primary objective of ATT&CK Evaluations is to help cybersecurity solution providers—and the organizations they support—make better decisions to combat cyberthreats and improve threat detection capabilities. However, MITRE survey results have shown that it’s challenges related to people (training and hiring), not cybersecurity products and technology, that are the main limitation preventing organizations from advancing their security operations programs.

In its survey of more than 400,000 information security professionals worldwide, MITRE Engenuity found that 58% of organizations rely on managed services to either complement their in-house security operations center (SOC), or serve as their main line of defense. This number was even higher (68%) when considering companies under 5,000 employees. At the same time, roughly half of these organizations reported a lack of confidence in their managed service’s people or technology.

In response to the rapid adoption of managed services and associated cybersecurity challenges, MITRE Engenuity developed and administered a new evaluation methodology that allows end users to better understand how security services like Sophos MDR address adversary behavior.

What is OilRig?

The MITRE Engenuity ATT&CK Evaluation for Security Service Providers evaluated Sophos MDR and other vendors’ abilities to detect and analyze attack tactics and techniques simulating those used by OilRig, an Iranian government-affiliated threat actor – also known as APT34 and Helix Kitten.

OilRig has conducted operations relying on social engineering, stolen credentials, and supply chain attacks, resulting in the theft of sensitive data from critical infrastructure, financial services, government, military, and telecommunications.

This threat actor was selected for use in the MITRE ATT&CK Evaluation for Security Service Providers based on its evasion and persistence techniques, its complexity, and its relevancy to industry.

Unlike MITRE Engenuity’s ATT&CK Evaluations for Enterprise, which follow an open book methodology where participating vendors know in advance the adversary being emulated, the MITRE Engenuity’s Security Services evaluation did not disclose the adversary group or the technique scope.

How did Sophos MDR perform in the MITRE Engenuity ATT&CK Evaluation for Security Service Providers?

Sophos Managed Detection and Response (MDR) successfully reported malicious activity across all 10 MITRE ATT&CK steps, excelling in its ability to detect and respond to sophisticated threat actors with speed and precision. This was a detection-only evaluation, meaning that MITRE Engenuity did not evaluate vendors’ ability to execute threat response actions.

It is important to note that ATT&CK Evaluations are not competitive analyses and do not designate a “winner.” And while there is no singular way for analyzing, ranking, or rating the participating vendors, Sophos MDR recorded an exceptional performance with results that validate our position as one of the top performing security services vendors in the market.

For more details about the evaluations and their results, visit https://attackevals.mitre-engenuity.org/managed-services/managed-services.

Source: Sophos