CrowdStrike and Meta Launch CyberSOCEval for Evaluating AI Systems in Real-World Security Operations

CrowdStrike and Meta have introduced CyberSOCEval, a new suite of benchmarks for evaluating AI systems in real-world security operations. Built on Meta's CyberSecEval framework and CrowdStrike's frontline threat intelligence, CyberSOCEval evaluates large language models across critical security workflows. The benchmark suite is open source and intended to help the security and AI community unlock the potential of AI in protecting against advanced attacks.

CrowdStrike (NASDAQ: CRWD) and Meta have launched CyberSOCEval, a suite of open-source benchmarks designed to evaluate the performance of AI systems in real-world security operations. Built on Meta's CyberSecEval framework and CrowdStrike's threat intelligence expertise, CyberSOCEval aims to establish standards for testing and selecting large language models (LLMs) for use in security operations centers (SOCs) CrowdStrike and Meta launch benchmarks to test AI in cybersecurity^[1].

The benchmarks, introduced at the Fal.Con 2025 event in Las Vegas, address the growing need for clear standards in evaluating AI systems in cybersecurity. As organizations increasingly turn to AI technologies to manage the volume of security alerts and evolving threats, the lack of established benchmarks has posed a challenge. CyberSOCEval evaluates LLMs across critical security workflows such as incident response, malware analysis, and threat analysis comprehension. By testing AI systems against real-world adversary techniques and expert-designed security reasoning scenarios, the benchmarks help organizations validate performance under pressure and prove operational readiness CrowdStrike And Meta Deliver New Benchmarks For The ...^[2].

Vincent Gonguet, Director of Product, GenAI at Superintelligence Labs at Meta, stated that the collaboration introduces "a new open source benchmark suite to evaluate the capabilities of LLMs in real world security scenarios." Daniel Bernard, chief business officer at CrowdStrike, emphasized the significance of the partnership, stating that "when two leaders like CrowdStrike and Meta come together, it’s larger than collaboration, it’s about setting the direction of cybersecurity for the AI era" CrowdStrike and Meta launch benchmarks to test AI in cybersecurity^[1].

The introduction of CyberSOCEval reflects CrowdStrike's strong financial position and commitment to innovation. With a market capitalization of $109.4 billion and a current ratio of 1.88, the company demonstrates its financial strength to lead such initiatives. Recent developments, such as exceeding fiscal guidance and raising its fiscal year 2026 guidance, further highlight CrowdStrike's robust performance CrowdStrike and Meta launch benchmarks to test AI in cybersecurity^[1].

The open-source nature of CyberSOCEval encourages collaboration and improvement within the AI and security community. By making the benchmarks available through Meta's CyberSecEval framework, organizations can leverage these tools to evaluate model capabilities and drive the adoption of effective AI solutions in cybersecurity.