25 Feb Attackers Ran Ahead of Defenders on Log4j
Defenders can turn the tables
– Boston, Mass. – Feb. 25, 2022
The announcement of the “Log4Shell” vulnerability in Log4j led to waves of concern, exploitation, and urgent remediation last December. As we start to catch our breath and address this widespread vulnerability, there are subtle lessons worth taking away from this episode. Most notably, the timelines around Log4Shell reveal both fundamental differences in the speed of innovation today between attackers and defenders, as well as how we can turn the tables on adversaries and accelerate past them in our ingenuity and responsiveness.
Creativity at Hacker Speeds
Expanding upon the chronology of Log4shell by Pete Freitag, we can see how just how fast exploitation of this vulnerability took compared to the slower defensive response:
- Jul.12, 2014: Log4j version 2.0 is released for General Availability
- No 24, 2021 (7+ years later): Chen Zhaojun of Alibaba Cloud Security raises the Log4Shell vulnerability to the attention of the Apache Software Foundation
- Dec.1, 2021 (Discovery +7 days; Public disclosure -9 days): First detected exploitation of Log4j, found by Cloudflare
- Dec. 6, 2021 (Discovery +12 days; Public disclosure -4 days): First attempted patch for Log4j released as version 15.0; unfortunately, this patch does not fully protect against Log4Shell
- Dec. 9, 2021 (Discovery +15 days; Public disclosure -1 day): Vulnerability details shared on Twitter, first in China, then expanding globally. Cloud Security Alliance has a great exploration of the discussion’s spread on Twitter and, interestingly, among Minecraft players.
- Dec. 10, 2021 (Discovery +16 days): MITRE publicly lists Log4Shell as CVE-2021-44228. The story is picked up by some press and, importantly, organizations start patching. (Public disclosure +12 hours): 40,000 exploitation attempts, as detected and reported by CheckPoint
- Dec. 11, 2021 (Discover +17 days; Public disclosure +1 day): 120,000 exploitation attempts, as detected and reported by CheckPoint
- De 12, 2021 (Discover +18 days; Public disclosure +2 day): 400,000 exploitation attempts, as detected and reported by CheckPoint
- Dec. 13, 2021 (Discovery +19 days; Public disclosure +3 days): Second patch attempt for Log4j released as version 16.0. Offers more protection than 2.15.0, but (as will be discovered) is still vulnerable itself. 830,000 exploitation attempts, as detected and reported by CheckPoint
- Dec. 14, 2021 (Discovery +20 days; Public disclosure +4 days): MITRE publishes CVE-2021-45046, revealing how Log4j 2.15.0 was still vulnerable to exploitation; they would later upgrade the severity from “Moderate” to “Critical”
- De 17, 2021 (Discovery +23 days; Public disclosure +7 days): Third patch attempt for Log4j released as version 2.17.0. Again, while improving on prior patches, this version would turn out to still be vulnerable to remote code execution (RCE).
- Dec. 18, 2021 (Discovery +24 days; Public disclosure +8 days): MITRE publishes CVE-2021-45105, revealing how Log4j versions 2 through 2.16.0 were vulnerable to Denial of Service (DOS) attacks.
- Dec. 28, 2021 (Discovery +34 days; Public disclosure +18 days): Fourth and (at the time of this article) current patch for Log4j released as version 17.1. MITRE publishes CVE-2021-44832, revealing how Log4j versions 2 through 2.17.0 were vulnerable to RCE
- Full patching of Log4j up to version 2.17.1 or later… time unknown.
Cybercrime Radio: Talking Cyber Ranges
Self-service threat scenarios
The timeline of events paints a stark contrast between offensive and defensive actions around the Log4Shell vulnerability. Defensively, three measures of time stand out: 7+ years to find the vulnerability; nearly two weeks until the first patch; and over one month until a stable patch.
As for the attackers, initial exploits occurred before public announcement. This implies either a parallel discovery or a leak. And again, it took less than 12 hours until a significant exploit was in the wild; and only a day until that exploit had permeated, or attempted to penetrate, hundreds of thousands of systems.
CheckPoint’s reporting on the pace of exploitation also notes the diversity of these attacks, with over 60 variants introduced in the first days. It will likely take months, or years before we can fully remediate the hundreds of millions of devices running Log4j. Even so, this is an accelerated rate compared to typical patching cadences by the publicity Log4Shell has received.
It Doesn’t Have to Be This Way!
You might be thinking, “This isn’t surprising.” Attackers are fast, defenders are slower (although kudos to the broader security community for the urgency shown in responding to Log4Shell).
The WannaCry attack of May 2017 was mitigated through the discovery and activation of a global “kill switch” by cybersecurity researchers Marcus Hutchins and Darien Huss, likely protecting innumerable likely victims. Again, in June of 2017, the NotPetya attacks were blunted when Amit Serper, another cybersecurity researcher, found a “vaccine” capable of protecting individual machines.
There was similar defensive creativity around the Log4Shell vulnerability.
We at SimSpace released a webinar and added to our platform hands-on training modules on Log4Shell for both blue and red team operators within days of the announcement. Endpoint security firm Cybereason released a “vaccine” (initially on the evening of the day of disclosure, with updates on December 15th and 17th) in the form of a script that could help rapidly mitigate the risk of exploitation in advance of proper patching. Additionally, Breach and Attack Simulation vendors like Cymulate introduced attack simulations to reveal exposure to Log4Shell in the environment.
Building Increased Resilience on the Range
Cyber Risk Management Platforms like SimSpace offer several important tools to prepare for and respond to incidents like Log4Shell/Log4j, decreasing the time to discover and address such risks when minutes matter.
With high-fidelity cyber ranges that clone your environment, you can find vulnerabilities in your environment without disrupting production. Inside of a safe, isolated range, you can trial potential discovery or mitigation tools, like Cybereason’s vaccine and Cymulate’s attack simulation, to ensure there are no unintended risks before you push to production.
Log4Shell showed a perfect example of why it is important to confirm the safety and efficacy of patches before adopting them. Vendors can use their range to pressure test patches before release, ensuring they work as intended, and don’t introduce new vulnerabilities (like, a DOS vuln). Operators can verify that a vendor patch works for their environment’s unique needs, building range testing into their deployment cycle for greater speed and confidence in their deployment. Moreover, you can discover how your defense-in-depth tool stack will work in tandem with patches to offer protection and tune your stack to increase protection in advance.
Lastly, having a persistent cyber range available allows teams to accurately validate if they acted quickly or proactively enough to stop the potential damage. Using the range to run simulations of breaches and potential impacts helps to quantify and report the value of your remediation efforts to executives, boards, regulators and even cyber insurers.
Log4Shell was a wake-up call for the entire cybersecurity community. With greater adoption of open-source software, often supported by small, independent contributors, our code security is only as strong as the worst maintained common library or plugin.
We don’t have to resign ourselves to second place in our race against attackers. We can seize the initiative through collaboration across the cybersecurity community, innovative responses to active threats, and proactively building a more mature cyber risk practice empowered by Cyber Risk Management Platforms like SimSpace.
About the Author
David Berliner is the Director of Security Strategy for SimSpace where he explores cybersecurity market trends, thought leadership, company positioning and competitive analysis. Prior to SimSpace, he served as Director of Product and conducted research on cybersecurity trends at Cybereason. David holds a Bachelor of Arts from Brown University and an MBA from the Kellogg School of Management at Northwestern.
About SimSpace
Reduce your cyber risk. Improve your security posture. Increase your cyber readiness.
The SimSpace Platform is powered by our cyber range, a safe, open, and secure hyper-realistic environment delivering advanced risk mitigation solutions.
Optimize your cybersecurity using real-world attack simulations, product evaluations, specialized training content, and individual/team-based assessments.
