The Security Industry Isn’t Taking Responsibility for the Poor State of Industry That Led to CrowdStrike Outage
The massive outage caused by a faulty update by the equally massive security company Crowdstrike should be leading to a lot of soul searching in the security industry, but based on reporting from a major industry conference, it isn’t.
Crowdstrike has released what they call a preliminary Post Incident Review (PIR) on the incident. What stands out in that is how much of it is not focused on explaining what went wrong and how it will be fixed, but instead on things that are unrelated to the incident. For example, they have three paragraphs related to something that isn’t relevant to the incident and then in the next paragraph they admit that, “[t]he event of Friday, July 19, 2024 was not triggered by” that.
That response would probably be criticized by PR people for not focusing on taking responsibility and getting it right from now, but it is in line with the security industry belief that they should be allowed to do a bad job and not face any consequences for it. One commenter on Slashdot put it well, “The fact that supposed security experts would think so lightly of a piece of security software bringing down the systems it was supposed to protect shows how abyssal the state of the industry is. Availability was supposed to be one of the aim of security, and that seemed to be lost somewhere in all the buzzwords.”
A Fortune story by Sharon Goldman about the response to Crowdstrike at the security conference, Black Hat, had some troubling comments from attendees. The CEO of a “cyber risk management” company, CyberSaint, was quoted saying that a company impacted by this should take some responsibility:
“They have to take some responsibility for understanding their environment, understanding where their biggest risks are,” he said, pointing out that most organizations think most about being attacked as the biggest cyber risk, but flaws in software updates can also impact the entire business. “Throwing it all on Crowdstrike is not fair.”
On the one hand, this person seems to be using a big screw-up by the security industry as a marketing opportunity (as their company help manage “cyber risk”). On the other hand, the flaw in the software update was the fault of Crowdstrike. Blaming others for that part of that is perverse.
Someone else was blaming Microsoft:
“Microsoft should not be giving any third party that level of access,” said Eric O’Neill, a cybersecurity expert, attorney and former FBI operative. “Microsoft will complain, well, it’s just the way that the technology works, or licensing works, but that’s bullshit, because this same problem didn’t affect Linux or Mac. And Crowdstrike caught it super-early.”
That defense is stunning. While that “cybersecurity expert” claimed the problem didn’t affect Linux, Crowdstrike’s software actually caused problems against multiple Linux distros in the recent past. That makes the situation worse, as Crowdstrike clearly had prior knowledge of the potential risk caused by what they were implementing. They didn’t learn from that, considering the subsequent incident involving Windows.
Crowdstrike woo’d conference goers not with suggesting that they were going to lead the security industry to better practices, but with swag:
Back at the Crowdstrike booth, staffers busily operated machines to create custom-pressed shirts at the “T-Shirt bar,” while others handed out small boxes containing the coveted figurines. The figurines, dubbed “Aquatic Panda” and “Scattered Spider,” represent famous hacker groups and cyber criminals.
One security researcher in line said he didn’t know what the collectibles were, but hard heard they were a hot item. Then again, the researched added, as if to avoid setting his expectations too high, “they probably aren’t anything fancy.” After all, he said, “the company lost like 40% of its stock.”
As noted in that quote, Crowdstrike’s stock price dropped in the aftermath of the incident, but it is still higher than it was a year ago and they still have a market cap of over 60 billion dollars.
Crowdstrike’s large financial size tells you they could have afforded to handle things better here before the incident, but they didn’t. They are not alone in that. Despite having vast resources, the security industry isn’t delivering the promises they make or even avoiding preventable mistakes like this.
WordPress Isn’t Immune
The WordPress security space is a microcosm of what is going on in the larger industry. Where major players don’t use their resources to get things right, but instead ignore or outright lie their way through the problems they are causing. Take the example of a major provider, Wordfence, whose CEO claimed that they provide “impeccable data” on plugin vulnerabilities when confronted with the reality that its data isn’t reliable. Despite the impeccable data, earlier this year they caused panic by falsely claiming that the then current version of WooCommerce was vulnerable, because they didn’t do basic vetting. (They also are failing to warn about a real vulnerability that hasn’t been fixed in WooCommerce.)
Recently, we found that multiple popular security plugins contained a vulnerability because the authors haven’t properly vetted a third-party library they include in their plugins. That library is used in over a thousand other plugins, so those security providers’ lack of due diligence has had a much wider impact. In another recent instance, we found that a plugin from a security provider contains a vulnerability while reviewing their claim that they did a security review of it. They claimed the review affirmed that the plugin complied “with stringent security standards and protocols,” despite the vulnerability being caused by a lack of basic security.
The problem in the WordPress space is exacerbated by that those not involved in security are frequently critical of those in the WordPress security industry and other who are trying to shine a light on the problem, instead of those causing the problem.
The end result is that security remains bad and the bad actors benefit from more business to solve a problem they are not going to solve (or even make worse). The cycle will continue unless those outside the industry stop accepting the security industry acting this way.