13 Dec 2024

Complaints About “AI Slop” Vulnerability Reports Ignore That Security Spending is Going to The Wrong Places

Despite billions and billions being spent on security, security remains bad. That applies to software in general and with WordPress plugins. Maybe more money needs to be spent, but it is more likely that the money is being spent on the wrong things. The latter issue applies when it comes to reviewing of the security of software and the handling of reporting security issues. But it keeps being ignored. The Register published a story this week by Thomas Claburn about “AI slop” vulnerability reports that involved AI generated claims of vulnerabilities that are not true. What the story didn’t address is why those are occurring. Here is one example included in the story:

As if to underscore the persistence of these concerns, a Curl project bug report posted on December 8 shows that nearly a year after maintainer Daniel Stenberg raised the issue, he’s still confronted by “AI slop” – and wasting his time arguing with a bug submitter who may be partially or entirely automated.

If you follow the link there you find that the report was filed through HackerOne. There are a variety of issues with handling vulnerability reporting through HackerOne (some of which are a problem with reporting issues to WordPress). Here the incentive for junk reports is pretty obvious, there are payouts made for finding vulnerabilities:

There wouldn’t be that same incentive if you hire professionals to do security reviews. But those almost never happen. HackerOne wouldn’t make money off that, but they make money even if there system is filled with junk reports.

Another example linked to from the story didn’t mention where the reporting was going through:

For example, urllib3 recently received a report because a tool was detecting our usage of SSLv2 as insecure even though our usage is to explicitly disable SSLv2.

It turned out it was also through it was also through third-party security provider, Tidelift:

To report a security vulnerability, please use the Tidelift security contact. Tidelift will coordinate the fix and disclosure with maintainers.

Security reports shouldn’t be routed through a third-party. Filtering out false reports seems like a decent argument against that position. But Tidelift isn’t managing to do that and for some reason that was being ignored.

Tidelift isn’t payed by software developers, but by others looking for better security. Tidelift is presumably charging a lot of money for their service, as their pricing page requires you to contact them to get a quote. It isn’t clear what you are actually getting in exchange for paying them.

A Better Approach

A better approach here would be to spend money on getting security reviews of software done. With either developers paying for that, if they are a for-profit entity, or users otherwise. And paying people actually involved in the software to manage to handle processing security reports.

Offering bug bounties is probably a bad idea. You might get some legitimate reports, but you are mostly asking for people to find things that pay out a lot but don’t really matter from a security perspective.

Leave a Reply

Your email address will not be published.