While looking to see if anyone had disclosed a vulnerability in a WordPress plugin we were looking into, we clicked on a Google search result for a competing data source for WordPress plugin vulnerabilities, the WPScan Vulnerability Database. Why Google returned this page as a result is unclear since the page is basically empty:

The one piece of information on that page is something labeled as coderisk with a number value:

That turned out to be a score from RIPS CodeRisk, which we noted eleven months ago doesn’t seem all that reliable. Considering the amount of inaccurate information coming directly from the WPScan Vulnerability Database it probably shouldn’t be surprising that they would promote that despite very good reason not to.

We were curious to give RIPS CodeRisk another look, to see if the situation with its scores had improved.

To recap, those scores are promoted with this:

RIPS CodeRisk rates the security risk of WordPress plugins by analyzing them with the RIPS static code analyzer and combining the results into an easy to understand value.

And this:

RIPS CodeRisk helps users to determine the security state of their blog

To us there seem to be multiple problems with that. One being the black box process to come up with score, so there is no way to confirm their reliability. Another one being that static analysis is likely to provide a limited measure of the security of a plugin.

Something like that would appeal to the public, since it gives the appearance of an easy way to check on the security of plugins, while being more likely to provide an inaccurate picture of the security and causing problems for developers of plugins with inaccurate high scores.

To see what was going on with that now, we first looked at the plugin that started this all, HubSpot All-In-One Marketing – Forms, Popups, Live Chat. The plugin currently has a score of 60:

Considering that it has a vulnerability that hackers likely would exploit, that is not necessarily all that accurate a score. But what seems more concerning is the history of the score:

It dropped from the top score possible, 100, to 71 with the release of version 1.5.0. The changes made in that version don’t look to us to like they could have a major security impact, which gets to the problem with a black box score. The score also went down three points in the version that allowed the exploitable vulnerability to become fully realized.

In a coincidental situation, a plugin we had looked at the score of in our previous post has been something we have been looking at again. The plugin, Advanced Contact form 7 DB, currently has a score of 1:

That is while the plugin is currently removed from the WordPress Plugin Directory due to a security vulnerability of the type you might think that static analysis would catch. The plugin also contains other security issues as well. So the score there seems inaccurate.