In reviewing reports of vulnerabilities in WordPress plugin to add to the data set for our service we have seen that the quality of those reports can be quite bad. From inaccurate data on vulnerabilities, including claims that a vulnerability has been fixed when it hasn’t, to what look to be intentionally fake reports of vulnerabilities, we seen it. The end result of us reviewing the reports is that we our able to provide our customers with a higher quality data, without the falsely labeling unfixed vulnerabilities as being fixed or including fake vulnerabilities, than the other data sources for plugin vulnerabilities out there, since they don’t actually review the vulnerabilities.
One of the important things we do when collecting vulnerability data in WordPress plugins for our customers is that we test out each vulnerability. That allows our customers to know what versions of the plugins are vulnerable, which can be rather important when dealing with a hacked websites, since it could be that an outdated version in use is not actually vulnerable since the vulnerability only existed in newer versions (vulnerabilities can exists in as little as a single version). It also allows us to identify when vulnerabilities have not actually been fixed, other similar service don’t do the same so you are left thinking you are protected against the vulnerability when you are still vulnerable. It also means that we don’t include false reports of vulnerabilities in our data set.