We have now done three tests to see what protection WordPress security plugins provide against a real threat to WordPress websites, vulnerabilities in other plugins. We can find little in the way of similar testing having been done in the past, which should be troubling when you consider how often people are recommending these plugins. But it is in line what we find on wider basis when it comes to security products and services, a lack of evidence back claims about them, many of those claims being rather extraordinary to anyone who has a good understanding of security and therefore really need strong evidence to back them.
The results of the test haven’t been good for those claiming that these plugins will protect your website. In the first test, of a persistent cross-site scripting (XSS) vulnerability, only 2 of the 11 plugins tested any protection and that protection was easily bypassed. In the second test, of arbitrary file upload vulnerability, only 3 of the 12 test plugins provide any protection and the protection 2 of the 3 provided was easily bypassed. In the third test, of an information disclosure vulnerability, none of the 12 plugins provided any protection.
The one plugin that we could not easily bypass in the second test was NinjaFirewall (WP Edition). To briefly recap that test we used a vulnerability that we discovered in a plugin that would allow uploading arbitrary files to try to upload a file with a .php extension to the test website. With the other two plugins that provided some protection we found that making a small change, uploading with a different extension, .jpg, allowed us to get past their protection. Because you specify what you want the file named on the server in the request that exploits the vulnerability, changing the extension of the file being uploading doesn’t impact the final result. That change had no impact for NinjaFirewall (WP Edition).
To get better understanding of what was caused that plugin to provide the protection, we have done some further testing. Our first thought was that the plugin might be blocking any requests from those not logged in to WordPress that include a file in POST data. That turned out to be true. But when we tried sending a request with file in the POST data when logged in as a Subscriber lever user (the lowest level user), we found that was also blocked. We moved up the user levels and found that was true for any user level below the top level, Administrator. So even an Editor level user could not successfully upload new media through WordPress’ built-in functionality.
We then looked to see what setting was causing that. In the plugin’s Firewall Policies page we found that there is File Uploads setting that is set by default to “Disallow uploads”. After switching that to “Allow uploads” lower level users could upload files, but the protection against the arbitrary file upload vulnerability was also gone (you didn’t even need to change the file extension of the file being uploaded). If you are on a website that only has an Administrator account this tradeoff is usually going to be big deal, unless you need those that are logged in to be able to upload files. But for other websites, with this plugin you either can allow users to upload files as they normally would be able to or you have protection against this vulnerability.