One of the ways we measure how much protection that WordPress security plugins provide against the real threat of vulnerabilities in other WordPress plugins, is to run software we have designed to make sure that our own firewall plugin’s protection isn’t broken when we make changes, against other plugins. Since May we have been doing a monthly run of that and logging the results, so that we can monitor changes in the results of the other plugins.

Until this month, there have been only two changes. One was that the amount of protection changed for plugins when we added tests for more exploit attempt variants, with most plugins not providing protection against the new tests. The other was that we detected that Shield Security’s protection became entirely broken. That first occurred in the June test and hasn’t been fixed yet.

This month’s run saw improvement in one plugin, All In One WP Security & Firewall. It protected against 16.77% of tested exploit attempts up from 14.84% the previous month. That moved it up a slot in terms of which plugins provide the best protection, passing Pareto Security. It still is far below the second best plugin in the testing, NinjaFirewall, which provided protection for more than two times as many attempts (our own plugin provided protection against them all). That result comes with a big asterisk, which we will come to shortly.

So what changed to increase the protection and could it lead to further improvements going forward?

We found the improvement was caused by a change made in version 5 of the plugin, which was released last month, but after our test was run that month. A bit more testing showed that it wasn’t the PHP-based firewall the developer introduced in that version and claimed gives greater protection:

Our PHP-based firewall has been created to give you even greater protection.

Instead, it was the addition for “other settings” for the 6G Firewall, which was added in that version:

With all of those settings enabled, the increased protection occurred.

So there shouldn’t be any further security improvements as the developer of this plugin isn’t the developer of the 6G Firewall, they just incorporate that into their plugin.

The last time we looked at All In One WP Security & Firewall, in July, we found that the protection offered by the plugin with its recommended setting was far below the result as configured in our testing. That is because they recommend enabling the 6G Firewall and not the 5G Firewall:

The 6G Blacklist is updated and improved version of 5G Blacklist. If you have 5G Blacklist active, you might consider activating 6G Blacklist instead.

In our testing, we enable both to provide the most protection possible from the plugin. With the latest version of the plugin, if only the 6G Firewall is used, the protection falls down nearly 6 percentage points, from 16.77% to only 10.97%. With that level of protection, three more plugins currently provide better protection than it does.