One method we have to measure the protection that WordPress firewall plugins offer is part of the regression testing software for our own firewall plugin. That software allows us to make sure the default protection against zero-days, which are vulnerabilities being exploited before the developer or others know about them, that our plugin offers isn’t broken as we make changes to the plugin. Once we started developing that, we realized that could be repurposed to test to see if other firewall plugins provide protection in the same situations. In May of last year, we started doing a monthly run of that against other firewall plugins, so we could get a better understanding of how the WordPress security landscape is changing over time.

This month we added a new plugin to our test set. The name of the plugin is Anti-Hacker. It’s been available on the WordPress Plugin Directory since June, but we only ran across it now. Not much of anyone else seems to have run across it either, as it only has 10+ installs. The marketing makes plenty of impressive claims, but provides no evidence to back them up. The developer claims it provides protection against “XSS, SQL Injection, PHP Injection, CMD Injection and Transversal Directory” vulnerabilities. The problem we found when we went to add it to our testing system is that it isn’t possible to enable that protection, as the settings checkbox for it is disabled:

We manually changed the plugin’s settings in the database to enable the protection and the result was pretty good. It provided protection against 16.38% of the tests. That doesn’t sound like much, but compare it to the top 10 tested plugins this month with protection that can actually be used in normal operation:

1. Plugin Vulnerabilities Firewall – 100.0%

2. NinjaFirewall – 39.0%

3. Wordfence Security – 23.2%

4. Pareto Security – 19.8%

5. All-In-One Security (AIOS) – 13.6%

6. Web Application Firewall – 9.6%

7. Hide My WP – 6.2%

8. Hide My WP Ghost – 8.5%

9. Bulletproof Security – 7.9%

10. Anti-Malware Security and Brute-Force Firewall – 4.0%

It provided more protection than All-In-One Security (AIOS), which has 1+ million installs. That result is more striking when you consider that we configure All-In-One Security (AIOS) against the recommendations of the developer, which leads to it providing much more security than it otherwise would.

The plugin that came in 4th, Pareto Security, only has 400+ installs. The results of this testing continue to show a lack of correlation between the security provided by a plugin and their popularity. Likely caused by a lack of knowledge as to what protection plugins offer, which is yet another reminder of the failure of news outlets to provide useful coverage on WordPress security.