Ploogins

As part of working on our Plugin Security Scorecard last year, we spent a fair amount of time using the search functionality of the WordPress Plugin Directory. Through that, we again and again ran across search results that prominently featured plugins with high install counts that were not relevant to the search results, while relevant plugins were sometimes buried later in the results. One of the examples were you can see that happening is on a search for "translation", which has as its fourth result, a 3+ million install backup plugin: Similarly, a search for "export" brings up an 8+ million install ecommerce plugin in the sixth spot: A search for "events" brings up a 5+ million install security plugin ahead of events related plugins: We could go on and on like that. This is a Known Issue After seeing that, a search of the Trac ticket system for the WordPress website found an open issue related to this. That ticket noted that "plugin search greatly favors the plugins with the most installations." There was a response from an employee of Automattic, who appears to control the weighting of the search results, that made this claim: The goal behind the ranking alg is giving end users the best plugin experience possible. The user is trying to solve a problem and plugins that have a larger install count are much more likely to provide good long term support. There are also a number of other factors used in the alg that help newer plugins improve their rankings (support thread resolution, ratings, etc). An irrelevant plugin with a high install count isn't going to better solve a problem than a relevant plugin with a smaller install count. Plugins can have high install counts, not because they are better, but because they are being automatically installed by other plugins or web hosts. Developers can and sometimes do radically change a popular plugin that gained a high install count to do something very different (sometimes with serious negative security implications). What Explains the Results? There is another explanation that comes to mind for the poor weighting of the results. It benefits someone. Who would it benefit? It would benefit the developer of a plugin that has packed a lot of unrelated functionality in to one plugin and it has millions of installs. As their plugin would rank highly across a wide range of searches even when it is less relevant than plugins that offer more targeted functionality. Who has that plugin? Automattic with its Jetpack plugin. The same Automattic that employs the person controlling the weighting. If that were true, you would assume that person wouldn't have openly admitted to weighting the search results to promote Jetpack. Amazingly, they did and linked to a post on an Automattic blog touching on that in the Trac ticket. The most relevant portion is this: I expressed this on the search relevancy ticket and I know that some of it is controversial. Opinions usually are. Let me try an example. (Disclaimer: I’ve worked on Jetpack Stats in the past.) Users search for “stats” 93 thousand times a year. (It’s the 13th most frequent search on the plugin directory.) Here are the top four search results with the old algorithm: The old search suggests that 90 thousand people per year go and install plugins that collectively, have only proven that they can “handle” 20 thousand users. Scaling stats as a service (as many plugins do) can be quite hard and expensive. Sending 90k new users per year to these plugins seems unrealistic. Even if the plugin doesn’t work as a hosted service, it still needs to scale answering support requests from end users. Let’s compare that to the top four suggestions from the new search algorithm: Among the issues there, an Automattic employee had access to what the top search results are. They don't appear to be made public. That person not only works for Automattic, but was involved in the feature of Jetpack he is admitting to shaping the algorithm to push it to a top result. We should note that it would possible to provide more relevant results while keeping Jetpack ranking high for stats, but it would not rank highly for other terms it does now. In another Trac ticket linked to in the quoted texted above, there are a lot more concerning things mentioned. In that, he states that there are only two real options for statistics: There are really only two options in the world that can work for a hundred thousand sites a year: Google Analytics and Jetpack. Notably, neither of those options are handled by plugins, but third-party services that a plugin can connect to. Relying on a third-party service can have a big downside, that many recent negative reviews of Jetpack are highlighting. Automattic started charging a lot of money for the previously free stats. It also seems incongruous with WordPress, which is an opensource project. Elsewhere, he says he is worried about people trying to game the search results: I certainly understand the concern, and that is a big part of the reason I am going into such depth on this ticket with my thinking. The search query is all out in the open for folks to see it (which was a big discussion because we are worried about people trying to game it). Which is exactly what he is doing. Another aspect is that the install count outweighing relevancy was noted in 2016: It seems to many weight is given on active installs other relevancy which cause some plugins to be in almost every search result, like MailPoet or TablePress. It wasn't fixed. Bizarrely later in the ticket, thread the Automattic employee claims everyone agrees what he has done is better than the previous result, despite there being numerous comments expressing the exact opposite opinion: So from my perspective the current search is not at all answering the question of "what plugin would serve me best for tracking stats on my website?". Everyone seems to agree that the new search is significantly better. Personally, I still think it is pretty bad and has lots of room for improvements, but its a good start. Lack of Conflict of Interest Policy A fairly obvious question this should raise is isn't this against WordPress' conflict of interest policy? It seems like it should be, but there isn't a conflict of interest policy. Automattic Doesn't Use This Algorithm for Their WordPress.com Plugin Search In 2023, there was controversy over the copy of the WordPress Plugin Directory that Automattic had created on WordPress.com and how it was outranking the original in Google's search results. Notably, Automattic has chosen not to use the same search algorithm despite it having been created and maintained by one of their employee. That can be seen by looking at the results for "translation": The backup plugin isn't in the results. Alternative Search Options Seeing as the poor state of the results of the WordPress Plugin Directory is not seen as a mistake and unlikely to be fixed, trying to work to fix it seems futile at this point. The next option would be to look for alternative options. There are now two that we are aware of. One has existed since last year, called Ploogins. That is marketed as being a "Cutting-edge AI-Driven WordPress Plugin Advisor". In our testing, the results of doing the types of queries they suggest provide results that look the same if you did a simpler keyword search. The AI element does create a significant lag in producing the search results versus alternatives. The search also includes some premium plugins. There are some glaring limitations, like a limit of 21 results to filter from, and months out of date information in some cases. Overall, it is a better alternative to WordPress' broken results, but there is a lot of room for improvement. The second option is our own search tool that we have soft launched. While still early in development, across a range of test queries, it has been providing relevant results while not have the same performance issue as Ploogins. We also give searchers more control of what they are searching on. For example, if you want to search by plugin author, you can do that. Or search only for plugins that extend WooCommerce or Elementor, you can also do that. A good example of how different the results are across the three options is a search for "series", which we came across as being mentioned as one the searches that the WordPress Plugin Directory produced bad results for in the past. With the WordPress Plugin Directory search, only the first result looks relevant: Ploogins only has a few results, but two that are relevant: Hidden away are more relevant results, though plugins that don't appear to be supported by their developers anymore: Whereas our search exposes those plugins as well: You can also see from that that we are incorporating our Plugin Security Scorecard grades in to the results. Our rankings incorporate that as a minor ranking factor as well. In light of what has been going on with WordPress recently we are working to include not just plugins from the WordPress Plugin Directory, but elsewhere in the ecosystem. We include plugins from WooCommerce.com (filtering out those also in the WordPress Plugin Directory), the Not WP Repository, and we just started adding plugins from CodeCanyon. We also allow developers of plugins not in any of those, the option to provide information on their plugins in a JSON formatted file to incorporate those plugins in our search results (those files could also be used by other search providers as well).

Plugin Vulnerabilities

A service to protect your site against vulnerabilities in WordPress plugins.

Tag Archives: Ploogins

Automattic Employee Changed WordPress Plugin Directory Search Algorithm to Promote Automattic’s Jetpack Plugin

Locking Down Security With WooCommerce Plugins Involves Assessing Its Security, Not Unrelated Things Like When It Was Last Updated

Locking Down Security