Major update: More precise Mangools Keyword Difficulty

Keyword difficulty has been an important metric and a vital part of KWFinder since the beginning. Now, we have updated its algorithm to make it even more precise.

Welcome to the age of Mangools Keyword Difficulty 2.0!

Why would we do this?

We are really proud that KWFinder’s keyword difficulty (KD) has been considered (one of) the most precise difficulty metrics on the market, because of the highest correlation with real rankings in Google SERP.

Many reviews and analyses went deep into the numbers and the algo we’ve developed proved to be one of the most accurate ones when compared to Ahrefs, Moz, SEMrush, LongTailPro and others.

Check the reviews by Moz, Authority Hacker or Authority Website Income, just to name a few.

Thanks to this, there was no reason to change or update the metric at all for a long period of time.

The first thought about a possible update of the KD algorithm in KWFinder came to our minds when we noticed that Moz has updated their metrics.

The original keyword difficulty

The original calculation was based on the Moz metrics:

• Domain Authority (DA)
• Page Authority (PA)
• MozRank (MR)
• MozTrust (MT)

This is a simplified version of the algorithm:

First, we calculated the overall Link Profile Strength (LPS) of each URL on the 1st SERP based on the metrics mentioned above. Then, we made an average of them. The final number represented the keyword difficulty.

Moz metrics changes

In May, Moz announced that they have improved the calculation of their metrics. What has changed? Well, a couple of things:

1. The backlink index got bigger and now it’s updated more regularly (in the past, it could take more than 40 days to index a new backlink)
2. Domain Authority and Page Authority metrics are updated more often
3. Moz got rid of MozTrust and MozRank metrics

What happened with KD?

One would think that more advanced input metrics mean more advanced KD. Or at least, that’s what we thought at first. (Un)fortunately, there were more and more questions abouth the “keyword difficulty jumps” so we decided to analyze the changes in detail.

The topic was widely discussed on forums too. This is one of the comments:

Here, we would like to express our deepest gratitude to all our users for their feedback. It was them who pointed out that the changes were deeper than we thought and made us look at what happened. Thanks for that!

What we found out

The main problem was a significant jump in PA values for URLs that had a very low PA before. The biggest increase concerned the PA values between 20 and 40, often jumping to 50 or 60.

As a result, their LPS increased and so did the KD metric of the keywords concerned. The “easy” keywords in the so-called green zone suddenly reached the “possible” or “hard” levels.

Here’s a quick comparison of how the difficulties changed in a randomly picked niche:

I could continue with many other “low-difficulty” niches, the numbers would be more or less the same.

Link Profile Strength vs. Keyword Difficulty

Now, there is one super important question:

Does increased LPS of some websites on the 1st SERP automatically mean that it’s more difficult to get on the 1st SERP?

No! And that means the KD as a metric that estimates how difficult it is to start ranking on the 1st SERP, should not be influenced by the changes of some of the websites so easily.

The magic of Google algorithm is that thanks to the relevance, websites with very low authority are able to outrank authoritative websites. In other words, the big players won’t always be in front of the underdogs because they may not be that relevant.

Moreover, Moz got rid of MozRank and MozTrust, so having an algo based only on two metrics wouldn’t make any sense. That’s why we decided to add the Citation Flow (CF) and Trust Flow (TF) by Majestic to the calculation. And we got rid of MR and MT.

A lot of tests and analyses were made to mix the inputs so they would correlate with organic search results as the KD used to before.

Keyword Difficulty 2.0

Thanks to the Moz update, the LPS itself became more precise and up-to-date. However, it didn’t reflect the overall keyword difficulty so well.

That’s why the KD is no longer an average of LPS values on the 1st SERP. The LPS is far more sensitive to changes in metrics and their weights. From now on, you will probably see higher LPS scores when compared to the overall KD.

Keyword difficulty reflects how hard it is to get to the 1st SERP, not how hard it is to rank first.

Although the old KD formula did the same, now, we have decided to highlight this even more by giving more weight to the websites with lower LPS when calculating the final keyword difficulty.

New ranges

Based on the changes, we have also updated the ranges. It reflects exactly what the updated algo is about. Originally, there were 8 ranges.

Now there are 6 ranges with less colors so it is easier to distinguish between the values.

But most importantly, since the KD estimates how hard it is to get to the 1st SERP, the “green zone” shifted to lower values from 0 to 29 (compared to 0 to 39 before).

Keyword Difficulty caching

We deployed the new KD algo on 25th September 2018. Due to this change, all previously calculated KD values are now outdated.

To indicate it, we have decreased the opacity of the values and added “N/A” in the tooltip.

From now on, all KD values that were calculated more than 30 days ago, will have decreased opacity as well.

What’s more, you can see the exact date when the KD was last updated in a tooltip. Just hover over the KD value.

This is a comparison of the old algorithm before the Moz update, the old algorithm after the Moz update and the brand new keyword difficulty:

We are looking forward to your feedback

We will keep checking the results and listening to your feedback. Don’t hesitate to contact us if you have any concern or input that would help other users to understand the keyword difficulty.