Search marketers' use of ranking reports and metrics have, since way back in 2008, fallen sharply out of favor. Early objections to using them to measure organic search success were often technical in nature (search engines were clamping down on automated tools) and business related (clients were only focused on their rankings).

Focusing on metrics 'down the chain' of the consumer journey such as organic traffic and conversions became more strongly emphasized at the same time that search engines reduced our access to reliable data via search personalization/localization and removing keyword data from Google Analytics. So whither the keyword ranking report today?

Ranking Reports vs the Ranking Index

Your typical ranking report shows the position and movement of each keyword and often some aggregate data like 10 keywords moved up and 12 moved down. These kinds of reports rarely imply actionable insights or even a sense of the overall success or failure of organic search marketing efforts. Additionally these reports can lead to 'where do I rank' myopia - where the client obsesses over rankings to the exclusion of metrics such as traffic, conversions, and sales.

In "The Death of Keyword Ranking Reports, 10 Superior SEO Stats" Cyrus Shepard writing for Moz mentions your usual suspects, traffic, business objectives and what not but first he brings up the concept of the keyword index.Search Index

WebPosition, back in the early days, used this concept in their desktop software - it was little understood and probably even less frequently used. On their ranking reports they would assign a point value for every keyword that ranked in the top 30. First position got 30 points, position #30 got one point and so on. Using this index, as keywords moved up and down (as they do), you could get a sense on the overall movement of rankings across all keywords being tracked. The 'score' was an arbitrary measure (i.e. an index) that allowed for comparison month to month or however frequently you ran your reports. In his article Shepard mentions that this can obviously be done manually but also that Moz can do it - however, at this time, I'm not seeing this feature within Moz. Certainly this feature doesn't exist combining the concepts of an index with metrics such as search volume and CTR as I do below.

The Weakness and Strength of the Keyword Ranking Index

The weakness of the 'index' approach is that you need to set it up right - especially if you're using automated tools - changing the keyword group you're tracking obviously changes the index, making comparison not possible. An additional weakness is that you have to get the keywords in the keyword group(s) right - stuffing your dataset with a bunch of easily-rankable-but-low-volume keywords or impossible-to-rank-for-but-high-volume terms will affect the value of the index (hyphens!).

What is, and has been, quite clear is that your standard 'keyword report' has limited usefulness. Pursuing through the ups and downs of 50+ keywords doesn't leave you with much actionable insight or a decent idea if your efforts have been, on a whole, working or failing or kicking ass.

Creating an Index that Incorporates Search Volume and Position CTR

Below, I walk through how we might use keyword rankings from specific (keyword ranking reports) to abstract (keyword indexes) to value-driven (weighting based on volume and CTR).

After that I walk through some examples of how you might calculate 'value' based on recent research on click-through-rates for organic search rankings. While I find this value-based index is better measurement, its biggest weakness is sensitivity to exact search position combined with that there really is no 'exact and true' ranking.

However, the value of looking at keyword rankings is not to obtain some objective 'truth' (look to traffic, conversions, and sales for that). It is instead valuable because it gives you one piece of the puzzle at an earlier part of the process - "are our efforts having an appreciable effect on keyword ranking"? And, as I outline below, "are our efforts having an appreciate effect on keyword rankings for keywords that matter and provide value"?

Let's look at ways that we can order keyword data from general to specific:

  1. Obtain the raw keyword data: where do you rank for given terms in search engines (presently and over time)?
  2. Create raw data aggregates by averaging keyword rank
  3. Start building your index by assigning point values to keyword rank
  4. Weight keyword position value by CTR estimates
  5. Use CTR estimates of keyword position and estimated search volume to weight keyword value
  6. Use keyword specific CTR estimates of position and volume to weight keyword value
    • Branded keywords have higher CTR
    • Ads number and placement significantly affect CTR - also the placement/number of ads affects this
    • Long tail has higher CTR

Example calculation of keyword ranking value

Below I calculate some keyword ranking values. For organic clickthrough rate numbers I rely on this research (http://www.advancedwebranking.com/ebooks/Google-Organic-CTR-Study-2014.pdf). To truly apply this metric you'd have to do some number crunching and segmentation using this dataset for CTR (https://s3.amazonaws.com/caph.awronline/resources/keywords/publicstats.zip).

Sample CTA distribution

From the research paper I extracted the approximate values below.

Aggregate (% of clicks per position/page): 31, 14, 10, 7, 5, 4 (#6-10), 4 (2nd page), 2 (3rd page)

Branded (% of clicks per position/page): 51, 12, 7, 4, 3, 2 (#6-10), 5 (2nd page), 1 (3rd page)

Unbranded (% of clicks per position/page): 26, 15, 11, 8, 7, 5 (#6-10), 4 (2nd page), 2 (3rd page)

The study also has numbers for ads/no ads however it's not fully segmented with branded/non branded - we'd have to dive into the raw data to get the proper distributions - because of this, the example keywords below do not return ads. 

In theory if we had a good idea of things like conversion rates and conversion value we could take this even further. Below, we assume that traffic from branded keywords convert at 5% and non branded at 2% and we've calculated each conversion has a value of $2000.

We can also use this to calculate the estimated value of changes in rankings.

In the example above, if Example C moved from #5 to #2, the new position would have an estimated value of 15% x 500 = 75 estimated traffic. 75 x .02 x $2k = $3000 which is nearly double the expected value of the keyword in the fifth position.

Since we can't directly know the conversion rates of specific organic keywords we could use an aggregate conversion rate. Leveraging insights from AdWords to approximate conversion rates at a keyword level is possible, but potentially problematic.

Interpreting and Utilizing Expected Ranking Value

While expected value is interesting, it's likely not real useful as the number of keywords that you would ever monitor is usually going to be a small subset of the total number of keywords bringing in organic traffic.

Because of this, it couldn't be used to say "our SEO efforts are expected to be worth $X over the course of the last month" - obviously you should be looking at real data on the analytics side of the fence to determine that. It could be worth looking at if you had a small number of absolutely crucial keywords that likely brings in the vast majority of conversions.

However, totaling up 'estimated traffic' can be used as an index and tracked. 'Estimated traffic' should be referred to generically (e.g. score) due to the same issues with 'expected value'. This should provide better insight into tracking the success of organic search marketing efforts as opposed to just tracking a sampling of keyword rankings. In the later method, a bunch of keywords that have no search volume that move from #20 to #15 would be equally weighted (and seem as valuable) as keywords with heavy search volume moving from #5 to #1.

Next Steps: Streamlining calculation

Currently, there are no existing tools out there that allow us to fully automate these indexes. Looking at the data we need:

  • Keyword ranking/position - this can be pulled down from reports from services like Moz or programmatically via API to one of those services such as http://www.ginzametrics.com/features/api/.
  • Search volume - should be able to pulled down via the AdWords api as described here https://moz.com/blog/search-volume-data-excel
  • CTR data (aggregated and segmented) - the raw data from the study cited above could be used to calculate the segmented CTR otherwise the aggregated CTR is easily obtainable from the study
  • Number of placements of ads on per keyword basis - I have the least amount of knowledge on this. If you use just the aggregate CTR rates then you don't need to bother with this anyway. However, if we did want to use it a couple questions need to be answered:
    • How reliable are manual spot checks of ad arrangements for a given keyword?
    • How dependent are they on geography and personalization?
    • How frequently do ad placements/layouts change?
    • Is there a way to automate checking this?

My next step is to put this in action in a few different ways and report back on the value of the resulting indexes.

Chris Olberding Chris Olberding

Co-founder and Creative Partner at Station Four

Share