The other day, as I perused the latest “nerd news” from news.ycombinator.com I came across a very interesting blog post that captured my attention. It was a summary of the properties of Benford’s Law.

By way of brief introduction, Frank Benford first became aware of this property while working at General Electric, when he noticed that pages of a book containing logarithmic tables showed much more wear on the page for the number 1, than they did for other numbers.

After considerable research, he found that any man-made data has a tendency for the leading digit of each number to be a 1 around 30% of the time, and a 9 only about 4% of the time. This is true with tax returns, baseball stats, and just about any other data you can name. This is very odd, because one would assume that the numbers 1-9 follow a normal distribution (and they do, when generated randomly).

This article stuck in my mind, and somewhere over the last couple days it connected with something else that had been bothering me.

Benford and Search

By now, you probably know about the research that was done on the AOL Study Data that gave us a baseline for click-through rates on search results. This study showed that the number 1 result on a page gets clicked on about 35% of the time. The graph looks something like this:

Although this data has been very useful, what has always bugged me about it is that the study was only performed on the first 10 results, and so the only way to extrapolate a formula would be to graph the results, add a trendline, and use the equation of the trendline to predict other values greater than 10.

But take a look at this graph of numbers following Benford’s law (which has a known equation):

Eerily similar isn’t it? Putting the numbers side by side really makes it obvious that there is an interesting correlation between Benford’s law and SERPs behavior.

Result #	% of total traffic	Benford’s
1	35.64%	30.10%
2	17.82%	17.61%
3	11.88%	12.49%
4	8.91%	9.69%
5	7.13%	7.92%
6	5.94%	6.69%
7	5.09%	5.80%
8	4.45%	5.12%
9	3.96%	4.58%
10	3.56%	4.14%

Hammering Out the Details

First I took the equation for Benford’s Law and fed it the inputs of all numbers from 1 to 100. Unfortunately, beginning at #11, the numbers depart from what my gut feeling is on the actual click-through for pages beyond 1.

For example, the equation tells us we could expect a ranking at #11 to receive roughly a 3% CTR. Obviously, this seems high for the first result on the second page. But what if we assume that a similar number of people will click on the 10th result as will click on the ‘Next Page’ button?

Point of Clarification: The original algorithm provides for a 4.14% CTR on the 10th result. Assuming that another 4.14% of people will scroll down the page and click on ‘Next Page’ we use that percentage as the total amount for the next page.

Download an XLS file with the comparison and formulas.

By applying the same equation to the percentage of people who are predicted to click on #10, we see that roughly 1.14% of searchers will be predicted to click on #11. That seems reasonable, doesn’t it? I would say about 1 out of 100 times I will hit the second page of results and click on one of the listings.

Using this formula, it is predicted that searchers will basically not go past result #110, which is a pretty good prediction based on everything I’ve ever seen about search.

It’s All Voodoo

Of course, much of this is just arbitrary tinkering with numbers. But there is a great body of evidence that says that Benford’s law has applications to any human system of data.

Because search engines are merely aggregating data and applying a formula for ranking, it makes sense that it may be subject to some of the same underlying laws as the stock market, baseball stats, and other masses of human data.

Tags for This Post: aol study, logarithmic tables, nerd news, trendline, baseball stats, frank benford, couple days, normal distribution