Ever since my good friend and agency-world refugee Luke Van Deman introduced me to www.abtests.com I’ve been following it closely, checking their feed every day for new insights.

Opening their RSS the other day I saw an interesting test: a 22.5% improvement over control.

The difference between the two pages? Absolutely nothing, except that the test page had Google Website Optimizer code on it.

Now, the whole story hasn’t come out on this yet, and I’m sure there are more tests in the works to confirm the results, but it is important to remember compounding/confounding factors can still play a role, even in tests where the control and experiment are (nearly) identical.

My first step would be to time the average load time of the Google scripts that load the experimental code and see if it adds materially to the load time of the page itself.

This could easily be done via Pingdom tools, or Google’s Webmaster Tools (screenshot below). Either way, load time plays a huge role in user experience.

Tags: google, load time, new insights, van deman, world refugee

This entry was posted on Friday, January 22nd, 2010 at 9:35 am and is filed under Strategy, Web Basics. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.