Controlled Experiments on the Web: Survey and Practical Guide by Kohavi et al.
“If you are not doing controlled experimentation, you are not doing web analytics”
Eric Peterson
http://www.webanalyticsdemystified.com/
Controlled experimentation is an extremely important area for web analytics, but as yet it is not well understood, let alone practiced.
It is a technique that can and should be used for optimisation of email campaigns, landing pages, pay per click (PPC), and web site design.
Ron Kohavi, Roger Longbotham, Don Sommerfield and Randal M. Henne wrote an informative and comprehensive whitepaper on the subject.
The paper offers a comprehensive description of controlled experiments covering A/B tests, multi variable tests, hypothesis testing, sample sizes as well as limitations and implementation advice.
The paper blends real life examples from companies including Amazon.com, MSN and InterContinental Hotels with technical or statistical explanations. It is hence a useful resource for both the novice and advanced practitioners of controlled experiments online.
Real life examples will clarify the need for experimenting to those who are not convinced yet about the potential improvements to online performance through testing.
Reviewed by Lars Ammitzboell, Head of Analytics – Bienalto
Bienalto offers AB and Multivariate testing services in Australia.
Please call Lars on +61 2 8356 3007 to find out more.
