Using this model, the time between identifying a problem and resolving it to unlock value can be as long as three years. Clearly, this is not ideal.

Fortunately, along came Web 2.0. One of the main principles of Web 2.0 product development, which has underwritten the rapid rise and success of its products, is the concept of ‘always beta’.

‘Always beta’ simply means designing and launching online web assets rapidly and frequently. At every redesign, as long as user-centred design principles are applied, the business notches up an incremental improvement on the experience.

Embracing the ‘always beta’ principle requires a major shift in mind-set for the heads of e-business, the online channel, and website managers.

It’s a huge departure from doing one major overhaul every three years – which involves an element of crystal ball gazing as you try to build a site that will work for your business over that period. With ‘always beta’, you manage ongoing development all of the time.

To overcome the organisational challenge of mobilising this approach for managing the online channel, you need a plan.

First, gain the full support of the internal and external technical development teams, so you are all working together on this ongoing optimisation path.

Secondly, arm yourself with the right tools. A key tool in capability development is a platform to conduct real-time user testing, also known as multivariate testing (MVT).

The notion of MVT is to get users to help you discover the optimum online experience. The qualitative research and expert knowledge is the basis of your product design. Once built, put it to the test in a democratic environment, where real users vote with their clicks.

MVT isn’t limited to website redesign. Marketers can use this technique to optimise email or paid search campaigns, or to optimise conversions on a website landing page.

For more information about transforming your business from a three-yearly web updater to an ‘always beta’ updater, get in touch with Bienalto today.

Yet MVT, when done right, can greatly enhance the page’s performance.

What is it?

Multivariate testing (MVT) is a technique to test the optimal combination of multiple factors within a web page to achieve the desired visitor behaviour.

Typically, MVT solutions will test and measure various combinations of factors, such calls to action, copy, positioning of elements, font size, colour, and so on. You can test multiple variations of each factor – for example, you may test three or four different images at the same time as you test different variations of a call to action.

To find the optimal combination, you need to measure the conversion rate against the same goal. The goal is any significant action carried out by the visitor, such as purchasing a product, downloading a PDF, or signing up to a newsletter.

Additionally, most MVT solutions will allow the marketer to design different landing pages depending on what segment the visitor falls into. So you can show different landing pages to visitors based on their keywords, referral sites, time of day and so on.

MVT can automatically be deployed using specific solutions such as Optimost, SiteSpect, Omniture Test&Target and Google Website Optimizer,

How does it work?

The MVT solution directs equal samples of traffic to the variations of the page being tested, in order to measure what the optimal combination is.

It can be a simple test, or very complicated.

In its simplest format, you would just test one change against another. This is normally called AB testing, where you essentially test two variations of the same factor.

Site owners can also choose to test three different images, together with three different versions of a call to action. So this makes nine tested combinations.

However, say you would like to test four factors and three variations at the same time within a landing page:

Image

• Image 1
• Image 2
• Image 3

Call to action

• “Apply now”
• “Download application form”
• “Contact us”

Color of font

• Black
• Blue
• Red

Position of “Call to action”

• Top right
• Top left
• Bottom right

You would then be looking at:

• 4 factors and 3 variations
• 34 = 81 tests (combinations)

For websites that do not attract a lot of traffic, it would take a long time to generate sufficient data that would make 81 tests statistically significant.

In this situation, you could use experimental design (some solutions use the Taguchi method). Using this technique, you can reduce the number of combinations required for testing but still achieve statistically sound results for each factor.

Remember, though, even though experimental design can help to reduce the number of combinations to test, more combinations will always mean that you require more traffic and longer time to make the results significant.

Most solutions do the maths for you, calculating upfront the amount of web traffic required to obtain significant results, given the number of combinations being tested.

A good starting point is to conduct incremental single factor analysis where one does not expect other factors to interact, and to conduct tests on several factors when one suspects interaction.

Why use MVT?

When you look at the benefits of MVT, you start to wonder why you have never considered it before. Here are three compelling reasons to give it a go:

1. Increased ROI

MVT is one of the fastest and most efficient ways to optimise the effectiveness of a campaign or web page, in order to increase ROI for marketers.

Marketers can use this technique to optimise email, paid search, or ‘DM to Web’ campaigns, or simply to optimise conversions on a website.

In our experience, there is often substantial difference in conversion rates between the worst and best performing combinations. By simply testing three different images or three different calls to actions, you could make a big difference.

2. Action-oriented research

MVT is one of the most action-oriented forms of online research available. Within a short timeframe, you know what works best for a particular page; then you can make the change, and see the improvement.

Action-oriented research works equally well for the main website, landing page optimisation, email campaigns or paid search campaigns.

3. Target different segments

From a marketing perspective, the ability to present different segment groups (based on keywords searched, links clicked, cookie recognition, and so on) with a different landing page is also a very powerful feature. In this case, you may not even want to do any tests, but simply design a customised landing page for different segments.

If you are interested in building this capability, get in touch with Bienalto. We work with leading MVT solutions, and can help you build in-house capability or offer MVT services as an outsourced provider.

In particular, there are three areas in which web analytics will add value in the next couple of years:

1. Web analytics integration with CRM
2. Multivariate testing and optimisation
3. Analytics driven Dynamic Content

The breakthrough has really been the data warehouse solutions provided by vendors such as Omniture, Webtrends, Unica, SAS. Although the first of these was launched a couple of years ago, it is only now that we are starting to see these being fully utilised by organisations.

Instead of just looking at summarised data, web analysts now have access to granular data that can be segmented, scored, or merged with offline data if ids exist.

Any Data Analyst would tell you that getting access to granular data is a key requirement for good analytics.

It is the granular data that has allowed a greater convergence with marketing. When you are able to identify and profile individual web behavior – anonymous or not – suddenly dynamic content and CRM become much more relevant.

You can store individual online scores in the data warehouse, and use this information for targeted campaigns (email, DM, newsletters, SMS, etc.).

Or you can present the visitor with dynamic content based on a database profile stored at a cookie level.

I expect that this will mean that many larger organisations will choose an in-house solution rather than on demand, but that will also involve significant hardware cost due to the massive amount of behavioral data. In fact, the key will be to decide which data to keep.

This post is a response to “What is the future of web analytics?” on the Web Analytics Demystified blog.

“If you are not doing controlled experimentation, you are not doing web analytics”
Eric Peterson
http://www.webanalyticsdemystified.com/

Controlled experimentation is an extremely important area for web analytics, but as yet it is not well understood, let alone practiced.

It is a technique that can and should be used for optimisation of email campaigns, landing pages, pay per click (PPC), and web site design.

Ron Kohavi, Roger Longbotham, Don Sommerfield and Randal M. Henne wrote an informative and comprehensive whitepaper on the subject.

The paper offers a comprehensive description of controlled experiments covering A/B tests, multi variable tests, hypothesis testing, sample sizes as well as limitations and implementation advice.

The paper blends real life examples from companies including Amazon.com, MSN and InterContinental Hotels with technical or statistical explanations. It is hence a useful resource for both the novice and advanced practitioners of controlled experiments online.

Real life examples will clarify the need for experimenting to those who are not convinced yet about the potential improvements to online performance through testing.

Download the full whitepaper

Reviewed by Lars Ammitzboell, Head of Analytics – Bienalto

Bienalto offers AB and Multivariate testing services in Australia.
Please call Lars on +61 2 8356 3007 to find out more.