A recent Forrester report analysed the state of online testing in the US market. The findings of this report can be used as a barometer for what’s happening here in Australia, given that we’re still in “early development” phase locally – just a couple of steps behind our US counterparts.

Forrester surveyed 71 online testing practitioners. These are people who use a “statistically sound methodology for conducting experiments to determine which content, promotions, and designs resonate most effectively with site visitors” (Forrester, 2012).

In other words, a customer-led way of seeing what works online, and what doesn’t.

Here’s what the Forrester report found:

• Reason for testing: Usability is most cited reason for testing (70%) – more so than revenue generation (58%), customer engagement (42%) and segmentation and targeting (42%).
• Benefits of testing: Increased conversion is the most popular benefit (73%), followed by increased landing page conversion (39%), increased order value (28%), and improved segmentation and targeting (24%).
• Industry trends: Retail and wholesale distribution is the largest adopter of online testing (39%), followed by media, entertainment and other consumer services (17%). Finance and insurance only accounts for 7%.
• Target markets (B2B vs B2C): Only 9% primarily or exclusively target businesses, compared to 78% who target consumers.
• Ownership of testing: Marketing / eCommerce departments own online testing in 56% of cases, and a dedicated analytics department owns it in 30% of cases.

The report then does a deep-dive into some interesting stats about the types of tests that organisations use (think multivariate techniques like full factorial, partial factorial-Taguchi, partial factorial-vendor tuned and more), as well as measures for the results of these tests.

Despite painting a positive picture of the state of online testing, the report found that optimisation is still a relatively new activity in organisations. This finding alludes to the importance of getting executive-level support for testing – a theme explored in the Bienalto Breakfast Briefing held last week.

It also found that there is a major lag in adoption of mobile optimisation, with few users testing anything beyond fixed internet websites.

If you are at all interested in fine-tuning the performance of your digital assets, then we encourage you to read the full Forrester report: The State of Online Testing 2011.

Read the notes from the Bienalto Breakfast Briefing: Online optimisation

• The industries represented in the survey mimic that of last years in terms of their participation levels. The largest sectors were Public Sector (42.3%), Telecommunications, media and technology (21.2%), Financial services (banking, insurance investment) (13.5%) and Agriculture, construction, manufacturing, resources (9.6%). All other industries were represented under 5%.
• Over half of the respondents worked for organisation employing more than 1000 people. Small business employing 50 people or less were 11.5% of the respondents.
• 81.8% of the respondents rate their web analytics knowledge as intermediate or advanced.
• 53.2% of the respondents reports to have either effective or highly effective online channel which is marked increase from last year’s 47.9%.
• 51.1% of the respondents describe the business impact as critical if the online channel has availability issues as it is a primary channel and any issues would negatively impact sales and operations. Another 29.8% report it to have some impact as it supplements sales and operations. These results show that that Australian businesses started to rely heavily on the online channel.
• Three biggest challenges to the adoption and success of web analytics with an organisation are reported to be correlating data from multiple systems, taking actions based on results and finding staff with relevant experience.
• Optimising websites using analytical insights is the top priority for they respondents followed by formalisation of the reporting process.
• 20% of the organisations represented exclusively use a paid web analytics solution, 60% use both paid and free and 40% rely exclusively on free web analytics solutions.
• Google Analytics is the most popular solution used by the respondents (77.5% of respondents) followed by Omniture Site Catalyst and Web Trends.

Here’s a summary of what Derek about. We encourage you to have a read and post your own comments or questions below.

The Australian optimisation scene

Optimisation is very much an outgrowth of the old marketing maxim: “Half of marketing dollars are wasted, but which half?” As analytics gives us more and more data, how do we make use of it? Enter optimisation testing – an inevitable evolution of analytics.

The Australian market is very much on track with the US market – yet it lacks the big player early adopters who are out in front pulling the pack behind them.

I would say the local market is in “early development” phase. Optimisation testing is mostly ad-hoc, a separate line item from general operations. But this is changing as results come in and companies see the ROI possibilities.

As it matures, organisations see value in a number of ways, for example:

• Increasing online sales conversions to generate more revenue
• Increasing online support conversions to cut costs associated with call centres or bricks and mortar branches
• Identifying the most effective assets or pages on the website/s to reduce the total amount of web real estate in the business, thus reducing the cost of maintaining pages and assets that play little to no role in conversions.

Australian companies are nearly there – and will soon find out what works for them with assistance from specialised agencies.

Where to start with optimisation

The approach can be as varied as the business models and goals of the companies doing the optimisation. For example, do you want to increase lead or sales conversions? Increase online engagement or online support usage? Are you trying to leverage segmentation targeting to cross-sell or up-sell?

Many advanced optimisation groups will focus on one area at a time, addressing all of these varied interests along a timeline.

Once the strategic areas are decided upon, you need to identify your starting position. For example:

• Do you start at the top or the bottom of the funnel? The answer to this question benefits from solid analytics.
• Do the analytic reports show a current bottleneck in the funnel? Starting at that bottleneck is a good approach.
• Do you have a significant online ad spend, or are you increasing our online marketing spend?  Starting at the landing pages or other entry vectors for this traffic may help to maximise the value of getting these visitors to your site.

Regardless of your starting position, you should then follow traffic through your conversion funnel – whatever it may be – with the goal of widening this path and ultimately increasing your strategic conversion metric.

A simple measurement formula

So how do you measure optimisation? While there are many ways to measure the online customer experience, I like the clean mathematical approach where 
C=4m+3v+2(i-f)-2a (C=conversion, m=motivation, v=value statement, i=incentive, f=friction, a=anxiety).

It’s not a perfect mathematical formula, but gives you a nice idea of how each factor relates to each other. Notice that motivation has a multiplier of 4 – it’s hard to stop properly motivated traffic!
The other positive factors – value and incentive – also play a key role. Lowering the overall score are friction and anxiety.

Let’s take a look at each factor.

• Motivation – converting unmotivated traffic is more difficult so you want to drive motivated traffic that wants to interact with your site.
• Value statements – why choose your company?
• Incentive – is there a current special or promotion?  Why should they act now?
• Friction – is the site difficult to navigate?  Is it hard to find what you’re looking for?  Or, simply, is the page readable? Is the next step clear?
• Anxiety – are there any unaddressed concerns about transacting? For example, are privacy or return policies made clear?  Zappos.com solved a key anxiety problem about buying shoes online by introducing free shipping to its returns policy.

These variables are great conversation starters when discussing the holistic optimisation strategy for your site.

Benefits of optimisation

The benefits are easily seen.  I have seen companies realise full ROI, fast. I managed an optimisation program that increased online sales of our flagship product by over 416%.

But it pays to set realistic expectations.  Everyone wants instant success, and sometimes that happens. But as in all business, success may not come overnight. Think of optimisation as a marathon, and ultimately part of how we do business.

Companies with mature optimisation and testing programs don’t just discuss the homepage. Which homepage, for which segment, or for what test experience?  These companies test everything – very much guided by their customers. For some businesses, putting power in the hands of customers and letting the data decide is a big departure to how things were traditionally done. But it’s worth it.

Challenges of optimisation

Mostly, there is not enough of the crucial executive level support.  Without this, the program can wither on the vine from lack of frontline support and the resources to get the job done.

Another political pitfall is choosing the proper success metric.  If a test increases one metric but lowers another, or the metrics are split between multiple experiences, which one wins?  If these metrics are “owned” by separate groups then it can turn political, as each group has been mandated to meet their numbers.

To avoid these political challenges, decisions should be governed by bigger picture goals. What helps the company’s bottom line more? Maybe online conversions go down, but customers purchase higher value items with better margins. Perhaps lifetime value increases through support or engagement interactions.

It’s important to have these discussions to guide the optimisation program and shield it from potentially stifling political fallout.

Another challenge can be implementation, and basic go-to-web processes.  Keep an eye on how much effort it takes to launch a test, and optimise this process to allow room for your program to grow. This can be done through a framework-style implementation, or by ensuring that the proper resources are available and dedicated to make it happen.

Examples of companies getting it right

It’s all about having executive support to ensure proper resources are dedicated to optimisation. In these companies, site optimisation gurus are evangelising the results to the various groups and stakeholders, and getting the right people involved.

Symantec and Dell are the global leaders in optimisation.  At Symantec, testing is a global culture and they run 160+ tests per month. To govern and handle this volume they have built their Test and Target implementation into a global framework across all of their properties.

Knewton and Flat World Knowledge are both smaller educational sites that are both taking a very adaptive learning approach to optimisation.

Snapshot: Forrester reveals the state of online testing

A recent Forrester report analysed the state of online testing in the US market. The findings of this report can be used as a barometer for what’s happening here in Australia, given that we’re still in “early development” phase locally – just a couple of steps behind our US counterparts.

Forrester surveyed 71 online testing practitioners. These are people who use a “statistically sound methodology for conducting experiments to determine which content, promotions, and designs resonate most effectively with site visitors” (Forrester, 2012).

Read what the Forrester report found

What are your thoughts on online optimisation? Where is your organisation at? Post your comments or questions below.