Blogs

With the evolution of digital marketing and the increased availability of sophisticated digital technologies, the customers today have been put in the driver’s seat when it comes to making a purchase decision or becoming loyal to a particular brand. Customers today demand more information than ever in completing the purchase journey. Providing a complete experience to the customers during their online journey (e-commerce/non e-commerce) is thus, the main prerogative of a marketer. The marketing today, due to the evolution of mobile and social media platforms, is more pull based than push based. The content on these platforms in addition to the organization’s website is increasingly driven by what the end consumers expect.

Letting the end consumers have their say

So a thought that might be intriguing is that if the content is driven by the users, then why ignore the webpage design which is an equally important aspect of providing the consumers a complete online experience? UI design services try to ensure that the design is user centric but user centricity is very subjective.  To resolve this subjectivity in providing a complete user experience to the customers, today the major digital marketing solution providers are offering A/B Testing & Multivariate testing solutions. For organizations that are web centric are looking at the web as one of the major channels for lead generation, A/B or Multivariate testing is the path to look at in achieving those numbers. The major organizations that provide tools to enable A/B or Multivariate testing are Adobe Test and Target, Autonomy Optimost, Web trends optimize, Google Content Experiments, Amadesa etc.

 A/B testing and Multivariate solutions help marketers experiment with multiple versions of their websites and understand what combination and placement of the features of a webpage most appeals to its users. Depending on the type of solution, multiple versions of the websites can be launched at once with each version having a disparate group of users accessing it. Based on the success parameters or metrics for website, a marketer can then select the version which fits the bill the best. These success parameters may vary from one organization to another. For example no. of leads obtained, no. of customers signed up, no. of purchases made online, no. of users who accessed promotions etc. could be a few metrics that a marketer would want to measure.

A typical result dashboard of an AB or Multivariate test would look something like this

In the above dashboard, “control” refers to the original version of the Webpage without any planned variations. The other variations are the changes in the page which are being tested. ”Chance to beat original” is the confidence level which is statistically calculated based on the number of visits to the page.  “Impact” parameter tells that which section of your page has most influence on the conversion rate.

The three key pillars of successfully executing a website test are

1) Prioritize– Find key areas in your website which may be the weak link & may require your immediate attention. Test results here can help you identify low hanging fruits.

2) Iterate– Online site testing is a long term strategy for website optimization. Highly complex and multi-step web operations can be broken down into a series of simpler steps to find strong insights.

3) Use planning tools– The power of accurate planning is a key requirement for the success of such campaigns. Each test must be planned for time and traffic that it is going to attract and the key parameters it is going to measure.

Source: Forrester

However A/B or Multivariate testing may not be successful for all business scenarios. Reliabilityof these test results also varies on certain parameters which must be clearly understood before taking actions based on test results. The parameters are

1)  No. of Visitors – The more the visitors to your website, better will be the insights derived from these tests. There are tools available online to determine what should be an indicative number of visitors that would be required for a test to be considered successful.

2) Conversion Rate – Pages with higher conversion rates tend to produce significant results in much quicker time as compared to pages with low conversion rates. Conversion rates here are once again subjective to the business under consideration.. 

So what’s the difference between A/B and Multivariate testing?

The difference between a Multivariate and A/B test should understood very clearly. A marketer may need this understanding in determining what kind of test may suit his/her requirements. In an A/B test, we analyze different versions of the same web page which have been exposed to different group of users. The pages can differ in a few elements or can be radically different versions of the same page. A/B test is the perfect tool when you are inexperienced with testing and need quick results in understanding which variation of the page fits the bill in terms of the business outcomes expected.

 Multivariate test on the other hands tests very small changes within the same page helping the business identify the anchor element which is affecting the overall desired outcome. This test is ideal when there is a design or branding restriction on the page under consideration .This is usually performed to fine tune the results that a business gets out of the A/B test. Multivariate test is most useful when you have large amounts of traffic coming onto your website to specifically the page under test and you want to test how each element in coordination with the other element provides an efficient layout to attract your users. Multivariate testing can be done either by splitting incoming traffic to multiple variations of your website or via tagging various components of the page to be loaded differently in different variation

It’s definitely not a cakewalk

As easy as it may sound, AB or Multivariate testing plans come with their own set of challenges if there is a lack of clarity on how and what to measure. The tool selection and deciding on the key metrics to measure itself is a herculean task. The execution of these tests is of little value if there are no actionable insights derived from the test which can be documented and reused for future implementations as well. A test is only considered complete once these final steps are also performed.

The template driven nature of most Web Content Management and e-commerce tools require database access for the content to be populated at runtime. This poses a challenge in the testing process as there might be some additional work that must be done in order to ensure accurate testing execution. 

Tagging is yet another challenge to deal with when initiating the testing process. Based on the tool used, users might need to repeat the tagging process with all the elements on the page each time a test is run & the tagging process typically needs the intervention of the IT teams. Today some enterprise level multivariate testing platforms allow for a one time tagging process and let you test repeatedly without the need to change the tags with every single variation.

Concluding thoughts

The look and feel of a page is a very subjective issue and there may be no single answer as to what is the best way to design a webpage. Multivariate and A/B testing provide a solution which helps to an extent in reducing this confusion by giving the users the right to decide what is the most appropriate design for them. It is a tool for the organizations of tomorrow who wish to empower their users to decide what the web interface should look like. It is a tool for marketers who wish to reduce the cost for lead acquisition by optimizing low cost channels like the web and mobile for best user experience.