WebShare Search & Conversion Marketing Blog

Search & Conversion Marketing Blog

Welcome to our blog! Here you'll find posts on the topics of web design & traffic acquisition, paid and organic search strategies, analytics & business intelligence, conversion testing, interviews, videos and more. We hope you find it useful, and invite you to read, share, subscribe, leave a comment, or just have a look at what's here.

Author Archive

Page Speed Tips – Part I

Thursday, October 21st, 2010 by David Evans

Search engines are continually improving their methods for determining which websites and advertisements gets ranked and placed higher than others. We know now that Google has added new ranking factors in both their search results and AdWords quality scores for page load times. While not (yet) a major factor, it is essential that your website teams make every effort to give your site every edge it needs to succeed.

In this segment of “Page Speed Tips”, part one focuses on the use of content delivery networks (CDN) for serving various data, script and/or visual elements from somewhere other than your own web server.

What is a Content Delivery Network (CDN)?

Simply put, it is a large network of servers, located in various geographical locations (generally worldwide) that contain copies of any files that you store with them. In turn, any user that visits your site will receive this content from the closest instance of a CDN server.

This is an example of how Akamai’s CDN is configured (image source: technofriends.in)

How does this benefit (and speed up) my site?

The primary benefit is that by serving files, such as images, javascript and css from an external source, your server will have less load against it which leads to an overall improvement in site performance. Secondly, the content being delivered by the closest CDN is likely to arrive quicker than if were to come directly from your (most likely) single location site. Lastly, search engines who crawl your site are still able to index your content for search results.

Can I trust my data to a CDN?

There are many CDN providers that exist today and more are continuing to enter the market. The simple answer is: Maybe. You need to find the right CDN for your site and weigh the pros and cons of cost versus service. A good CDN provider will have an impressive network of highly-optimized and highly available servers that provide solid, uninterrupted service and reliability. Bottom line is that if you choose a reputable CDN provider, you can count on them to make your data and files available.

How much is this going to cost me?

There are two costs at play here: The actual monetary cost of the service and the opportunity cost of missing out on the benefits of using a CDN. This market is becoming more and more competitive, so costs are beginning to fall and setup, management and rich features are constantly improving. While you can expect prices to start at just a few hundred dollars each month, consider the opportunity costs and understand the rich benefits you are gaining when making this decision.

Additional concerns?

One concern with having content spread across the internet is getting dinged for having duplicate content. In the case of a CDN, these are not configured as domains and are therefore not visible to search bots, such as “Googlebot”. The relationship between your site and the closest CDN remains transparent to human users as well as bots.

Which CDN provider should I choose?

We’re not going to promote any specific provider here, so you need to do your homework. Your particular situation is going to be unique, make sure the provider you select satisfies all of your

requirements.

How can I learn more about improving my page speed?

Google Web Page Performance Best Practices

Yahoo! Website Exceptional Performance Best Practices

Recommended Firefox Plug-ins

Google Page Speed

Yahoo! YSlow

David Evans
David heads up software and web design efforts at WebShare. You can find out more about David here.

See more posts by David Evans

Posted in Uncategorized | No Comments »

Are These Design Elements Providing the Expected Value Add?

Thursday, August 26th, 2010 by David Evans

We help answer questions like this all the time! And with this simple method, you can too.

With WebShare redesign projects, we do more than just build pretty websites. A truly good website is a combination of being aesthetically pleasing, functional and highly measurable. The purpose of the site has to be clearly defined and the results must be tracked, tested and analyzed in order to make informed decisions to better serve its purpose. Testing, measuring, analyzing: This is what we do.

We recently completed development on the new C3 Concerts website (www.c3concerts.com) and have configured some very common additional Google Analytics tracking to provide the necessary insights to make better decisions about the site. After collecting enough data, seeing how effective various design elements are on the site is a snap!

Example: There are two banners on the home page to showcase various events, promotions or news articles. An internal debate exists over the necessity and/or effectiveness of these banners.

Enter WebShare and Google Analytics. In their native state, these banners are simply links to other pages within the website. Clicks on these banners send the user to the expected page and GA records a standard pageview of that resulting page. However, while this shows how many visitors are viewing a particular page, the method doesn’t provide the insights of how I got to that page (other than knowing I came from the home page). For C3 Concerts, we add virtual pageviews to the onclick event of the anchor tag that links to the target page.

Using an organized naming convention for virtual pageviews makes it very easy to see in the Content Drilldown reports:

A banner was clicked
Which banner was clicked (1 or 2)
What type of announcement (event, news, promotion)
Info about the specific announcement

For example, Banner 1 links to a specific event:

<a onclick="_gaq.push(['_trackPageview', '/vpv/banners/banner1/event/name-of-event']);"
href="event.html" >BANNER</a>

Note the “vpv” that leads it off…by putting all virtual pageviews that we create in this base “folder”, we can easily create profiles in Google Analytics that filter this “fake” data out so as not to throw off our true pageview counts, bounce rates, etc…

In Google Analytics, a few clicks through the Content Drilldown report provides the answers needed to make decisions:

How many banners were clicked?
Which banner was clicked more often?
How many events, news or promotions were clicked via banners?
How many click-thrus per promotion?

About C3 Concerts

C3 creates, books, markets, and produces live experiences, concerts, events, and just about anything that makes people stand up and cheer. Among others, C3 produces the Austin City Limits Music Festival, Lollapalooza, as well as more than 800 shows nationwide. In additon, C3 offers representation services and publicity to artist and entertainers.

David Evans
David heads up software and web design efforts at WebShare. You can find out more about David here.

See more posts by David Evans

Posted in Google Analytics 101, Website Design | No Comments »

Tag Your Tweets and Track Your Social Campaigns in Google Analytics

Wednesday, February 17th, 2010 by David Evans

Businesses and individuals who use social tools, such as Twitter, Facebook, Blogs and YouTube, to reach their audiences are on the cutting edge of social media. What started as a simple conversation tool for individuals, social media has become a viable marketing tool for companies of all sizes. As such, it is imperative that organizations track the effectiveness of their marketing efforts. Unless you’re particularly savvy, promoting a sale or event by tweeting a link back to your site doesn’t give you an opportunity to properly credit the tweet as the source of any business.

Google AdWords does a thorough job tagging each paid campaign for Google Analytics to track and report the success of your advertisements, but what about Twitter, Facebook or Blog posts?

Enter the concept of campaign tagging for social media.

WebShare has a Social Media Tagging Tool that will do this for you, but if you want to know more about how and why to do this, read on…

Google Analytics has five “Traffic Source” dimensions for the purpose of tracking campaign traffic: Source, Medium, Campaign, Ad Content and Keyword. In AdWords, for example, the source is “google” and the medium is “cpc”. The campaign is the AdWords campaign name, the Ad Content is the headline of the ad and the Keyword is the word or phrase entered by a user to find your ad. For the purpose of tracking social media traffic, we can utilize these dimensions in a similar fashion.

Tagging your social media blasts involves appending GA tags to your URL. And while this can cause a lengthy URL, we further recommend using one of the many URL shortening mediums out there (e.g. bit.ly, j.mp, awe.sm).

There are three required variables in order for your tagging to succeed:

1. Source
2. Medium
3. Campaign

The two remaining variables are optional and are referred to as:

4. Content
5. Term

NOTE: In Google Analytics, “Content” translates to “Ad Content” and “Term” translates to “Keyword”

For social campaigns, we recommend using the following scheme:

Source:	“twitter”, “facebook” or “blog”
Medium:	“social”
Campaign:	A top-level category that describes the purpose of your social blast, e.g. “SeminarsForSuccess”
Content:	(optional) If anything, use this field to further describe the campaign, e.g. “FEB2010″ to define which seminar you are promoting
Term:	(optional) You could use this value to further define the content, e.g. “LasVegas” to describe the location of the FEB 2010 Seminar.

Honestly, there aren’t many cases out there where including “Content” and “Term” are that necessary as you could just as easily call the campaign (in the above example), “SeminarsForSuccessFEB2010-LasVegas”. In Google Analytics, there isn’t an intuitive way to view the granularity of all five campaign tags.

How do you tag your URL?

Tagging your URL is accomplished by adding specific campaign tagging values (as query string parameters) to the URL you want people to visit. The exact parameters (respectively) for Source, Medium, Campaign, Content and Term are:

1. utm_source
2. utm_medium
3. utm_campaign
4. utm_content
5. utm_term

Again, these will be query string parameters and must be appended to the URL properly so they allow the user to end up on the intended page and cause GA to consume the parameters. Easier said than done.

Let’s say, for example, you want your clients to visit http://seminars.websharedesign.com. By itself, this is a valid URL, but it is NOT tagged. A tagged version of the same URL might look like this: http://seminars.websharedesign.com?utm_source=twitter&utm_medium=social&utm_campaign=SeminarsForSuccessFEB2010-LasVegas.

In Google Analytics, you will see traffic for this link in your Traffic Sources > All Traffic Sources report. Initially, this will show you all traffic by source and medium:

View traffic by campaign Source and Medium in Google Analytics

HINT: If you wish to reduce the list to look only at “x / social” traffic, you may want to filter the list by entering “/ social$” in the filter and click “Go”.

Next, you should customize the report to include the campaign information. Just select the “None” drop down list (which is next to the “Source/Medium” heading of the report table) and select “Campaign” from the list. This will add the campaign value to the report and you will see stats for each social media blast you sent out.

Add Campaign to the Source and Medium report for better tracking data of your social media blasts

Now you can see that our tweet for the Las Vegas Seminars for Success in February 2010, successfully generated 1,039 visits to our site!

David Evans
David heads up software and web design efforts at WebShare. You can find out more about David here.

See more posts by David Evans

Posted in Google Analytics | 6 Comments »

Focus on Google but Don’t Leave Traffic on the Table

Thursday, September 13th, 2007 by David Evans

Nielsen / Netratings company reported that Google continues to lead the way as the search engine of choice for many users. The market research company also reported that Google has the highest retention of users. For at-home users, Google sees 79% of its users return, followed by Yahoo! at 69%, and MSN/Windows Live with 65%. The trend was similar for at-work users with slightly higher retention numbers. However, Nielsen / Netratings in the same article reported that search engine users are choosing more than one search engine for their search providing needs. In the big three, 84% of the MSN/Windows Live Search visitors went to either Google or Yahoo! or both as well as their usual MSN/Windows Live. 78% of Yahoo! and 63% of Google visitors went to the other engines as well.

What we should take from this data is that Google is clearly important to your search marketing plan – this is your classic 900 pound gorilla. But you should not forget about the other search engines unless you’re happy leaving potential traffic on the table. For example, when starting a pay per click campaign, there is nothing wrong with choosing Google’s Adwords to get a proof of concept. Since it has the highest amount of users, Google can give you an idea of whether or not your ad campaigns will be profitable. However, after your campaigns are proven, remember to consider expanding into Yahoo! Search Marketing and potentially Microsoft’s adCenter for access to more traffic. Spreading out to more than one search engine can often put you in a less competitive arena and will also help cover your business in the event users choose a different search engine from their typical choice. As the Nielsen / Netrating report shows, more users are searching more than one search engine these days.

It is important to pay attention to market research done by companies like Nielsen / Netratings to understand how to appropriately divide your search marketing efforts amongst the search providers. In the search marketing programs that we offer, we monitor and work towards results in Google, Yahoo!, and MSN / Windows Live, as well as a host of other paid and organic traffic sources. We work with our clients to help them understand the importance of diversity in supplying targeted visitors and how to appropriately budget to the different search engines, because as the Nielsen / Netratings report points out, the internet is becoming a very competitive place, and more of your potential customers are finding you from a more diverse range of sources than ever before.

For details of the Nielsen / Netratings report please see Top U.S. Web Sites Enjoy High Visitor Retention Rates, but Face Significant Audience Overlap with Competitors, according to Nielsen//NetRatings.

David Evans
David heads up software and web design efforts at WebShare. You can find out more about David here.

See more posts by David Evans

Posted in Pay Per Click Marketing, Search Marketing | 3 Comments »

Search Ads on Google’s Mobile Search Pages Set to Launch

Wednesday, September 12th, 2007 by David Evans

Google announced that within the next few days, it will begin including ads on the Google Mobile Search pages as an extension to its Adwords program. Google is promoting this new service to its advertisers on the premise that it will help advertisers reach additional qualified customers. The ads will be displayed to users searching Google on their cell phones free of charge until November 18th, 2007. Beginning November 19th, however, Google will begin charging its advertisers for clicks on the mobile ads unless you tell them otherwise. The trial period is available to all Adwords account holders, which will allow them time to experiment with the mobile platform by reaching actual qualified customers.

We highly suggest that you take an extra close look at your Adwords metrics to see just how profitable these ads are for your business and then make an informed, data-driven decision on how to proceed with your campaigns come November 19th. If you’re taking advantage of our Google Adwords management services, rest assured we’ll be doing this with your accounts.

Google is using this opt-out approach versus an opt-in approach, making this an opportune time to review your Adwords accounts. All Adwords account holders should review their ads campaigns to ensure that the campaigns not benefiting from the new mobile ads are excluded from that network. Additionally, if you plan to use mobile results then you should ensure that your ads are targeted towards mobile users.

Adwords account holders should also be advised that their mobile ads should reflect a landing page that can viewed in a mobile browser. Google is reporting that they have seen an upward trend in mobile usage this past summer, making this an opportune time for Adwords account holders to reach a new section of potential consumers. As wireless internet becomes more widespread and mobile devices (such as the Apple iPhone) become more user-friendly for the internet, this is going to be an exciting area to watch.

Read more in Google’s Adwords Help Center…

David Evans
David heads up software and web design efforts at WebShare. You can find out more about David here.

See more posts by David Evans

Posted in Pay Per Click Marketing, Search Marketing | No Comments »

WebShare Rolls out Managed Email Marketing Services

Monday, September 10th, 2007 by David Evans

This week we’ve rolled out a new service in response to the overwhelming need we’ve seen recently for a solid email marketing solution that allows for easy management, conversion testing, and perhaps most importantly, industry leading deliverability.

Frustrations with the management and legal aspects of opt-in list management are commonplace these days, and being able to perform A/B/N split testing or even

multivariate testing on your email blasts is becoming a necessity in the online marketing world. And once the mails are sent, keeping your email broadcasts out of the SPAM filters has become a problem that requires an entire department to manage.

We’re proud to offer our clients access to a system that can address all of these concerns and more – and has been helping businesses for more than 6 years and boasts over 10,000 users around the world. Depending upon your level of expertise or customization, we offer services from complete self-service email marketing to advanced services such as statistical experiment design and analysis of individual email broadcasts.

If you’re already doing email marketing, then you know that mailing lists, newsletters, email blasts, and customer communications can mean a big boost to your website’s ROI, and using professional hosting can help you make sure that you’re getting the most out of these efforts.

If you haven’t yet explored the powerful tool of email marketing, then what are you waiting for? You could be reaching your customer base and tapping into a fantastic

source of traffic and conversions for just pennies per email!

Find out more about WebShare’s email marketing solutions here and contact us when you’re ready to take your email marketing efforts to the next level.

David Evans
David heads up software and web design efforts at WebShare. You can find out more about David here.

See more posts by David Evans

Posted in Search Marketing, Website Promotion & LinkBuilding | No Comments »

On the Spot with WebShare: Three Questions with Google Website Optimizer Product Manager Tom Leung

Thursday, September 6th, 2007 by David Evans

by David Booth
Founding Partner at WebShare, LLC

In the world of Ecommerce, those of us who have been involved in conversion marketing and statistical testing know just how big an impact even the smallest of details can make to website performance and the bottom line. Some buttons just work better than others. Some product images provoke more purchases time and time again. Some colors are more likely to cause newsletter signups than others, and even details like punctuation and capitalization can have an effect on the rate at which your site converts visitors into monetized customers.

For some time now, the timeliness, breadth and depth of analytic data available about an array of usage metrics for websites has created a statistician’s dream – the capability to test virtually any aspect of a website and observe its relationship on conversions, time spent on a website, or any other measurable aspect of the visitor’s experience. Well, thanks to tools like Google’s Website Optimizer, you don’t have to be a statistician to reap the benefits of statistical conversion testing and ensure that you’re constantly improving the performance of your online activities.

We caught up with Google’s Website Optimizer Product Manager Tom Leung to talk about the new set of Website Optimizer features that was released yesterday, and here’s what he had to say:

WebShare:

How will the recent GWO improvements help new users begin setting up and running their first tests?

TL:

Our new A/B testing support reduces the set up time for a GWO test by roughly two thirds. We spent a lot of time making the A/B set up process as simple as possible so it is especially appropriate for those who are new to content testing or who aren’t as comfortable adding lots of JavaScript tags to their sites.With the A/B set-up process, so long as you already have two or more existing pages you’d like to test against each other, you can generally set up a test in around 5 minutes and start seeing data a few hours later.

WebShare:

What specific benefits does the new A/B Experiment option provide for GWO users?

TL:

A/B tests are great because they require minimal tagging to set up, they are great for trying new layouts and look and feel changes, and they can also be used to test alternate workflows. Since they generally involve only a few pages, they also require a smaller number of conversions to get statistical significance.On the other hand, multivariate tests are great when it comes to testing dozens or even hundreds of versions of a page. Multivariate tests give you much more granular results, uncover unexpected positive interactions, and once instrumented, allow you to test a lot more content. That said, multivariate tests do require a bit more tagging up front and since they generally cover a larger number of versions of the page, they require more conversions to get conclusive results.

In many cases, we’ve seen some of the most successful testers start with an A/B to find the general layout and look and feel that works best and then follow it up with a multivariate test to boost the conversion rate even further.

WebShare:

What is the most unique success story you’ve heard of that’s come from the GWO tool?

TL:

We hear from companies who have drastically improved conversion rates all the time. It’s not uncommon to hear about GWO tests finding pages that increase conversion rates easily by double digits and sometimes even more by trying new headlines, images, promotional copy, and call to action button designs. There’s a number of these discussed in detail on our case study page.Probably the most unique story I’ve heard was from an ecommerce company who used GWO not only to improve their landing pages but for making a business decision about where to invest their website improvement budget. This company was on the receiving end of a vendor pitching a trust seal which they claimed was far superior to the ecommerce company’s existing trust seal. When the ecommerce company indicated the proposed seal was a lot more expensive, the vendor said the new seal was so much more well known to consumers that the incremental sales generated by their superior seal would pay for the difference in no time.

Instead of the ecommerce company taking the vendor’s word for it or declining their offer altogether, the ecommerce company decided to use Website Optimizer to test the two seals against each other and make their decision based on actual customer data . In this case, they found that the new trust seal did not justify the cost. We’ve been told about other Website Optimizer tests where trust seal logos were shown to have very positive effects so you never know what works for your own situation till you test. Nevertheless, I love this story because it shows that Website Optimizer isn’t only about improving site content for conversions but can also be used to increase transparency and accountability for potential third party changes to your website.

A good conversion marketing strategy includes consistent experimentation of every aspect of your website. With tools like Google’s Website Optimizer available, it’s now easier than ever to set up tests and find out what aspects of your website can be changed to improve your bottom line. If you’re not converting at 100%, then you should have a test running!

David Evans
David heads up software and web design efforts at WebShare. You can find out more about David here.

See more posts by David Evans

Posted in Conversion Marketing, OnTheSpot, Search Marketing | No Comments »

Google Weighs in on Geography, Cross Linking and Web 2.0 Elements

Wednesday, September 5th, 2007 by David Evans

In a recent post from Google’s Webmaster Central Blog, Greg Grothaus and Shashi Thakur have shed some light on a few questions that are likely close to many web developer’s hearts.

1. The relationships between search results served to varying geographies, the geographic location of a website’s server, IP addresses and domain extensions are playing a part in which sites top the lists of search results pages.

Domain extensions can tell you something about a site before the first glimpse of a page actually loads. If you’re running a website in Australia and targeting Australian customers, then the .com.au extension will likely help you in Australian search results pages. “Because we attempt to serve geographically relevant content, we factor domains that have a regional significance,” states Shashi Thakur in the post. Additionally, IP addresses can be used to determine geographic location, and Google is not ignoring this information. Be smart about who your target audience is and where they’re located – you can use this information to your advantage.

2. What effects can be expected from cross linking a number of websites that you control, and what guidelines are there for linking strategies if you own and maintain multiple websites?

Shashi weighs in on this issue in a manner that is very consistent with any other linking guidelines that we’ve seen from Google – no surprises here. As with any other linking strategy, the general advice is that links between sites that “provide value” to a user are good, and links that are there simply for the sake of linking are bad. The interesting piece here is the explicit mention of links between topically or thematically related sites: “If the sites are related in business …then it could make sense — the links are organic and useful.”

This advice can be applied to any links, whether between two sites that you own or otherwise. The importance of links to your rankings cannot be understated, and making sure that your limited time and resources are applied to linkbuilding strategies in the most efficient manner is essential.

3. What suggestions are there to helping search engines “comprehend” websites that use DHTML, AJAX, Flash and other Web 2.0 technologies?

Greg Grothaus in his post reaffirms our belief that search engines are not human, and not yet capable of human comprehension. As human visitors to a website, we can look at a photo of a car, and in a fraction of a second draw the conclusion that we are looking at a car. We can watch animations and video and make sense of what we’re seeing. Search engine spiders, on the other hand, can “see” only a mess of pixels and compiled code that does not assist in comprehending what the images or animations are about. For this reason, when designing websites and pages, one must be conscious of the balance between what a search engine sees and a human user will see.

Google officially recommends using Flash “only where it is needed,” and warns of the dangerous ground you enter when your website displays different content to a search engine spider than it would to a human visitor. Without malicious intent, it would make sense to find a way to tell a visitor that does not have Flash installed (or a search engine spider) what they’re missing in a non-Flash manner. Techniques such as cloaking, doorway pages, scripted redirects, and the use of CSS to hide text CAN be used to accomplish this, but as these are the same techniques that are commonly abused in hopes of manipulating a search engine, using these on your site is begging to be penalized.

In a separate post, Mark Berghausen of Google’s Search Quality Team writes “the only hard and fast rule is to show Googlebot the exact same thing as your users.” We’d heed that warning.

David Evans
David heads up software and web design efforts at WebShare. You can find out more about David here.

See more posts by David Evans

Posted in Search Marketing, Website Design | No Comments »

What we can Learn from Fifth Graders about Conversion Marketing

Thursday, May 31st, 2007 by David Evans

Take a look at the following two sets of text, and as you read them try to gauge which you think would be more likely in converting a website visitor into a website customer:

1) “Have you been looking for the perfect blue widget to outfit your home or office? Then you’ve come to the right place at bluewidgets.com. We have lots of blue widgets in stock and you’re sure to find the right one for your needs.”

2) “Are you searching for absolute perfection in blue widgets for your residence or business? Let us cater to your widget requirements and provide all that you ardently desire. We maintain a full selection of blue widgets for your perusal.”

What do you think? Does the second version sound much more sophisticated? Does the first version describe a product that would fit your needs? Well, as Bill Nye the Science Guy says, “one test is worth one thousand expert opinions” – and THAT is conversion marketing.

The major differences between the two sales copy versions is the reading level that they represent. According to the Flesch–Kincaid Grade Level calculation, the first version is written at a 5^th grade reading level, and the second at a 9^th grade level. The way to find out which is better at selling product is to set up structured tests and measure the results. In this example, a set of standard A/B split tests could be used to measure the effect of reading level alone on different variations of sales copy.

Well, here’s the result:

When we pit sales copy at the 5^th grade level against copy written at a high school level in conversion tests, we find that – almost without exception – the 5^th grade copy will get more visitors to purchase the product that’s being described.

Why is this? Well, it’s not because your potential customers are all 5^th graders, or because all of your potential customers read at a 5^th grade level. The truth is that they’re not dumb, they’re just in a hurry! You have less than 8 seconds to convince a website visitor to stay once they’ve hit a page of your website, and 5^th grade level reading material is just plain faster to digest. A visitor can absorb the information much more quickly, and in a day and age where 1,000 more results are just a Back button click away, no one wants to read a white paper when they first hit your site.

So what other action items can we take from this result? We have another hypothesis to test: the more information a user can absorb in a shorter period of time, the more likely they are to convert. Let’s use the principles of conversion marketing to think of some more tests that could help us increase our conversion rates.

How about bullet points versus paragraph format product descriptions? We know bullet points are easier to digest!
How about instructional images (i.e. product demonstrations) instead of standalone product images? Tell the user how the product will meet their needs without writing a word!
Number of sentences or word count in your product descriptions, use of whitespace, location of pertinent information, the list goes on and on.

These variables can be tested one version against the other in A/B split testing or page version testing, or you could even measure interaction effects by designing and running a multivariate experiment.

David Evans
David heads up software and web design efforts at WebShare. You can find out more about David here.

See more posts by David Evans

Posted in Conversion Marketing, Search Marketing | No Comments »

The Power of Many – Conversion Marketing

Wednesday, April 25th, 2007 by David Evans

The Benefits of Multivariate Testing Over Traditional A/B Split Testing

We all know it’s easier to convert more of the visitors you already have on your website than it is to go out and get more traffic – it’s what we call Conversion Marketing. If you’re already running A/B split tests on your website’s pages, paid ad campaigns, and landing pages, congratulations – you’re doing the right things right. If your web marketing plan calls for, and implements, multivariate testing then you’ve moved on to the next level and are very likely enjoying the benefits of multiple variable conversion testing.

Multivariate testing provides us a way to statistically evaluate the immediate effect that certain variables (small changes to your website) have on any measurable goals of your website, as well as the effect that these variables exert upon one another (interactions).

To illustrate the power of multivariate testing, here’s an example:

As many of us know, “trust logos” can have a significant impact on the performance of a website. Let’s assume, for example, that in an A/B split test we tested the condition of A) having a particular logo against B) not having that logo using an appropriate sample size. Let us further assume that we found a statistically significant higher overall conversion rate for scenario A.

Another variable that we might consider is background color. For example, we can compare a blue background with a navy background, and find that the navy background color provides us yet another incremental gain in our overall conversion rate.

Logically, we would want to implement the two conditions that performed the best – having the logo on and using a navy background. When we do, however, we might notice that our overall conversion rate plummeted! What happened?

Well, the two individual winning conditions, when used together, did NOT constitute a winning combination. For whatever reason (and that could be something as seemingly small as even a slight clashing of colors) using the logo with a navy background provided unfavorable results even though using the logo or navy background alone gave us the boost we were looking for.

Design of Experiments (DOE – also known as fractional factorial designs) can help us to identify the main effects and the interactions of an array of variables on a control signal (or in this case, conversions). Let’s continue with the example we just ran, and explore the various combinations of this two-level factorial design:

Variable setting/combination 1: Logo ON, Background BLUE
Variable setting 2: Logo OFF, Background BLUE
Variable setting 3: Logo ON, Background NAVY
Variable setting 4: Logo OFF, Background NAVY

By measuring the overall conversion rate (or average page views per visit, or virtually any metric you may be tracking), we can find which of these combinations works best together. Now, this particular method calls for 4 tests to evaluate just 2 variables, and you’re probably thinking ahead. Three variables would require 8 test runs (2^3) – and what if I wanted to test, say 7 variables? That’s 128 individual tests I’d have to run!

The good news is that tools such as Orthogonal Array Testing Strategy (OATS) and fractional factorial designs can help us set up our test runs in a way that reduces the number of runs while allowing us to obtain statistically significant estimates of all main and interaction effects. When implemented correctly, our test of three independent (two-state) variables using OATS would require not 8 but 6 test runs, and at the expense of confounding the main effects with our two-factor interactions, if we wanted to try 7 variables at once we could cut our test set from 128 to just 8 runs. It’s certainly possible to try more variables – just remember that the more variables you introduce the more difficult the tests will be to keep track of, the more your results will be diluted, and the longer you’ll have to wait to introduce any other changes to your site (while these tests are being run everything else on your website must remain constant).

Multivariate testing and DOE can be a powerful way to understand how changes you make to your pages affect the goals of your website, and a good website should always be testing submit pages, funnel pages, landing pages, and product pages to make sure you’re making the best of the traffic you already have.

David Evans
David heads up software and web design efforts at WebShare. You can find out more about David here.

See more posts by David Evans

Posted in Conversion Marketing | No Comments »