Optimization Blog

Large download button vs "Pancake" download button: Which version gets more people to click and go on to the download page?

Setup

Original (A)


vs
Alternative (B)

I used google’s website optimizer to show a large download button with a single line of text (A) as well as a “Pancake” style download button (B) with two lines of text on http://www.amfsoftware.com/windows/cdlabelmaker.html. The two pages were shown in a 50/50 split. The area of both the variations is EXACTLY the same. I basically doubled the height and halved the width of the large download button to create the “Pancake” variation. All other variables like wording, font, color etc. remained the same.

Measurement

The number of visitors who clicked the download button and went to the download page for the CD Label Maker.

Outcome

After an initial strong improvement for the “Pancake” button variation, the improvement dropped off and stabilized after about 2 weeks. The number of visitors who went to the download page varied from a decrease in conversions of 1.4% to an improvement in conversions of 2.3% for the “Pancake” button variation. Google was unable to determine a winner even after about 6 weeks. The variation in conversions was too small to be conclusive.

Uncertainties and Errors

Was the “Pancake” too different from the rest of the buttons on the site? Did this unfairly throw throw people off? I purposely left the Buy button alone, since it’s smaller than both the button variations and shouldn’t have affected the outcome.

Result

I plan to switch off this experiment soon and keep the large button variation. Large download buttons that stood out in the website proved a winner in the Download Buttons experiment.

Recommendation

Medium confidence: The large button variation that I tested against the “Pancake” button variation already stood out on the page. Furthermore, it is also already quite large, larger than most typical download links and buttons. I might also be pushing the limits of how much I can improve my download buttons. I do plan to test out other variations in the future for the sake of completeness though.

“Pancake” buttons do have a few advantages:
1) “Pancake” buttons can fit into a wider variety of website layouts, for instance on sidebars on most websites.
2) “Pancake” buttons with two lines of text are less likely to be mistaken as a fancy heading.

As long as you have download buttons that have a large surface area and that stand out at first glance in your website, use either variation according to your preferences.

Here are the statistics on screen resolutions as recorded by Google Analytics for visitors to http://www.amfsoftware.com:
1. 1024x768 40.00%
2. 1280x1024 18.60%
3. 1280x800 13.17%
4. 1440x900 6.04%
5. 800x600 5.47%
6. 1680x1050 4.67%
7. 1152x864 3.61%
8. 1920x1200 1.56%
9. 1280x768 1.54%
10. 1280x960 1.27%

Here are screenshots of the home page at the two lowest resolutions:
800x600

1024x768

During the Above the fold experiment, I saw a 68.2% increase in conversions to the CD Label Maker product which I moved to the top of the page. I targeted the CD Label Maker product since the Business Card Designer has MUCH stronger traffic via search engine positions to interior landing pages. The CD Label Maker has a few search engine positions that point to the home page of http://www.amfsoftware.com. The results were so conclusive that Google’s Website Optimizer determined the version with the CD Label Maker product above the fold to be a winner in less than a week.

As you can see my home page is arranged so that the two top selling products are above the fold at the most common resolution (1024x768). Placing the CD Label Maker product at the top of the page put the product in view for people at 800x600 resolution, placed the Buy, Try and Explore buttons above the fold for users at 1024x768 with two or more toolbars and placed the product and the buttons front and center for users at higher resolutions.

The bottom line is that more people were able to see the buttons that I wanted them to press without having to scroll. However one question still remains unanswered - was the presence of the buttons above the fold that increased conversions or does conversion continue to increase as content is placed higher and higher on a page? Is there a limit to this? For example, do people ignore content in the header of a site?

I’ve got an experiment in mind that should help me answer this question. I’ve got a big header that takes up space on http://www.amfsoftware.com. I plan on testing how removing this affects conversions. As it stands now, the Try button for the CD Label Maker product is visible to almost all my visitors without them having to scroll. Visitors can click on the buttons without having to scroll even if they have extra toolbars that push down content. Removing the header will push the content up; all the buttons will still remain above the fold but they will be shifted up. I’ve determined that content above the fold seems to convert better, this experiment should determine if the conversion increases as the content is pushed above even further.

First of all, I’ve updated the feed to include the entire post. I never intended for the feed to only include the title of the posts, but because of my relative inexperience with ExperienceEngine, it took me a while to figure out how to change the RSS template to include the entire post.

A big thanks to Patrick McKenzie for mentioning me on his own blog. Patrick McKenzie is one of those relatively rare bloggers who isn’t afraid to post actual numbers on his blog. And when I say actual numbers I mean numbers like these: Bingo Card Creator Sales Figures. This is information that NO ONE else out there is willing to post - information that I’m wary of publishing myself. Patrick’s blog is an amazing, amazing blog for anyone who has ever wanted to get out of the 9 to 5 trap.

He did bring up a interesting point. Just what kind of experiments do I plan for this blog?

The answer is simple. I’ve got two sets of experiments that I plan on doing:

The first set is basic web site optimizations focusing on size, position, fonts and images etc. Some of these experiments will probably not affect the bottom line for most websites. But I love to get as much data as possible and in case I do come up with some interesting conclusions, you’ll benefit without having to run these experiments yourself.

The second set of experiments will be far more interesting in general. I plan on testing different shopping carts and ecommerce systems head to head, test the efficiency of press releases and download sites in getting backlinks and more.

Finally I’m a big fan of continous improvement in all aspects of running a mISV and I plan to blog on how you can build products faster, with fewer bugs and that outsell your competition.

How does the location of content with respect to the fold on a website affect conversions?

Setup

Original (A)

vs
Alternative (B)

I used google’s website optimizer to show the CD Label Maker product information in two locations on the home page of http://www.amfsoftware.com in a 50/50 split. In the Alternative (B), the CD Label Maker product information was located at the top of the page, while in the original (A), the CD Label Maker product information was located below the Business Card Designer product information and therefore below the fold.

Measurement

Visitors rarely directly click on the Buy buttons on the home page of http://www.amfsoftware.com. Rather visitors click on Try or Explore for all the products. I decided to measure the number of people who clicked on the download button and went to the download page, since I was trying to improve downloads. It didn’t matter if less visitors clicked on the Explore button because of the change since I wanted more downloads vs more people clicking on the Explore button.

Outcome

I saw a 68.2% increase in the number of people who clicked on the Download button. The results were so statistically significant that Google declared alternative (B) to be the winner in about 6 days.

Uncertainties and Errors

Both the CD Label Maker and the Business Card Designer have landing pages deeper within the site that get significant amounts of traffic from search engine positions. The home page however gets a wide variety of visitors looking for old products, who come there from back links, download sites etc. I am not sure if the visitors who clicked on the links were interested in the CD Label Maker product or just clicked because the link was right in front of them. I am also not sure if the downloads actually led to conversions in sales.

Result

I placed the CD Label Maker product above the fold on the front page and I saw a boost in downloads as measured by Urchin. The idea of the fold on the website is so important that I plan to post about what the fold is, post statistics on screen resolutions for the visitors to http://www.amfsoftware.com and why the fold is so vital.

Recommendation

High confidence: Inspite of the experiment being specific to product placement on the home page of a multi product website, I believe that the region above the fold on the home page gets a greater number of conversions than content below the fold. I recommend that all mISVs place vital information, action buttons and key benefits above the fold. I’ll post more about the fold and explain this recommendation soon.

What am I?

You can do this with paper
And you can do it with cards
Those who keep this in mind
Will get the most rewards

Apologies for the cheesy riddle, but I’m also trying to have some fun with this blog alongwith providing useful information.

Hopefully most of you will be able to guess what the answer to the riddle is, but regardless of whether you are able to answer it or not, stay tuned. In a recent experiment I got a huge boost in conversions with a simple change that just took me 10 seconds. I think that this is the most important optimization for all mISVs and I’ll spend the next week blogging about the optimization AFTER I reveal the results of the experiment.

Large companies like Amazon spend millions of dollars optimizing their sites using A/B split testing. Some estimates place the conversion rate of visitors to customers on Amazon’s at an amazing 10%. If all mISV sites had a 10% conversion rate for visitors to customers, each mISV owner would be able to retire in a few months on the money that their site made. Instead conversion rates to customers are much lower on most mISV sites, usually around 0.01% to 1%.

Over the past year, I’ve learnt that the two most useful (non engineering) techniques that all mISVs can use to drastically improve their sales are Search Engine Marketing (SEM) and optimization. If done right, these two techniques will increase your sales by double or triple. The best part is that mISVs can do SEM and optimization without investing any money, unlike other traditional techniques like advertising.

While SEM is not simple, it is simple to see if you are succeeding. As long as your search rankings in the various engines improve, you know you are doing SEM right.

However optimization is a whole different animal. If you don’t change exactly one variable, you’ll never know what affected your conversions. If you change more than one variable, you might also get less of an improvement, since one variable might cancel the affect of another one. Finally, if you don’t carry out careful experiments and document the results, you might take two steps back for every step forward, actually hurting, rather than improving your conversions.

I started experimenting with optimizing my website http://www.amfsoftware.com sometime around July 2007 and over the course of these experiments I learned a lot of valuable lessons, both in desigining the experiments as well as optimization. I carefully documented these results and I realised that this was valuable information that could benefit the mISV community.

So here you are. All the results of my experiments will be posted to this blog, along with posts on optimizing in general.

Welcome to the optimization blog!

How to improve the download button to increase downloads?

Setup

Original (A)

vs
Alternative (B)

I used google’s website optimizer to show both “Buy” and “Try” buttons of equal size as well as a large “Download a Free Trial now” and “Buy” button for the CD Label Maker explore page (http://www.amfsoftware.com/windows/cdlabelmaker.html). The two pages were shown in a 50/50 split. This page was also the landing page for an adwords campaign which pushed more traffic to this page for the duration of the experiment.

Measurement

The number of visitors who clicked the download button on the CD Label Maker explore page. For this experiment, I not only used google’s website optimizer, but also measured bounce rates on the landing page using Urchin and Google Analytics.

Outcome

The number of visitors who went to the download page improved by 10% for the alternative (B) landing page.

Uncertainties and Errors

I made a fundamental error in experiment design - I changed too many variables during the experiment. Was it the size, order or the wording of the new download button that increased the conversion rate? Did the people coming in via Google Adwords skew the results in any way?

Result

I immediately changed the download button to the alternative (B). I also changed the download button in all CD Label Maker pages. There was an immediate increase in downloads and a drop in bounce rate as measured by both Urchin and Google Analytics for the landing page. I also resolved to change only one variable in future experiments.

Recommendation

High confidence: Make the download buttons in your website much more prominent by changing it’s location and size so as to “pop out” visually. Use a clear call to action as the wording for the download button. In future experiments, I will test the wording on download buttons to determine exactly what phrases get more downloads.

"Free Trials" vs "Downloads": Which phrase (in the main navigation menu of a site) gets more clicks from visitors?

Setup

Original (A)

vs
Alternative (B)

The phrase “Downloads” or the phrase “Free Trials” was shown in a 50/50 split using Google’s website optimizer on the home page of AMF Software (http://www.amfsoftware.com). The experiment was run for 10 days. 

Measurement

I measured the number of visitors who clicked on the navigation link and went to the downloads page using Google’s website optimizer.

Outcome

About 5% more visitors clicked on the tab with the phrase “Downloads” vs the tab with the phrase “Free Trials”.

Uncertainties and Errors

10 days might not have been a long enough period for this particular experiment to run.

Result

I kept the phrase “Downloads” on the main navigation tab. The phrase “Downloads” seems to be clicked more by visitors. The phrase “Downloads” might have a lot of recognition because of free music downloads and sites like download.com etc. I also resolved to run future experiments either for a minimum of 30 days or until Google’s website optimizer tool gave me a conclusive answer.

Recommendation

Low confidence: Use the phrase “Downloads” over the phrase “Free Trials” on any main navigation menu of a website.