Remember that time when you paid for a brand new Optimizely account?
Overwhelmed with excitement, you set up a couple of split tests in between sips of coffee, ready to prove to the world that changing the color of your beautiful call to action would dramatically increase your growth rate?
It never did, huh?
Because your website just had really lousy traffic.
source: my side project
Unless your website drives enough traffic, brilliant split testing will never give you meaningful results. Never.
In my early days as a startup founder, I quickly learned one hard-ass lesson in statistical significance.
When testing changes to a website, you must test for a period long enough for the results to become statistically significant. This means that the result is not likely to occur randomly but rather is likely to be attributed to aspecific cause.
The most important first step in this process is determining your target confidence level. The higher your percentage of statistical significance, the more confidently you can claim that changes you made on your website lead to specific outcomes.
A good rule of thumb is to go with at least 80% confidence.
A 95% confidence rate is what testers aim for but if you’re feeling like Kanye West, just remember, with numbers, you can’t fake confidence.
The reason we aim for statistical significance is if your website doesn’t have enough traffic, you won’t be able to distinguish between users actions that are random and actions that are happening because of a change made to the site.
Yet, there’s hope for low-traffic sites.
How To Make Statistical Significance Work For Your Puny Numbers
If you’re feeling down about the lack of interest in your site, just remember,according to Hubspot the average B2B company with 6-10 employees gets 124 unique visitors a week. That’s paltry 17 visitors a day! (I bet your numbers are looking better now)
1. Test big changes. Avoid the local maximum (hat tip to Andrew Chen)
Unless your website generates tens of thousands of hits per day, testing small changes will take too long to obtain meaningful results (Check with a sample size calculator). Test big changes that will give you new insight. For example, pit two completely different landing pages against each other or change entire layouts.
2. Record your visitors (with their permission)
The best way to determine what your visitors need is to watch them, LIVE! Thankfully, through tools like Crazy Egg, you can record live user sessions and heatmaps to create testable hypothesis.
3. Talk to your visitors
Don’t be creepy about it. Engage your visitors transparently by asking questions. Use a tool like Qualaroo.
As visitors use your product, target questions to them in order to obtain as much insight as possible before running any type of A/B testing.
4. Just Say No to multivariate testing
In theory, multivariate testing sounds amazing. Why wouldn’t you want random changes to multiple sections of your website shown to visitors automatically?
Sounds like a great idea right? Wrong! The more variables you test, the longer testing will take, chief.
Stick to one change at a time and be methodical about what you choose to test. Big decisions yield big outcomes so when faced with testing a button color vs. a redesign, go with the bigger choice.
5. Some A/B tests just never work
The more insight you have before you run your tests, the more chances you will have at a successful outcome. Remember, not all A/B tests succeed. Sometimes, when you don’t have enough information to build a proper hypothesis, you will fail because you tested the wrong thing.
Driving additional traffic to your website
On occasion, you just need an extra push to run your tests within a reasonable amount of time. There are several things you can do to drive additional traffic to your pages, lowering your test time.
1. Guest blog and contribute to sites within your target market
In the early days of search engine marketing, you could drive a ton of traffic to your page simply by paying someone to create thousands of backlinks to your website. Ah, those were the days.
Comment spam, keyword stuffing, and other black-hat SEO methods no longer generate traffic effectively. I sure don’t miss them; I hope you don’t either—they were skeezy.
Today, you have to put in some real effort in order to generate meaningful traffic.
Using BuzzSumo, search for keywords relevant to the content you’re going to be testing and collect a list of 10 or more websites that accept guest postings. Browse through the site to determine what the readership enjoys reading and write your own content for the site. Then, pitch it to the contact email for each website, light a candle, and spend a few minutes praying to your favorite deity.
Getting even one guest post could drive enough traffic both short term for the benefit of your test, and long term for your site overall.
2. Giveaways. Free makes everyone happy
Everyone loves free stuff. If you’re website offers a product or service, you’ve already done half the work!
Using BuzzSumo again, discover sites that your users read. Contact the site owners and offer special promotions to their users.
If you’re willing to pay a small fee, purchase a WordPress plugin like KingSumo Giveaways to create your campaign. On occasion, site owners might decline your giveaway and instead steer you towards sponsorships but, depending on your goal, that may be an excellent option. Which leads us to our final suggestion…
3. Pay up (or as marketers call it, “PPC”)
PPC (pay per click) can be used in conjunction with any other traffic generating efforts. Purchase clicks on on Google Adwords for intent driven traffic.
Because your ads display when relevant search queries are made in Google, the users that click your ads are looking for what you’re advertising. They will be more engaging and, more importantly, more eager to provide feedback. One thing to keep in mind with PPC is that it should never be used as a sole traffic generation method—use it sparingly.
If you’re ever in doubt, just remember, you ain’t Aziz Ansari. You don’t have the budget to make it rain. What Do You Think?
I’d like to hear about recent experiments that you’ve run.What are some low traffic experiments that have yielded great results for you?