Chapter 5.2: Where do we start: Look to Your Research

by James Young

Our next place to look for A/B testing ideas is research you’ve done (or could do) with your customers and prospects. There’s nothing quite like the phrase, “We have research data that shows …” to make your next marketing/creative meeting a lot more pleasant and productive!

Don’t have any studies coming up? No problem — dust off an old one. I’ve encouraged some of my clients to look at weeks-old (or even months-old) studies. I recommend you do the same — chances are you’ll find a data point you can build a test around! Here’s how …

 Usability testing

There’s almost certainly something in the last usability test you did that you could test. Look back at the videos, transcripts, or notes. You’ll most likely find that one (or all) of the test subjects couldn’t find a button, figure out the navigation, or complete a task. That page or experience becomes your “control” — and you can be creative about how you use design, color, layout, typography, wording, etc. to try to beat it.

 Focus groups

Believe it or not, focus groups aren’t just about sitting in a dark room behind one-way glass eating M&Ms all day. My writer pal Chuck Vadun views them as a great opportunity to hear customers talk about the benefits of your offering “… the way ‘real people’ would,” he says. “They won’t use words like ‘comprehensive diagnostics’ or ‘guided workflow’ or ‘enhanced productivity.’ They’ll speak in understandable terms you can use to inspire headlines, subheads, and bullet points to use in your test recipes.” (More from Chuck about the benefits of focus groups here.)

 Market research

This type of customer research is a rich mine for unearthing “gems” to test. You’ll learn what your customers’ biggest needs and concerns are. Then, outline the ways your product addresses those issues, and use the outline to create a narrative (more about narratives here). Weave the resulting story through your page, site, or experience; most likely it’ll read quite differently than what’s there now. How to determine which is best? You guessed it: test!

 Site metrics

Buy your web analyst a latte and ask him/her for site metrics: it’ll be the best five bucks you spend all week. He/she can show you all sorts of great stuff, like where your traffic is going — and where the biggest drop-off points are.

 Once you’ve identified the path that’s getting the most traffic, you can focus on that experience to make the biggest difference to the highest number of visitors. Also take a close look at the pages where you’re seeing a lot of visitors exiting. If you’re stumped as to why, do some “guerrilla usability” — grab someone who doesn’t spend much or any time looking at your site. Ask “what do you think we’re trying to do on this page?” Or give them a task to complete. Or even just watch them interact with it. There’s a high likelihood that you’ll find the problem(s) this way; from there, you can generate A/B test ideas for fixing the page.

Dig in

Remember that your research data isn’t just about seeing what happened; it’s there to help you move forward. Facts become learning … learning becomes insights … and insights become hypotheses. So start digging into your data — and chances are it won’t

Ideas by James Young. Structure by Chuck Vadun.


See also:

Chapter 1: Infusing the culture of A/B testing

Chapter 2.1: What exactly is A/B testing, anyway?

Chapter 2.2: What exactly is A/B testing, anyway? Part 2: Multivariate testing

Chapter 2.3: What exactly is A/B testing, anyway? Part 3: Lean Testing

Chapter 3: What to Test For

Chapter 4: What is a hypothesis, and why is it important?

 Chapter 5.1: Where do we start: Let’s just start with basic marketing