Chapter 2.3: What exactly is A/B testing, anyway? Part 3: Lean Testing
Here’s a testing concept I just love: lean testing. It’s perfect for testing the idea of a product or service offering, without have to actually FINISH the product or service offering. Think of it this way: before you spend time and money getting an offering to market, wouldn’t it be great to see if anyone would even be interested in it?
Send it up the flagpole
Lean testing came into vogue with Erik Ries’ book, “The Lean Startup.” Ries writes that by releasing what’s called a “minimum viable product,” you can gather customer feedback that will help you improve your offering — by tailoring it to better meet the needs of your target customer.
So, you can use lean testing to assess what your customers demand — and then determine how to meet that demand using the least amount of resources possible.
What’s lean got to do with it?
How do lean testing and A/B testing work together? Say you want to test an offering that, for whatever reason, isn’t ready for prime-time. Maybe you don’t have the resources or the time to build out the full offering, but you’d still like to know if the concept of it has legs. No problem — create an A/B test recipe as if you actually had the offering. Then put it up against your control version, and see what kind of a response you get.
“Vet” your idea
Let’s say we wanted to test the idea of providing free vet care with a pet adoption, before we were actually able to do so. We could get an idea of how much “lift” it would provide, and determine from there if we wanted to continue putting together the offering.
Creating the offering shown in Recipe B could take a lot of time, effort, and money — and might not even get the results you want. Why not do a quick A/B test to find out first?
Here’s how you handle it, if you’re not ready to deliver on the promise you just made in Recipe B: if a visitor clicks to learn more, they’ll see a message saying that the offering isn’t yet available. You can even give the visitor the opportunity to provide their email address, so you can notify them when (if) you make the offering available.
So now you know: the offer has some legs (four of them, like our furry friend?). You can proceed with developing the whole program. If Recipe B had been a flop, you’d have lost very little in the grand scheme of things.
Danger, Will Robinson …
There’s a caveat to testing like this. My writer pal Chuck Vadun says, rather bluntly, “It’s very likely you’ll piss people off.” This is because you’re putting an offer out there, which you have no ability to fulfill, right at the moment when someone wants to jump on it.
… Quid pro quo, Clarice
There are ways to mitigate this possible hazard, including a) testing it with a very small amount of your web traffic, and b) offering something of value to the visitor who shows interest — a discount code for one of your current products, for example.
Whatever you provide in return, it’s a small price to pay for information you can’t get any other way: in-market testing of interest in your offering. Focus groups and customer interviews may give you confidence in what you’re planning to offer, but there’s nothing like seeing how many people with vote their interest with a tap or a mouse-click.
Ideas by James Young. Structure by Chuck Vadun.