The A/B and See of Testing: Do It Early and Often - Lift361

Our Thinking

The A/B and See of Testing: Do It Early and Often

By Ed Higdon. Posted in Insight

July 03, 2018

Marketers understand the importance of A/B testing. However, many marketers get so hung up on designing the “perfect” test that they end up delaying and leaving money on the table.

A “perfect” test can take months to design and roll out. However, market conditions shift on a dime in our hyper-connected world and the test you were setting up could be completely irrelevant to your customers by the time it kicks off. Moreover, every week that ticks by is a week you can’t adjust your approach to meet the needs of your evolving customer base.

If you need information – and customers – quickly, a minimum viable test can start generating data that you can act on today as opposed to waiting months. For example, rather than establishing a test with 8 variables that takes 6 months to set up, perhaps testing only a few of those elements would still provide the insight you’re looking for while taking only a fraction of the time to set up.

Taking a test-and-learn approach allows for flexibility and a nimbleness that is impossible to achieve if you’re waiting on the “perfect” time to start a test.

Viable Test, Profitable Results

Has anyone ever really succeeded using only a minimum viable test? Absolutely. Savvy retailers and brands do it every single day. They know that waiting is a surefire path to missed opportunity. Take, for example, a Lift361 client that was struggling to acquire new customers. In today’s market, retail organizations aren’t investing in nearly as many new stores as they have in the past. As their existing stores age, it becomes increasingly difficult to attract new customers. Someone who spent years bypassing a store needs a highly compelling reason to disrupt their usual patterns and start shopping somewhere new. Yet, new customers are critical for retail survival.

Christmas was rapidly approaching when this retailer decided they needed to act. Had they been married to traditional A/B testing methodologies, nothing would have been ready by the time the holiday season arrived – it can take up to six to eight weeks for a new customer to be rolled into marketing campaigns. However, the retailer was open to a test-and-learn approach, and the Lift361 team was able to immediately put together a test for new customers.

The test consisted of two groups. The “A” group received a single touch during the testing period and the “B” group received three touches. The three-touch group not only outperformed the single-touch group, but they also outspent existing customers who received the same three touches.

Being open to launching a test that may not have been perfected allowed this retailer to build real relationships with their new customers and to see results in their bottom line that would have never existed if they had waited.

Success Is About Acting Now

Retailers afraid of a test-and-learn approach need only look to Amazon to see how it can make a difference. Amazon consistently rolls out products and services that could technically be classified as a “minimum viable product.” When Amazon Echo with Alexa technology first launched, it did not have nearly the same capabilities it has today. However, it had enough capabilities to be enticing for customers and Amazon used feedback to help refine Alexa – and they still use that feedback to refine it today.

The options are simple: You can either launch a minimum viable test quickly and start collecting actionable data that can help you build relationships and boost your bottom line, or you can wait months to launch and potentially miss out on a profitable opportunity. Success is always about seizing opportunity. How many customers could you reach today if you took action?