Optimize Marketing Campaigns in Real-time for Native iOS and Android

A/B Testing Example – Retale, a location-based app that aggregates retail circulars, ran a Black Friday campaign in the weeks leading up to November 28, 2014 that increased their product click-through rates in the busiest shopping week of the year. While time constraints prevent most apps from testing holiday promotions in a data-rigorous manner, Retale was able to create a few versions of their campaign, test them against each other at the start of the promotional period, and roll out the winning campaign to all users in real-time, therefore maximizing the benefit from the holiday.

The App

Retale is a market leader in bringing local deals to consumers by delivering digital circulars from major retailers. Price savvy shoppers can easily and quickly browse through the latest circulars to find active deals in their area. Retale is part of the Bonial International Group product line, which focuses on location-based services for retailers and consumers with its companies in the US, Germany, France, Spain, Brazil and Russia.

The Problem

Retale sought to run a preview of Black Friday promotions in the days leading up to the unofficial holiday as a way to excite customers about the offers that Retale’s partners were running. They had several challenges when designing this special promotion:

• The team was not certain which visual layout would get the most conversions for the Black Friday campaign.
• Timing of the campaign needed to be carefully controlled to match the dates of the flyer previews.
• Data also had to be collected so that the team could know the impact of their efforts and know which version of the layout would be an early winner and be shown to all users.

The Experiment

The Retale team created the following versions of the Black Friday preview.

The experiment not only compared the the Black Friday layouts against each other but also tested them against the original layout of the promotional flyers:

Retale designed this experiment using Apptimize’s Dynamic Variables which allows the app to set a “variable” that could be changed at a later time using Apptimize–in this case the variable pointed to the designated flyer thumbnails. Using this variable, Retale could set any number of new layouts for future campaigns in real-time without having to change the code in their app and therefore without having to redeploy their app through the App or Play Store.

The Results

Retale started the experiment between three and four weeks before Black Friday. Within ten days, they saw a statistically significant lift for every Black Friday layout when compared to the original. Within the Black Friday variants, the complete gray overlay (Variant B) created the best result by increasing click-throughs on the flyer by 3.2%.

This demonstrable increase in visibility for retailers, at a time when the market was most crowded and competition at its annual peak, formed a key part of a year-end effort to expand their customer base of major American retailers.