Feb 13, 2014
These days, it’s common to have a both a web and mobile experience that are core to your business. One probably reels in more revenue than the other, but if you’ve developed an experimental culture, there’s a good chance that you’re a/b testing and optimizing both. And you typically test more on web because it’s easier (no iTunes regulation, don’t need to wait for users to update, etc.). But you can’t just test on web and assume it applies to mobile. No, that would be too easy. Here are some DOs and DON’Ts for leveraging web a/b testing into your mobile testing strategy.
DO build a separate implementation strategy for mobile optimization. Mobile technology is very different from web technology in two important ways:
DON’T assume what works on web works on mobile. While it’s likely that your users are coming to both, there are many nuanced differences between the web user and the mobile user:
DO leverage learnings from web to create hypotheses for mobile. Since it is much easier to test on web, test on web more. Maybe you’ll find that out of 10 different images you tested on web, one type of image wins, two others are close, and seven perform terribly. Take the top three and retest them on mobile. The winner might be different, but you’ve reduced the number of versions you need to create for mobile.
DON’T dissociate mobile and web so much that users get conflicting messages. There’s a chance that someone seeing variant A on mobile might be seeing variant B on web. Make sure that you plan tests so that these version don’t conflict. There are some big changes that users might not notice but small changes that users might. For example, if you’re testing out a different check out flow on mobile than web, users mostly expect that web and mobile check out experiences are different. However, if you’re testing item pricing, customer might notice that a price they see on web is different from the same item on mobile.
DO utilize a mobile-first solution like Apptimize. We were born on mobile and know mobile A/B testing technology and philosophy better than anyone. Our framework not only lets you A/B test but also has other features that are vital to mobile product managers and developers such as feature flagging/toggling and slow rollout. Try it today or email us at firstname.lastname@example.org to learn more about enterprise solutions.
We’re thrilled to announce our partnership with Qubit! This exciting collaboration between two industry leaders will allow teams to personalize and optimize the customer experience across all platforms and combine data to gain deeper insights into how customers are interacting...Read More
I really didn’t need to know how well I knew Disney song lyrics… but I clicked on it last night anyway and spent 3 minutes realizing that there are Disney movies I’ve never seen. Then I shared it to all...Read More
I talk to a lot of app developers and product managers on a regular basis about A/B testing their mobile app. The feedback I hear the most is: “I know I need to start A/B testing, but we just haven’t...Read More