Key Takeaways From Mobilize 2017

In October, Apptimize hosted the second annual Mobilize conference, where over 250 product leaders joined us for a day of presentations, workshops, panels, and networking. Here are some takeaways from presentations throughout the day:

Products that Count: SC Moatti, founder of Products that Count, kicked the day off with a presentation about building and growing great mobile products. She started by discussing the importance of understanding your customer’s journey to find a product/market fit. This is an evolving challenge, as the more you grow, the more users you have with unique needs, and the more this fit will change.

She also covered basic business principles and the importance of picking a growth priority and focusing it. For example, instead of tackling growth plans for acquisition, engagement, and monetization at once, it is often better as an organization to focus on one metric at a time and effectively grow your pipeline.

The key to continuous growth is using effective measurement to glean insights. Apptimize and partners allow you to understand your customers’ journey from beginning to end. Once you solve this, she says, “what you can do is actually pretty amazing.”

Strava: Evelyn Cordner, Lead Engineer and Growth Manager at Strava, presented on how her organization increased their testing frequency per quarter by 10X, allowing them to make impactful, data-driven decisions.

Building a testing culture required a company shift towards experimentation, the addition of new tools to their teams, and establishing processes to ensure that they could monitor testing. For details about how they addressed each of these, read more here.

HotelTonight: Doug Suriano, Lead iOS Engineer at HotelTonight, discussed how moving fast enables HotelTonight to ship a better mobile product. The key issue that initially kept HotelTonight from moving quickly and executing on their roadmap was their existing release cycles. They had 3 main issues:

1. Releases were massive
2. Releases were stressful
3. Releases were limiting

… making it difficult to launch experiments. The team established clear goals:

To accomplish these goals, HotelTonight adjusted their release cycle to release a new version every two weeks. They also developed a new system for scoping work to allow them to focus on what they needed to do to have the biggest impact on customers.

The result? They shipped more versions and more features than they had previously been able to. They also used A/B testing, specifically Apptimize’s Dynamic Variables, throughout this process to learn what worked and what didn’t. As Doug noted,

Delivery Hero: Erin McLain, Global Head of Conversion Optimization at Delivery Hero discussed how testing has helped to accelerate in-app conversions and monetization. She notes that before seeing results, it’s important to establish a data-driven culture from the top down:

“Business leaders need to challenge product owners to validate their decisions.”

Also, teams need to focus on turning problems into opportunities. At Delivery Hero, they use funnel visualization to identify weaknesses in their product. They map out the experience that their customers have and use data to understand where they lose customers.

As Erin noted, “in order for any data-driven or experimentation company to be successful, it’s all about validation, iteration, and growth.” So, test!

Runtastic: We also heard from Benedict Bleimschein, Product Manager at Runtastic about how the company utilizes a diverse portfolio to personalize communication. Benedict describes that personalization is crucial for their strategy, as the company has several apps and offerings. To ensure healthy growth and learnings, they needed to have a clear understanding of user preferences to be able to serve them individualized content.

Their messaging is based on several factors, including the app that the user was on, the user status, the frequency at which the user uses the app, and their location. By understanding these segments, they are able to delight their users with extremely customized experiences.

Benedict noted that extensive testing that they used to understand how they could increase their premium sales resulted in a 40% uplift in premium sales!

Hotels.com: Michael DeHart, Senior Mobile Product Marketing Manager, and Jonny Roberts from the Product Analytics Experimentation team at Hotels.com covered how testing works at their organization. For Hotels.com, testing is nothing new, as they have been testing for 10 years and now run 500 tests per year. What is new for them is mobile app testing, which only started two years ago.

Testing is so successful and embedded at Hotels.com because they have a robust set of tools, teams that are focused on testing, and a culture built around testing.

The Hotels.com team covered the struggle that many people feel: getting buy-in. They cited these reasons about why it’s important to test:

• Manage Risk
• Return on Investment
• Competition
• Prioritization
• Knowledge Building

Three key learnings for impactful tests that Michael and Jonny called out are:

1. Have excellent data quality throughout the test
2. Don’t just try to understand the control and the variant, but understand the customer
3. Allow for wiggle room so you don’t just have iterations, but you also have big ideas

Jonny and Mike said the magic words that they operate by are: “We really are a test everything culture”

Busuu: Antoine Sakho, Head of Product at Busuu, focused his discussion on subscription-based apps. He gave an in-depth look into Busuu’s Subscription Optimization strategy and provided a firsthand look at their revenue equation and experimentation tactics that can potentially make an immediate impact to your mobile subscription revenue.

Antoine pointed out that when you reach a similar purchase conversion rate level as in-app purchases, you can actually achieve much higher, compounding revenue with subscriptions due to their recurring nature He noted that:

“Looking back, it was a painful switch from in-app purchases to subscriptions – but absolutely worth it in the long run. “

Busuu approached the problem scientifically by thinking of subscription revenue as the following equation:

Subscription Revenue = Number of Active Subscribers x Average Lifetime Value of Subscriber

Wall Street Journal: Peter Gray, Director of the Product Optimization team at Dow Jones, the holding company for the Wall Street Journal, discussed his team’s responsibility for digital experience testing across all of Dow Jones’ digital properties.

Peter explained the pains, processes, and data necessary for building a data-driven team at a legacy company. He defined experimentation as more than just simple marketing driven A/B testing, QAing, or the overused term “optimization”– it is instead a cultural change and practice necessary for growth and sustainability and should be described as such to gain buy-in and respect from the wider organization. Peter went on to define the role,

“We are persuasian scientists and we are the practitioners of the most powerful product development discipline on earth.”

So how did WSJ get to where it istoday?

Peter outlined a thorough five step recipe to get your company on the path to making better data-driven decisions, gaining executive buy-in, building a team, and proving the worth of experimentation.

The 5 Step Recipe to becoming more Data Driven

• A Great Partner Agency
• Evangelism
• Velocity
• Ruthless Prioritization
• Prove Value of Your Work

Peter covered some driving principles for experimentation:

Microsoft: Cindy Alvarez, Principal Design Researcher at Microsoft, ended the conference with an inspirational presentation about how to start taking the next steps on building your product. She presented question templates for potential users that you can use to help drive your product development process.

One key is understanding why customers want something and how it will fit into their world. This helps you understand if your customer request is actually a good solution– you might be surprised at how often it’s not!

When trying to get user feedback, make sure that you ask the right questions. Don’t just get their impression of your demo, but ask them to walk you through how they would do it. You’ll get a more useful story here for your roadmap.

As Cindy noted, “customers don’t know where the future will take them. That’s our job- we have to get them to tell us their problems, how they made decisions, what frustrates them and take that information to make great products.”

Watch our 2017 Mobilize highlight reel!