Forget Waiting for App Submission, Let’s Just Copy Facebook

Every day, twice a day (except on Tuesdays), Facebook quietly updates its website. One comes in the morning at 9:00AM and the other at 4:00PM Pacific Time. Each of those iterations can contain anywhere from 30 to 300 cherry picked changes, selected by Release Engineering Director Chuck Rossi. On Tuesdays, the team does a single shipment of a branch, a large release containing anywhere between 4,000-6,000 changes.

That’s an astounding number of iterations and a large number of changes that could go wrong each week. Yet while Facebook is always updating, iterating, testing, and improving, it’s proven itself as an incredibly stable platform. Their adaptation of the web release methodology in the mobile ecosystem has helped their applications reach the top of the app stores. Let’s take a look.

“How Can We Move Faster?”

Facebook’s Web Development Cycle

“Every day, we’re trying to answer the question, ‘How can we move faster?’” says Rossi. Since his beginnings as the lone release engineer in 2008, Rossi has been working hard to figure out how to live up to the Facebook motto of “move fast and break things,” with a little less emphasis on the breaking part. One of the big changes he made was to opt for schedule based releases.

Feature based shipments (releasing when features finish) can often lead to uncertain release dates, as well as a decrease in quality. With these schedules, developers often scramble to get all their features and updates in on time, even if they’re not 100% ready. The rush to get onboard with a release frequently results in buggy, unrefined code that can crash the site, and ruin the user experience.

On a schedule based release, there’s “always a next train.” “All that pressure, all that conflict, all that grinding of gears and rocks goes away when I say, ‘Relax, you can go out the next day, the next week. No big deal,’” assures Rossi. To complement this strategy, the release team at Facebook has stringent testing rules. If there’s any doubt about a change, it’s gone.

As a result, the developers are more focused on getting code right, rather than just getting it out.

This shift in development has led to two major benefits. The frequency and consistency of iterations allows Facebook to reduce developer conflict, as well as increase the quality of releases. They’ve invested heavily in this structure: In 2012, Facebook was updating web once a day. Now, that number has increased to twice a day. It’s been so successful that they’ve been adapting the methodology for mobile development.

Moving to Mobile

Facebook’s Mobile Development Cycle

Just like everyone else, Facebook has had its own frustrations with the app marketplace releases. However, they’ve been taking full advantage of the opportunities provided by the Android market.

“In mobile what do I do? I push a button, it goes to The App Store, the black hole that is Apple, and out comes, in some indeterminate amount of time, my binary. If I’ve made a mistake or if theres a fatal, or something silly in that app, its gone. That bullet has left the barrel, and I’m screwed.” – Chuck Rossi

Even with the challenges, Facebook is still doing a stellar job of applying its frequent iteration cycles to Android. As of now, they’ve been able to implement a 4 week development cycle very similar to what they do on web.

The big challenges of adapting this methodology were the difficulties of testing for stabilization, as well as the inability to revert changes if there are mistakes. So far, they’ve applied some good workarounds for the former.

A principal component of Facebook’s cycle is dogfooding its web releases. Without these checks, the entire process would fall apart.

With Android, Facebook is able to do dogfooding with auto-updating, forcing employees to run the latest tests. Scaling even larger, they use Android’s alpha and beta testing programs, pushing out their latest changes to millions of opt-in users each night, making the stabilization process a breeze.

Even with these techniques helping decrease development cycle times, Facebook is still bent on increasing iteration speed to a level comparable to their web development. “[We] want to ship both platforms every 2 weeks, eventually every week, shipping mobile…I can do that, but the tools at Apple and Google are right now one of my biggest hurdles to get that cadence.”

Facebook has shown us that there are huge advantages to iterating fast on both web and mobile. With the daily and weekly updates on the website, as well as their monthly mobile updates, Facebook is able to create a highly stable product while always improving.

Their system decreases conflict, increases quality, and ships consistently on time. Your development cycle should be the same. Take advantage of Facebook’s fast framework and Android’s alpha and beta testing models to take your product to the next level.

Facebook’s motto says “Move fast and break things.” We call it the fast mover advantage.

Find out more at www.apptimize.com/products