How We Pulled Off Scaling a Live Demo in a Week

As many of you already know, Apptimize does A/B testing for native mobile apps. Since we launched our Visual Apptimizer (our what-you-see-is-what-you-get visual editor) in January, the most common questions we’ve gotten are “what’s the catch? Does this only work on HTML5 apps? Do I have to build my app around a special framework, tag all my views, or otherwise change how I build my app?” People are especially skeptical when we say they can make changes to their app (including hotfixes) without any code–even in the install.

So last week we started brainstorming ways to prove to people just how easy it is. One of the more crazy ideas we came up with was a demo where you could type in any app name and we’d let them modify it with our Visual Apptimizer. No effort on your part required. It sounded awesome but who would build such a thing? Our team is moving full-steam-ahead getting stuff like the Visual Apptimizer for Android out to clients.

We called up our friends at Bitmatica and pitched them the project. Their initial reaction was a mix of excitement and terror. But as we talked through the details it was clear that we could make this happen. At the end of the meeting, Matt told us, “this actually doesn’t sound completely insane anymore!”

A few days later, they had a demo hooked up to a phone on Cody’s coffee table. Another day later, a co-location spot was partially set up with a drawer full of iPod touches. Another day and we had a sneak peak ready to share with trusted friends. Things were slow, there were lots of known bugs, and most of our friends didn’t quite know what they were looking at or supposed to do. (If you ever have done user testing you’ll know the agony.) But the feedback we got was invaluable.

And that’s why we’re launching our demo to the world now–before it’s completely done cooking. We’d love to have you check it out and share any feedback you might have with us while we continue making it a faster, more glitch-free, awesome experience.

Here’s what it looks like. Just type in the name of any app to get started.

When you hit return, you’ll be connected to one of about 30 iPods which will let you download your app from the App Store. Once it’s downloaded, we automatically update the IPA to contain the Apptimize framework and add a temporary app key to your plist. In short, we do the simple 0 line install for you. Within seconds, you’ll be playing around with the same Apptimize editor our customers use to make A/B tests and push hotfixes.

With the tireless efforts of Matt Hubert and Cody Walker at Bitmatica, we pulled this off in 7 days. We’ve still got a ways to go on the demo, but we’re ready for you to see for yourself how you can make A/B testing variants while bypassing the App Store!

For those who are curious, here are some behind the scenes photos: