Docs

Programmatic Code Block A/B Experiments

The Apptimize iOS Code Block interface is a flexible, code-based to A/B test anything you can imagine. If you can code it, you can A/B test it. The behavior for each variant is pre-defined in your app’s code. Code blocks require an explicitly named section of code for each variant and each experiment.

We’ll walk through a code block experiment example of testing a new login user flow by adding a variant for a guest flow option. Our example app, Urban Attic, is a retail commerce app that lets user log in, browse catalog, and make purchases. In the original flow, the user logs in and enters in information about themselves to make browsing and purchasing experience personalized. In our experiment, we’ll make a new variant that add a button on the login flow to sign in as guest, and then later in purchase flow asks user to login and personal information before paying. Let’s get started!

Here is the original variant in the app: 1) login page, 2) account registration, 3) checkout from bag, 4) payment page.
Screenshot showing original variant: image 1 of 4, login screen  Screenshot showing original variant: image 2 of 4, registration  Screenshot showing original variant: image 3 of 4, checkout  Screenshot showing original variant: image 4 of 4, payment

Here is the new variant “Added Guest Flow” that users allocated to variant will see: 1) login page with guest option, 2) checkout from bag, 3) account registration, 4) payment page.
Screenshot showing guest flow variant: image 1 of 4, login screen  Screenshot showing guest flow variant: image 2 of 4, registration  Screenshot showing guest flow variant: image 3 of 4, checkout  Screenshot showing guest flow variant: image 4 of 4, payment

  1. Create a new experiment. You can do this by clicking on the Create button on the top right of the dashboard list.

  2. In Step 1 Details, the default experiment name will be “New Experiment 1,” but you can change it to something more meaningful. We’ll call our example experiment “Add GuestFlow”, and can enter notes or tags to help describe the experiment for other users.

  3. Then we will continue to Step 2 Goals and setup our primary goal, which in this case may be a funnel of users who click “Checkout” and click “Pay” to finish purchase.

  4. Continue to Step 3 Configuration. For Experiment type, select Code Block. By default, your experiment will have one variant called original which is your current app and will be displayed to users not participating in the experiment. For each additional variant you wish to add, click the Add Variant button, and then name the resulting variant. In this example, we name the new variant “Guest Flow”. You can also add screenshots to each variant to remind yourself of experiment change you are testing.

    Programmatic Code Block Experiment example of configure page

  5. After you have added a variant, you will see a code snippet at the bottom of the page (select the Swift tab if you are writing in swift and refer to this FAQ). Notice that the code snippet references a code block variable, which must match in your code, and allows the code block to be reused in future experiments. In this case, we set the code block variable to the same name as the experiment, “addGuestFlow”. Paste the snippet into the place in your app’s code where you wish to run the experiment, and then insert code for each experiment in the designated areas. To run the experiment, we wrap the original code hook around the existing code, and add a guest button on login and change the layout order for the new variant we created.

    - (void)viewDidLoad
    {
        [super viewDidLoad];
    
        [Apptimize runTest;@"Add GuestFlow" withBaseline:^{
            //Baseline variant "original"
            _loginWithGuestButton.hidden    = NO;
            useGuestFlow.hidden     = NO;
        } andVariations:@{@"variation1": ^{
            // Variant "Guest Flow"
            _loginWithGuestButton.hidden    = YES;
            useGuestFlow.hidden     = YES;
        }}];
    

    Note that you can use the same code block variable multiple times throughout your app. Apptimize will ensure the user is consistently assigned to the same variant each time the same code block variable is referenced.

  6. Click Continue to Targeting at the bottom or Step 4 Targeting in the upper sub-navigation to setup who will see the experiment, as described here.. In this case, we set the allocation to 10% of our users exclusively. Then add a filter so only new users will see this experiment, since current users have already had to login and cannot participate in the experiment.

  7. Next, we’ll continue to Step 5 Launchpad that shows a summary of our code block experiment. We can see the variant name, allocation percent (10%), our target (“user is new”), and have the option to preview variants the preview variants tool. When we ready, we can start the experiment by clicking the “Launch Experiment” button at the bottom, which will make the experiment move from “draft” to “running” state, and end users will see the experiment on their devices within about 15 mins.

    Programmatic Code Block A/B Experiment Example of the launchpad screen

You can find more information about stopping the experiment, setting a winner and interpreting reading results in A/B Experiment Results.

You can also find documentation on our classes and methods in our CocoaDocs.

Join the Best-in-Class Mobile Growth Platform