Docs

Dynamic Variable A/B Experiments

What are Apptimize Dynamic Variables?

An Apptimize dynamic variable is a programming variable that has been defined in your code, whose value can be changed from our server. At the time of definition, a dynamic variable is assigned a default value and a name, and its value can be changed via the Apptimize server by running experiments or instant updates.

Why are they important?

Apptimize Dynamic Variables give you the ability to make modifications to code variables on-the-fly so your users immediately reap the benefits of a change without the need for an App store release. Integrating dynamic variables empowers continuous improvement as they can be used across multiple experiments, without any need to redefine them.

What can a dynamic variable be?

Dynamic variables can be strings, integers, doubles, floats, booleans, arrays, and dictionaries (note for Dictionaries, the keys must be Strings). Their types are set when you define Dynamic Variables in your code. Protocol Ref, Tech note 1, Tech note 2, Tech note 3

Why would I use dynamic variables instead of a code block?

Dynamic variables simplify the coding. Implementing code blocks requires a separate explicitly named section of executing code for each variant, and each experiment. Dynamic variables allow you to declare a variable with a single code snippet, and easily integrate it with the rest of your code. The next time Apptimize pairs with your device, the new variable will appear so that you can use it and modify it in any future experiment.

Example

We’ll walk through a dynamic variable A/B experiment example that tests the number of catalog categories to show a user. Our example app, Urban Attic, is a retail commerce app that lets user log in, browse catalog, and make purchases. In our experiment, we’ll change the number of categories the user sees on opening the app from the original value of 3, to showing 5 or 8 categories. Let’s get started!

NOTE: If your app is written in swift please refer to this FAQ for more information about the code snippets since this example is in Objective C.

Here is the original variant in the app, 3 categories, followed by the variants with 5 and 8 categories:
Screenshot showing original variant: 3 categories shown  Screenshot showing variant 1: 5 categories shown  Screenshot showing variant 2: 8 categories shown

  1. First, let’s setup the dynamic variable in the app code for this experiment. The dynamic variable(s) will need to be tagged in your app code the first time you use them to let Apptimize know what variable can be changed. In this example we will set an integer variable with name ‘categoryCount’ to a default value of 3, and add in code logic to modify the layout when other values (e.g. 5 or 8) are specified for the category count.

    1. First import Apptimize into the class that’s going to use it.

      #import <Apptimize/Apptimize.h>
      
    2. Then declare a dynamic integer variable outside the method to be used for number of categories to show and set its default value.

      ApptimizeInt(categoryCount, 3);
      
    3. Finally use the variable in the code to set the number of categories to be displayed.

      NSInteger categoryCountInteger = [categoryCount integerValue];
      

    After the app is run for the first time with the Apptimize dashboard open, the new Dynamic Variable becomes visible in your Dynamic Variable list for the App via websocket pairing. If your dynamic variable doesn’t appear in the Apptimize dashboard, see our FAQ answer for help.

  2. Now in the Apptimize Dashboard you are ready to create a new experiment. You can do this by clicking on the Create button on the top right of the dashboard list, and select the A/B Experiment project type.

  3. In Step 1 Details, the default experiment name will be “New Experiment 1,” but you can change it to something more meaningful. We’ll call our example experiment “Category Count”, and can enter notes or tags to help describe the experiment for other users.

  4. Then, we’ll continue to Step 2 Goals and setup our primary goal. In this case, our goal is to maximize purchases by presenting users with the optimal number of item categories. We may set the primary goal as my app’s purchasing funnel tracking the events from ‘Select Category’ to ‘View Item’ to ‘Add Item to Bag’ to ‘Checkout’ to ‘Finish Payment’.

  5. Continue to Step 3 Configuration. Under “Experiment type”, select Dynamic Variable. You will now be asked to select the dynamic variable you want for the experiment. For this example we see our variable, categoryCount, and will select the checkmark next to it and click Select Variables.
    Screenshot showing Configuration step for dynamic variable selection

  6. Now, setup the variant and variant values for the experiment. By default, your experiment will have one variant called “original” which is your current app and will be displayed to users not participating in the experiment. In this case, the default version of our app shows only 3 categories on the first screen. Note: the dashboard does not display the original value since the value is set in code and not pushed from Apptimize servers. For each additional variant you wish to add, click the Add Variant button, and then name the resulting variant. In this example, we name the first variant “5 categories” and the second variant “8 categories”. In the variable table, we enter in the value 5 and 8 for each variant. You can add screenshots to each variant to remind yourself of the experiment change you are testing.
    Screenshot showing Configuration step for dynamic variable categoryCount example with values set for variant 1 as 5 and variant 2 as 8

  7. Click Continue to Targeting at the bottom or Step 4 Targeting in the upper sub-navigation to setup who will see the experiment. In this case, we set the allocation to 10% of our users exclusively, and add no targeting filters.

  8. Next, we’ll continue to Step 5 Launchpad, which shows a summary of our dynamic variable experiment. We can see the variant name, allocation percent (10%), our targets (no targets), and have the option to preview variants with the Preview Variants tool. When we’re ready, we can start the experiment by clicking the Launch Experiment button at the bottom, which will make the experiment move from “draft” to “running” state, and end users will see the experiment on their devices within about 15 mins.

    Screenshot showing Launchpad, summarizing the experiment setup and showing the Launch Experiment button

  9. You can find more information about stopping the experiment, setting a winner and interpreting reading results in 6. Results.

Appendix: Protocol Reference

Variable “name” will be the name that appears in your dynamic variable list, Variable “value” needs to be in a objective-c literal syntax and is the value you want the variable to be when Apptimize is not running. It will also be the original baseline value in tests. Note, for Dictionary type variables only key/value pairs entered on the web dashboard where the value is not blank will be passed down as variants, similarly for arrays.

The full API is:

#define ApptimizeString(name, value) ...
#define ApptimizeInt(name, value) ...
#define ApptimizeDouble(name, value) ... 
#define ApptimizeBoolean(name, value) ... 

#define ApptimizeArrayOfStrings(name, value) ...
#define ApptimizeDictionaryOfStrings(name, value) ...
#define ApptimizeArrayOfInts(name, value) ...
#define ApptimizeDictionaryOfInts(name, value) ...
#define ApptimizeArrayOfDoubles(name, value) ...
#define ApptimizeDictionaryOfDoubles(name, value) ...
#define ApptimizeArrayOfBooleans(name, value) ... 
#define ApptimizeDictionaryOfBooleans(name, value) ...

More Example Dynamic Variable Declarations

Initialize variable name and original values outside of any class or method (ie. above viewDidLoad)

ApptimizeString(myString, @"original string");
ApptimizeInt(myInteger, 2014);
ApptimizeDouble(myDouble, 3.14);
ApptimizeBoolean(myBool, YES);
ApptimizeArrayOfInts(myIntArray, (@[@(-1), @(0), @(1), @(3), @(10), @(20)]));
ApptimizeDictionaryOfStrings(myStringDictionary, (@{@"user_name":@"Apptimize", @"subscription_level":@"enterprise"}));

Example of code to get the values of dynamic variables by calling typeValue, this will add log messages displaying the current values of the variables.

NSLog(@"my string value is: %@", [myString stringValue] );
NSLog(@"my boolean value is: %@", [myBool boolValue]? @"yes":@"no");
NSLog(@"my array values are: %@", [myIntArray arrayValue]);
NSLog(@"my dictionary values are: %@", [myBoolDictionary dictionaryValue] );

Appendix: String-Value Deprecations

Since Dynamic Variables replaces the functionality of our previous String/Integer/Double value tests, we will support for a limited times these code snippets. Currently running value tests and value tests as drafts will continue to work. Currently coded value test will show up as Dynamic Variables in your list and can be used for dynamic variables, but please update them at your earliest convenience. Please contact us if you would like to continue using value tests. The code snippets are:

NSString *testValue1 = [Apptimize stringForTest:@"test name" defaultString:@"original_value"];
NSInteger testValue2 = [Apptimize integerForTest:@"test name" defaultInteger:1];
double testValue3 = [Apptimize doubleForTest:@"testname" defaultDouble:3.14];

Tech note 1. Apptimize dynamic variables dictionaries and arrays need to contain items of the same type.

Tech note 2. If you need to access the variable across many parts of your code then you will need to make sure it is defined to be available across the full scope you need. In code block experiments the unique naming of experiment/variant allows Apptimize to provide global scope.

Tech note 3. String variable variants specified through the web interface use the same escaping as JSON strings. This means you can use any unicode character, with special treatment for the control characters backslash (\) and double quote (“). Backslashes are used to specify string formatting: for example, \n begins a new line. To specify a literal backslash, use the escape sequence \\. Double quotes must be escaped as \” to appear in string variables. For convenience, the web interface automatically escapes unescaped double quotes when saving the variant: so ” becomes \”.