Dynamic Variable A/B Experiments¶
What are Apptimize Dynamic Variables?¶
An Apptimize dynamic variable is a programming variable that has been defined in your code or the Apptimize Dashboard, whose value can be changed from the Apptimize Dashboard. At the time of definition, a dynamic variable is assigned a default value and a name, and its value can be changed at any point later via the Apptimize Dashboard as long as your code knows how to handle new values.
Why are they important?¶
Apptimize Dynamic Variables give you the ability to make modifications to code variables on-the-fly so your users immediately reap the benefits of a change without the need for an App store release. Integrating dynamic variables empowers continuous improvement as they can be used across multiple experiments, without any need to redefine them.
What can a dynamic variable be?¶
Dynamic Variables can be strings, integers, doubles, floats, booleans, arrays, and dictionaries (note for Dictionaries, the keys must be Strings). Their types are set when you define Dynamic Variables in your code. 1 2
Why would I use dynamic variables instead of a code block?¶
Dynamic Variables can simplify coding for certain types of experiments. Implementing code blocks requires a separate explicitly named section of executing code for each variant, and each experiment. Dynamic variables allow you to declare a variable with a single code snippet, and easily integrate it with the rest of your code. The next time Apptimize pairs with your device, the new variable will appear so that you can use it and modify it in any future experiment.
Dynamic Variables Across Platforms¶
Dynamic Variables defined in the Apptimize Dashboard are platform agnostic. That means that a single Dynamic Variable can be used to define an experiment that ensures a consistent experience across platforms.
Example¶
We’ll walk through a dynamic variable A/B experiment example that modifies the number of catalog categories to show a user. Our example app, Urban Attic, is a retail commerce app that lets user log in, browse the catalog, and make purchases. In our experiment, we’ll change the number of categories the user sees on opening the app from the original value of 3, to showing 5 or 8 categories. Let’s get started!
Here is the original variant in the app, 3 categories, followed by the variants with 5 and 8 categories:
1. First, let’s setup the dynamic variable in the app code for this experiment. The dynamic variable(s) will need to be declared during app startup to let Apptimize know what variable(s) can be changed. In this example we will set an integer variable with name ‘categoryCount’ to a default value of 3, and add in code logic to modify the layout when other values (e.g. 5 or 8) are specified for the category count.
a. First import Apptimize and ApptimizeVariable into the class that’s going to use it.
import com.apptimize.Apptimize;
import com.apptimize.ApptimizeVar;
#import <Apptimize/Apptimize.h>
#import <Apptimize/Apptimize+Variables.h>
import Apptimize
// Please see Installation instructions for JavaScript on steps for set-up
// Please see Installation instructions for Android on steps for set-up
// Please see Installation instructions for Node.js on steps for set-up
# Please see Installation instructions for Python 3 on steps for set-up
# Please see REST API documentation for setup
' Please see Roku SDK documentation for setup
b. Then, in AppDelegate’s didFinishLaunchingWithOptions method
(iOS) or in your MainActivity class (Android), declare a dynamic integer
variable to be used for number of categories to show and set its default value.
Note that in JavaScript, it is not necessary to declare the variable in the code; instead, you
must define the variable and its type in the Dashboard.
private static ApptimizeVar<Integer> categoryCount = ApptimizeVar.createInteger("categoryCount", 3);
[ApptimizeVariable integerWithName:@"categoryCount" andDefaultInteger:3];
ApptimizeVariable.makeInteger(name: "categoryCount", defaultInteger: 3)
// Please define your Dynamic Variable and its type on the Apptimize Dashboard
// In this example, we define a variable called "categoryCount" of type 'Int' on the Dashboard
// Please define your Dynamic Variable and its type on the Apptimize Dashboard
// In this example, we define a variable called "categoryCount" of type 'Int' on the Dashboard
// Please define your Dynamic Variable and its type on the Apptimize Dashboard
// In this example, we define a variable called "categoryCount" of type 'Int' on the Dashboard
# Please define your Dynamic Variable and its type on the Apptimize Dashboard
# In this example, we define a variable called "categoryCount" of type 'Int' on the Dashboard
# Please define your Dynamic Variable and its type on the Apptimize Dashboard
# In this example, we define a variable called "categoryCount" of type 'Int' on the Dashboard
' Please define your Dynamic Variable and its type on the Apptimize Dashboard
' In this example, we define a variable called "categoryCount" of type 'Int' on the Dashboard
c. Finally, retrieve the dynamic variable and query its value in the code to set the number of categories to be displayed.
Integer categoryCountInteger = categoryCount.value();
ApptimizeVariableInteger *categoryCount = [ApptimizeVariable getIntegerForName:@"categoryCount"];
NSInteger categoryCountInteger = [categoryCount integerValue];
let categoryCount = ApptimizeVariable.getInteger(name: "categoryCount")
let categoryCountInteger = categoryCount?.integerValue
// In this example, we set the default value of categoryCount to 3
var variantInt = Apptimize.getInt("categoryCount", 3);
// In this example, we set the default value of categoryCount to 3
String userId = "1234abcd";
int variantInt = Apptimize.getInt("categoryCount", 3, userId);
// In this example, we set the default value of categoryCount to 3
var userId = "1234abcd";
var variantInt = Apptimize.getInt("categoryCount", 3, userId);
# In this example, we set the default value of categoryCount to 3
userId = "1234abcd"
variantInt = Apptimize.getInt("categoryCount", 3, userId);
curl -X GET \
-H "ApptimizeApiToken: <ApiToken>" \
-G --data-urlencode "defaultValue=3" \
https://api.apptimize.com/v1/users/1234abcd/dynamic-variables/int/categoryCount
userId = "1234abcd"
m.requestId = m.apptimize.callFunc("getInt", {userid: userId, name: "categoryCount", defaultValue: 3, customAttributes: {}})
' Requires setup of setcontent callback as described in Roku installation steps
function setcontent(message as object)
apptimizeResponse = message.getData()
if (apptimizeResponse.id = m.requestId) then
?"Dynamic Variable: ";apptimizeResponse.response.value
end if
end function
In mobile, after the app is run for the first time with the Apptimize dashboard open, the new Dynamic Variable is detected and becomes visible in your Dynamic Variable list for the App via WebSocket pairing. Alternatively, you can add the variable name and type manually via the Dashboard. If your dynamic variable doesn’t appear in the Apptimize dashboard, see our FAQ answer for help.
2. In the Apptimize Dashboard, you are now ready to create a new experiment. You can do this by clicking on the Create button on the top right of the dashboard list, and select the A/B Experiment project type.
3. In Step 1 Details, the default experiment name will be “New Experiment 1,” but you can change it to something more meaningful. We’ll call our example experiment “Category Count,” and can enter notes or tags to help describe the experiment for other users.
4. Next, we’ll continue to Step 2 Goals and set our primary goal. In this case, our goal is to maximize purchases by presenting users with the optimal number of item categories. We may set the primary goal as a purchasing funnel tracking the events from “Select Category” to “View Item” to “Add Item to Bag” to “Checkout” to “Finish Payment”.
5. Continue to Step 3 Configure. Under “Experiment type,” select Dynamic Variables. You will now be asked to select the dynamic variable you want for the experiment. For this example, we see our variable, categoryCount, and will select the checkmark next to it and click Select Variables.
6. Now, setup the variant and variant values for the experiment. By default, your experiment will have one variant called “original” which shows the dynamic variable’s default value and will be displayed to users not participating in the experiment. In this case, the default version of our app shows only 3 categories on the first screen. Note: the dashboard does not display the original value since the value is set in code and not pushed from Apptimize servers. For each additional variant you wish to add, click the Add Variant button, and then name the resulting variant. In this example, we name the first variant “5 categories” and the second variant “8 categories”. In the variable table, we enter in the values 5 and 8 for each variant. You can add screenshots to each variant to remind yourself of the experiment change you are testing.
7. Click Continue to Target & Launch at the bottom or Step 4 Allocate & Target in the upper sub-navigation to setup who will see the experiment. In this case, we set the allocation to 10% of our users exclusively, and add no targeting filters.
8. Next, we’ll preview and confirm the criteria of our dynamic variable experiment. We can see the variant name, allocation percent (10%), our targets (no targets), and have the option to preview variants with the Preview Variants tool or set up early access and QA using Preview Groups tool in Step 5 Preview & Launch. When we’re ready, we can start the experiment by clicking the Launch Experiment button at the bottom, which will make the experiment move from “draft” to “running” state, and end users will see the experiment on their devices within about 15 minutes.
9. You can find more information about stopping the experiment, setting a winner and interpreting/reading results in A/B Experiment Results.
Manually Defining Dynamic Variables¶
For web or server-side experiments, dynamic variables must be manually defined in the Apptimize dashboard since they will not be automatically imported. The steps below will guide you through how to manually set up a Dynamic Variable for use in Apptimize Cross-Platform.
1. Define your Dynamic Variable(s) in the Apptimize dashboard by navigating to Step 3 (Configure) of your Experiment Setup.
Clicking on “Define A New Variable” will allow you to give a name and type for your variable(s).
After creating your variable(s), confirm the selection of the variable(s) that will be used in your experiment. You will then be able to modify the values for your variable(s) across your variants, similar to below:
2. The next step would be to retrieve the dynamic variable and query its value where needed across your various platforms (mobile, server-side, web).
Note that the value for the unique variable name that you define will be the same across all platforms where it is queried (for instance, users bucketed to a variant will see the same treatment on mobile and web.)
Dynamic Variables across your experiments can be created or hidden at any point from Manage -> Dynamic Variables in the top-right menu:
- 1
Apptimize dynamic variable dictionaries and arrays need to contain items of the same type.
- 2
String variable variants specified through the web interface use the same escaping as JSON strings except that single and double quotes must be entered as is, without a backslash. You can use any Unicode character, and you can specify white space, e.g. \r\n\t.