Codeless Visual A/B Experiments

Setting up your first experiment using the Visual Editor is incredibly easy. There is no programming needed after Apptimize installation, and the experiment can be launched live to your users within a few minutes. Please note that Visual Experiments only work with Android and iOS implementations.

The best way to show you how the Visual Editor works is to use an example that shows changing the Register call to action button. Our example app, Urban Attic, is a retail commerce app that lets user log in, browse catalog, and make purchases. In our experiment, we’ll change the text on the Register button from “Register” to “Join Now!”, and make a third variant that also has a button color change. Let’s get started!

Here is the original variant in the app followed by each variant with the Registration button changed: 1) original “Register”, 2) blue button “Join Now!”, 3) pink button “Join Now!”.

1. Create a new experiment. You can do this by clicking on the Create button on the top right of the dashboard list.

2. In Step 1 Details, the default experiment name will be “New Experiment 1”, but you can change it to something more meaningful. We’ll call our example experiment “Register CtA”, and can enter notes or tags to help describe the experiment for other users.

3. Then we will continue to Step 2 Goals and setup our primary goal, which in this case may the event of tapping the Register button.

4. Continue to Step 3 Configuration. Under “Experiment type”, select Visual – it should be selected by default. The experiment will start with one variant called original which is your current app and will be displayed to users not participating in the experiment. In this case, the default version of our app the Register button has the text “Register”. For each additional variant you wish to add, click the Add Variant button. In this example, we name the first variant “Join Now”. A screenshot of the variant is automatically generated when you set up the visual variant, and you can also add screenshots to the original variant to remind yourself of experiment change you are testing.

We have the Apptimize SDK already installed in my app and have the app open on my device, so after clicking Add Variant, the app shows up right away in the visual editor.

Note that you can use a physical device or a simulator for creating a variant, but your app must be running on the device or simulator with the Apptimize SDK installed. If you are not seeing your app, check out our FAQ answer about pairing your device with our dashboard.

5. As we move my mouse around over the app, notice that different elements of the app are highlighted. We want to change the “Register” button first so click on that button in the mirror image.

Now, Let’s change the Register button text. On the right-side panel, we can scroll up and down to see all the various properties that were programmed for that element that can be changed. In the Title field, we type in the new text “Join Now!”. The device shows the new text, and then the mirror image updates. The properties that are changed appear on the left-side panel under “My Edits”. If you don’t see a property you want to change, it means we haven’t added the property or can safely make a change to the element. Please contact Apptimize to make a product request. Since this is a what-you-see-is-what-you-get-editor, if you make a change and it doesn’t show up on your device, it means the change will also not show during an experiment.

When we click Done on the top right-hand side of the visual editor, the variant gets saved and the device screen reverts back to the original app. If we were to click Edit to make changes to the variant I just made, my device would again show the screen with the button changes.

6. Now let’s setup our second variant that has both a Register button text change and background color. Just like above, we click the “Add Variant” button to add a new variant. Again, in the visual editor we select the Register button from the mirror image and change the Title test to “Join Now!”. To edit the buttons background color, we select the down arrow next to the background color swatch. We use the eyedropper to pick up the reddish-pink color used in the “Registration information” label text to select the colors to change to, or we could have entered in the values.

7. When we’re happy with the variant, we can click the “Done” button in the upper right to close the modal and return to the Configure step. Changes in the visual editor are automatically saved as you make the changes. The Configure step now show the 3 variants: original with “Register”, “Join Now!”, and “Join Now!” with a button color change. While we could make more changes, such as changing the button background image, the button size and location, and hiding Twitter or Facebook login options, we’re keeping it simple for this first iteration on the Register Button.

8. Click Continue to Targeting at the bottom or Step 4 Allocate & Target in the upper sub-navigation to setup who will see the experiment. In this case, we set the allocation to 30% of our users exclusively. Then add a filter so only new users will see this experiment, since current users have already had to login and cannot participate in the experiment.

9. Next, we’ll continue to review our criteria to prepare for launch. We can see the variant name, allocation percent (10%), our target (“user is new”), and have the option to preview variants with the Preview Variants tool or set up early access and QA using Preview Groups tool in Step 5 Preview & Launch. When we are ready, we can start the experiment by clicking the Launch Experiment button at the bottom, which will make the experiment move from “draft” to “running” state, and end users will see the experiment on their devices within roughly 15 minutes.

You can find more information about stopping the experiment, setting a winner and interpreting reading results in A/B Experiment Results

Caution: Make your App changes on the app version and device you want changes to applied to. For instance, due to changes in layout between iPad and iPhone and changes between API between iOS 11-16 or Android 8-12, visual changes may only apply to the iOS or form factor (phone/tablet) that the visual experiment was setup on (changes between iOS 9 and 10 are OK).

Note

Visual Experiments set up on Android devices 11 and below will only work on those devices, while experiments set up on Android 12 and above will work on ALL Android devices. You will only be able to edit Android Visual Experiments with a device in the same class (11 and below or 12 and above) that was used to set up the experiment.

If your experiment is created on elements on the very first view of your app, please take a look at this FAQ answer to avoid testing pitfalls.