Glossary

A/B Testing

Advanced Verify

An advanced option before launching an experiment that allows you to review the experiment in an interactive manner.

Allocate

Setting the size of an experiment’s population.

Attributes

Attributes defined by you that can be used for targeting and segmentation.

Control

A variant of an experiment that is the same as the original app.

End users or users

Users or consumers of the mobile apps that Apptimize supports

Event

A specific piece of data with associated properties that we track in our analytics system.

Exclusive/Non-exclusive

Toggle that allows experiments to run in parallel with other experiments on a specific user’s device.

Experiment

An A/B test that includes a control and variant(s).

Funnel conversion rate

Percentage of users who progressed through all steps of a funnel out of all the users who enter into a funnel by converting on the first step of the funnel.

Goal

Event or combination of events that measure the success of an experiment.

Launch

Taking an experiment live so that users will start participating.

Safety switch

Capability to turn off experiments or variants in real time so end users are returned to the stable default experience in case of bugs or risky releases. This can also be called a kill switch.

System Attributes

A way to select a subset of your users for an experiment.

Variant

A version of an app. Variants that you create are typically different than the original app.

Winner variant

The experiment variant that’s chosen to be rolled-out to all end users after an experiment ends

Results

Conversions count

Number of unique users who triggered the event at least once.

Conversion Rate

Percentage of participants who have converted (i.e. fulfilled the goal at least once).

Enrollment

See Population.

Filter

An expression using an attribute, operator, and criteria used to sift through data.

Impact

Raw change in conversion rate from control to variant.

Lift

Percentage better or worse a variant is compared to the control in an experiment.

Metric Value (in CSV export)

The metric value column in the exported results CSV file represents the value of the computed metric you selected. For instance, if you are measuring conversion rate of purchases as a goal, the metric value column shows the conversion rate calculated.

Number of Sessions

Number of sessions is how many foreground-to-background transitions that we see.

Occurrences

Number of times the goal event was triggered.

Occurrences per user

Average number of occurrences per participant.

Participant

A user in an experiment’s population who has seen a variant. We determine this by whether the mobile devices runs the code that determines variant treatment.

Population (Enrolled)

Apptimize bucketed the user into the experiment and one of its variants or control. This means that the user met the filtering requirements and were randomly selected according to your allocation. Some users will be enrolled but never “Participate” in a variant because they don’t navigate to the part of the app that contains the experiment.

Retention

The number of users who have a second session- open the app at least a second time, in a given period of time.

Screens Viewed per Session

On Android this is the number of activities, popups, and fragments we measure during a session. On iOS it is the number of view controllers shown that we measure per session. Note that since this is how we measure screens view, it’s possible given how differently apps are designed that we overcount the number of screens viewed, however since in an experiment you’re measuring versus a control this doesn’t adversely affect your experiment results.

Session

A session is when a user is in your app, specifically measured by the time between when your app is foregrounded to when the app is backgrounded.

Session Interval

The session interval is time between consecutive sessions, specifically the time between when the app is backgrounded to when it is next foregrounded by a user.

Session Length

Session length is measured by the length of time between a foreground and a background.

Significance

Standard method for statistical significance by confidence. Statistical significance is the probability that the difference in the goal metric between variant and control is not due to random chance. P value equals 1 minus statistical significance.

Trajectory

Direction of lift.

Value

Total sum of all the numeric values associated with goal event.

Value per user

Total event value per participant.