Safety Switch

Apptimize can be used as a safety switch to roll back features or feature variations without releasing a new version of the app. In addition to Feature Flags, Apptimize experiments can also act as safety switches:

  • Experiment level safety switch to turn off an entire experiment within minutes

  • Variant level safety switch can turn off specific variants while allowing the experiment to continue running

  • Backwards compatibility Necessary SDK update to use the variant-level safety switch

Experiment Level Safety Switch

A live experiment can be stopped to return all end users to the original experience. This option is best used when multiple variants have a buggy experience; our recommendation is to fix the bugs and restart the experiment.

To stop an experiment, first click on the “Stop Experiment” option in either the Results Dashboard or Experiment Setup pages.

../_images/stop-experiment.png

Next, select Original as the winner to show the default baseline experience to all end users instead.

../_images/select-original.png

Variant Level Safety Switch

Apptimize can also stop individual variants by deleting them in a live experiments in real time. This option is best used if buggy experiences are limited to only one variant. Deleting individual variants does not interrupt the rest of the experiment.

When a variant is deleted from a live experiment, two changes will occur. First, all the users who are allocated to the variant will be removed from experiment and thus returned to the default original experience. The removed users’ data will no longer be tracked as a part of the experiment. Second, Apptimize will stop allocating more users to the variant. In the results dashboard, Apptimize shows which variant is deleted and the last known snapshot of results from the variant. Since no end user is seeing the deleted variant anymore, the results data will no longer update.

Variant deletion is best used to correct for buggy experiences. Whereas pausing allocation is meant for give your team flexibility in product roll-out and experiment analysis without impacting users who are already in the variants. Deleting is permanent; while pausing is meant to be temporary. We recommend using deletion only when user experience is deeply impacted.

Note: Variant deletion is irreversible. Please verify your changes in the Review Changes modal.

To delete a variant, click on the “Delete” option beside the variant in Experiment Setup.

../_images/delete-variant.png

Note: Apptimize requires an experiment to have at least one variant besides the original. Please stop the experiment if you’d like to switch off the only running variant. If your experiment has ended, and a bug has been discovered in the selected winner variant, please select a different winner to stop the buggy experience. If you’d like to change the default variant, please select a winner in your experiment and then set-up a new running experiment, instead of trying to “delete” the original variant.

Backwards Compatibility

Experiment level safety switch is fully backwards compatible for all versions of the SDK.

Variant level safety switch is a new feature that is supported by iOS SDK version 3.0.0+ and Android SDK version 3.0.0+. Please make sure to set-up separate experiments targeting users on older SDK versions vs. 3.0.0+ SDK versions. Deleting a variant on older versions of the SDK may cause unexpected behavior such as re-allocating users in deleted variants to other variants in the experiment.