Managing experiments

Overview

This topic explains how to view the Experiments list, edit experiments, and archive experiments when you are done with them.

The experiments list

You can view information about your new, running, and stopped experiments on the Experiments list.

The Experiments list includes the following information about each experiment:

• Name
• Status: Not started, running, or stopped. Hover on the status for an experiment to view more information about its status.
• Duration: How long the current iteration of the experiment has been running.
• Started: The date and time that you started the current iteration of the experiment.
• Stopped: The date and time that you stopped the latest iteration of the experiment. If this column is blank, the experiment has either not been started yet, or is still running.
• Metrics used: Any primary or secondary metrics and any metric groups used in the experiment. Hover on the metric name to view a full list of all metrics used.
• Type: Feature change, funnel optimization, Data Export, or Snowflake native.
• Maintainer: The member who created the experiment.

You can show or hide any of these columns by clicking the Display menu at the top of the list, and selecting or deselecting the appropriate column.

Click on the three-dot overflow menu for an experiment to copy the experiment key, clone the experiment, copy a direct link to the experiment, or archive the experiment.

Click on an experiment name to open the experiment. You can then:

• Edit the experiment design
• View experiment results
• Stop the experiment or start a new iteration
• View experiment data over set periods of time from the Iterations tab:

Experiment list filters

Click on Filters at the top of the Experiments list to filter the list by the following options:

• Status: Not started, running, or stopped
• Started: Experiments started within a specific timeframe
• Metrics used
• Type: Feature change, funnel optimization, Data Export, or Snowflake native
• Maintainer
• Holdouts: Only experiments included in a specific holdout
• Layer: Only experiments included in a specific layer
• Archived: Whether or not the experiment is archived

Experiment iterations

When you start an experiment, LaunchDarkly creates a new iteration for that experiment. Each iteration includes the results of the experiment over a period of time for its specific configuration. When you stop an experiment or edit its configuration, including its hypothesis, metrics, variations, or audience, LaunchDarkly ends the iteration. This ensures that your experiment results are valid.

To learn more, read Starting and stopping experiment iterations.

Edit experiments

You can make changes to the name, hypothesis, randomization unit, metrics, and audience of an existing experiment. Changing the audience of a running experiment requires you to stop the experiment first, and restart a new iteration after you change the audience.

After you create an experiment, you cannot edit its flag or the flag’s variations. If you want to use a different flag or a different flag variation, you must create a new experiment.

Change experiment settings

You can change an experiment’s name and hypothesis at any time without affecting the results of the experiment.

If you want to begin measuring a completely different randomization unit or metric as part of an experiment, we recommend creating a new experiment instead of editing an existing one. If you want to use a similar metric, you can change the metric associated with an experiment.

To change an experiment’s name, hypothesis, metric, or randomization unit:

1. Navigate to the Design tab of your experiment.
2. Click Edit design.
3. Edit the Name or Hypothesis as needed.
4. Choose new Metrics as needed.
5. Choose a new randomization unit from the Randomize by menu as needed.
6. Scroll to the top of the page and click Save.
   • If the experiment was running when you made edits, a “Save experiment design?” dialog appears. Enter a reason for the change and click Save & start new iteration.

Change experiment audiences

To edit the audience of an experiment, you must stop the experiment iteration, make the audience update, and start a new iteration. This lets you select a flag variation to serve to any contexts that will no longer be included in the experiment.

To edit the audience of an experiment:

1. Navigate to the Design tab of your experiment.
2. Click Stop experiment. The “Stop experiment” dialog appears.
3. Select a winning variation to serve while the experiment is stopped.
4. Enter a Reason for stopping.
5. Click Stop experiment.
6. Click Edit design.
7. Navigate to the “Audience targeting” section and update the Experiment audience as needed.
   • You can also update the Sample size if you need to change the percentage of contexts to serve the experiment to.

8. Scroll to the top of the page and click Save.
9. Click Start to start a new iteration of the experiment.

Archive experiments

You can archive experiments that have concluded, as well as the flags and metrics attached to them, but you cannot permanently delete experiments. Archiving experiments preserves the results so you can refer to them in the future.

Before you can archive an experiment:

• The experiment must not be running. To learn how to stop an experiment, read Stop experiment iterations.
• The experiment must not be included in a layer. To learn how to remove an experiment from a layer, read Remove experiments from layers.

LaunchDarkly hides archived experiments from the Experiments list. You cannot start new iterations for archived experiments.

To archive an experiment:

1. Navigate to the Experiments list in the environment you want to archive an experiment in.
2. Click on the name of the experiment you want to archive. The experiment detail page appears.
3. Click Archive experiment.

To view archived experiments on the Experiments list, click View and select Archived experiments. To switch back to viewing active experiments, click View and select Active experiments.

To restore an experiment, click Restore experiment from the experiment details page.

When you archive or restore an experiment, LaunchDarkly sends the maintainer and anyone following the experiment an email, an in-app notification, and, if you have the Slack app integration configured, a Slack notification.