Creating experiments using warehouse native metrics

Overview

This topic explains how to set up and configure an experiment in LaunchDarkly that uses metric events from your own data warehouse.

Before you create an experiment using warehouse native metrics, you must enable warehouse Data Export.

Configuring an experiment requires several steps:

1. Creating a warehouse native metric
2. Creating the flag or AgentControl config and its variations
3. Building the experiment
4. Turning on the flag or AgentControl config
5. Starting an iteration

These steps are explained in detail below.

Prerequisites

Before you build an experiment, you must:

• enable Warehouse Data Export in your LaunchDarkly account for the specific warehouse you’re using
• configure your external warehouse for LaunchDarkly Experimentation

Create a metric

Warehouse native metrics must:

• be any of custom conversion binary, custom conversion count, or custom numeric
• use the “average” analysis method

Experiments using warehouse native metrics cannot use:

• clicked or tapped metrics
• page viewed metrics
• metric groups
• LaunchDarkly hosted metrics
• a percentile analysis method

Create a flag

Before you begin an experiment, create a flag with the variations you plan to test the performance of. You do not need to toggle on the flag before you create an experiment, but you do have to toggle on the flag before you start an experiment iteration.

Limitations

You cannot run an experiment on a flag or AgentControl config if:

• any of its rules have an active guarded rollout
• any of its rules have an active progressive rollout
• any of its rules are in a running Data Export experiment
• any of its rules are in a running experiment with warehouse native metrics
• the flag is a migration flag

You can build and run multiple experiments on the same flag or AgentControl config as long as there is only one running experiment per rule. You cannot run multiple experiments on the same rule at the same time.

To learn more, read Creating new flags and Creating flag variations.

Build the experiment

To build an experiment, follow the experiment creation instructions. You must select a warehouse native metric in the metric selection step.

Turn on feature flags

For an experiment to begin recording data, the flag used in the experiment must be on. To learn how, read Turning flags on and off.

Start an experiment iteration

After you create an experiment and toggle on the flag, you can start an experiment iteration in one or more environments. To learn how, read Starting and stopping experiment iterations.

Experiment iterations allow you to record experiments in individual blocks of time. To ensure accurate experiment results, when you make changes that impact an experiment, LaunchDarkly starts a new iteration of the experiment.

You can also use the REST API: Create iteration