Creating experiments using warehouse native metrics

Overview

This topic explains how to set up and configure an experiment in LaunchDarkly that uses metric events from your own data warehouse.

Before you create a warehouse native experiment, you must enable Warehouse Data Export and configure the Snowflake native Experimentation app in your external warehouse for your LaunchDarkly account.

Configuring a warehouse native experiment requires several steps:

1. Creating a warehouse native metric
2. Creating the flag or AI Config and its variations
3. Building the experiment
4. Turning on the flag or AI Config
5. Starting an iteration

These steps are explained in detail below.

Prerequisites

Before you build a warehouse native experiment, you must:

• enable Warehouse Data Export in your LaunchDarkly account for the specific warehouse you’re using
• if you’re using Snowflake, configure the Snowflake native Experimentation app for your LaunchDarkly account

Create a metric

Warehouse native metrics must:

• be any of custom conversion binary, custom conversion count, or custom numeric
• use the “average” analysis method

Experiments using warehouse native metrics cannot use:

• clicked or tapped metrics
• page viewed metrics
• metric groups
• LaunchDarkly hosted metrics
• a percentile analysis method

Create a flag

Before you begin an experiment, create a flag with the variations you plan to test the performance of. You do not need to toggle on the flag before you create an experiment, but you do have to toggle on the flag before you start an experiment iteration.

Limitations

You cannot run an experiment on a flag or AI Config if:

• any of its rules have an active guarded rollout
• any of its rules have an active progressive rollout
• any of its rules are in a running Data Export experiment
• any of its rules are in a running warehouse native experiment
• the flag is a migration flag

You can build and run multiple experiments on the same flag or AI Config as long as there is only one running experiment per rule. You cannot run multiple experiments on the same rule at the same time.

To learn more, read Creating new flags and Creating flag variations.

Build the experiment

To build an experiment, follow the experiment creation instructions. You must select a warehouse native metric in the metric selection step.

Turn on feature flags

For an experiment to begin recording data, the flag used in the experiment must be on. To learn how, read Turning flags on and off.

Start an experiment iteration

After you create an experiment and toggle on the flag, you can start an experiment iteration in one or more environments. To learn how, read Starting and stopping experiment iterations.

Experiment iterations allow you to record experiments in individual blocks of time. To ensure accurate experiment results, when you make changes that impact an experiment, LaunchDarkly starts a new iteration of the experiment.