The ability to test, analyze, and iterate quickly is a must for all stages of product development. At LaunchDarkly, we've always been committed to helping teams make data-driven decisions with our Experimentation product, and our new collaboration with Snowflake takes this to a whole new level.
In our recent webinar, LaunchDarkly Experimentation Specialist Aaron Montana sat down with Tim Jones, Senior Director, Innovation Solutions Team at Snowflake to discuss how Warehouse Native Experimentation with LaunchDarkly and Snowflake works. They also did a demo to illustrate the use cases.
Click on the image below to watch the webinar, or read the highlights below:
What is Warehouse Native Experimentation?
Traditional experimentation often requires a lot of data to be extracted, transformed, and loaded into multiple tools for analysis. This introduces compliance risks and data governance challenges.
Warehouse Native Experimentation eliminates these barriers by enabling teams to run and analyze experiments directly within their data warehouse—in this case, Snowflake.
Using the LaunchDarkly-Snowflake integration, you can:
- Run and analyze experiments in your warehouse while keeping data secure and compliant.
- Use existing business logic and trusted metrics to measure results accurately.
- Accelerate experimentation velocity by eliminating data silos and reducing manual effort.
How Snowflake powers Warehouse Native Experimentation
Snowflake’s AI Data Cloud provides a robust, scalable environment for managing and analyzing data. Its cloud-based architecture ensures easy integration across AWS, Azure, and Google Cloud, allowing teams to unify data from various sources into a single source of truth.
Key components of Snowflake’s platform include:
- Elastic Compute for processing large-scale experimentation data efficiently.
- Cortex AI for advanced analytics and AI-powered insights.
- Snowgrid for cross-region and cross-cloud data sharing.
With LaunchDarkly sending experimentation data to Snowflake, teams can conduct deeper analyses using first-party metrics without additional data transformations or third-party analytics platforms.
Benefits for engineering, product, and data teams
Experimentation should be a planned part of the development process, not an afterthought. By integrating LaunchDarkly’s feature flagging and experimentation capabilities with Snowflake’s data cloud, teams can:
- Leverage a single source of truth. Instead of exporting data to multiple locations, experiment data remains in Snowflake, reducing discrepancies and ensuring compliance.
- Perform holistic analysis. Snowflake allows teams to analyze experimental data alongside financial, operational, and behavioral data for a more comprehensive view of business impact.
- Drive faster, data-backed decisions. Real-time access to high-fidelity experimentation data empowers product teams to iterate quickly and optimize feature releases based on statistically valid insights.
Three pathways to run experiments
When you use LaunchDarkly Experimentation, you can choose from three distinct pathways: LaunchDarkly-hosted analysis, custom warehouse analysis, and Snowflake Native Warehouse Experimentation.
LaunchDarkly-hosted experimentation
In this scenario, all experimentation workflows—from flag management to analysis—occur within LaunchDarkly. This is ideal for teams looking for an out-of-the-box solution with minimal setup.
Custom warehouse analysis
In this setup, LaunchDarkly assigns users to experiments and sends data to the warehouse for custom analysis. Teams with existing data science workflows can use their preferred models while maintaining LaunchDarkly’s feature flagging capabilities.
LaunchDarkly-powered warehouse analysis
In this scenario, LaunchDarkly runs analysis directly on warehouse-stored data, ensuring the highest accuracy and consistency. This bidirectional integration allows for real-time experimentation insights while keeping sensitive data secure.
Real-world example: Optimizing subscription conversions
So, when should you use Warehouse Native Experimentation? Let’s examine a practical example of when it adds value.
Imagine that a SaaS company wants to do some experiments using customers who don’t cancel their subscriptions. The challenge is finding the most accurate way to identify these users.
Traditional experimentation might measure this by examining users who visit the “thank you” page after a transaction. But that data point is imperfect because some people close the tab before the thank-you page appears.
Or perhaps some people have entered a fraudulent credit card, are part of a free trial, or had a failed credit card transaction—these users would still see the “thank you” page, but they do not have an active subscription. So the data is not completely accurate.
We want to reference users who have had a successful credit card transaction. This is where leveraging Warehouse Native Experimentation comes in handy, as it can provide high-fidelity data about credit card transactions directly from Snowflake.
With Warehouse Native Experimentation, only successful payments (verified transactions) are used as the source of truth. This eliminates guesswork and ensures business-critical decisions are based on validated, high-fidelity data.
How to set up a Warehouse Native Experiment
Check out this demo to see a step-by-step walkthrough of how to set up Warehouse Native Experimentation:
Getting started with Warehouse Native Experimentation
This integration is available now for LaunchDarkly and Snowflake customers. Teams can install the native application from the Snowflake Marketplace and begin running warehouse-powered experiments immediately.
Warehouse Native Experimentation is the next evolution of data-driven decision-making. By integrating LaunchDarkly’s industry-leading feature management and experimentation platform with Snowflake’s AI Data Cloud, teams can unlock new levels of experimentation velocity, accuracy, and scalability.
Like what you read?
Get a demo
