> ## Documentation Index
> Fetch the complete documentation index at: https://launchdarkly.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Databricks native Experimentation
**Warehouse native Experimentation is not available in federal environments**
To learn more, read LaunchDarkly in federal environments.
**Contact us for help configuring Databricks native Experimentation**
Access to Databricks native Experimentation requires both [Data Export](/docs/integrations/data-export/databricks) and [Experimentation](/docs/home/experimentation). If you do not have access or need help getting started, contact your LaunchDarkly representative or [start a Support ticket](https://support.launchdarkly.com/hc/en-us/requests/new).
This topic explains how to set up the Databricks native Experimentation integration. Databricks native Experimentation lets you run experiments in LaunchDarkly using data directly from your AWS Databricks data warehouse. This lets LaunchDarkly experiments read and analyze your metrics data stored in Databricks, without requiring you to send event data through LaunchDarkly SDKs.
Setting up the integration requires the following steps:
1. [Choosing an authentication method](#step-1-choose-an-authentication-method)
2. [Creating credentials in Databricks](#step-2-create-credentials-in-databricks)
3. [Granting Unity Catalog permissions](#step-3-grant-unity-catalog-permissions)
4. [Gathering your Databricks catalog and warehouse details](#step-4-gather-your-databricks-catalog-and-warehouse-details)
5. [Configuring the LaunchDarkly integration](#step-5-configure-the-launchdarkly-integration)
6. [Verifying the connection](#step-6-verify-the-connection)
## Prerequisites
Before setting up Databricks native Experimentation, ensure that you have:
* The [Databricks Data Export integration](/docs/integrations/data-export/databricks) set up in LaunchDarkly for the project and environment you use. Only environments with a Databricks Data Export destination are available when you create this integration.
* A Databricks workspace that uses Unity Catalog.
* A Databricks user who can create a service principal and grant Unity Catalog privileges. The service principal that LaunchDarkly uses does not need admin rights. To learn which privileges it needs, read [Step 3](#step-3-grant-unity-catalog-permissions).
* Catalog and schema names for experiment results, and optional metrics catalog and metrics schema names.
* An active Databricks SQL warehouse.
* A [LaunchDarkly role](/docs/home/account/roles) of Owner or Admin, or a custom role that allows the following actions:
* [add and edit integrations](/docs/home/account/roles/role-actions#integration-actions)
* [add destinations](/docs/home/account/roles/role-actions#destination-actions)
* [add metric data sources](/docs/home/account/roles/role-actions#metric-data-source-actions)
* Event data in your Databricks database that you want to use for Experimentation.
* An allow list in Databricks that includes the following outbound IP addresses:
- `52.21.152.96/32`
* `52.200.35.24/32`
* `52.200.50.23/32`
* `54.144.218.89/32`
* `54.221.221.197/32`
* `34.236.6.43/32`
* `3.78.21.121/32`
* `3.124.211.237/32`
* `18.153.39.243/32`
* `3.75.12.2/32`
* `3.68.26.17/32`
* `3.77.253.41/32`
For a full list of LaunchDarkly public IP addresses, read [Public IP list](/docs/home/infrastructure/ip-list).
## Step 1: Choose an authentication method
LaunchDarkly supports two methods for authenticating the Databricks native Experimentation integration to your workspace:
* **OAuth M2M**: LaunchDarkly authenticates with the client ID and client secret of a Databricks service principal. This is the default method when the authentication method control appears in the configuration form.
* **Personal access token**: LaunchDarkly authenticates with a Databricks personal access token (PAT).
You select the authentication method when you create the integration. You cannot change it after you save the configuration. To use different OAuth credentials or to switch methods, delete the integration and create a new one.
If the "Authentication method" option does not appear in the configuration form, OAuth M2M is not enabled for your account. Use a personal access token instead.
## Step 2: Create credentials in Databricks
Create the credentials that match the authentication method you chose.
### Create OAuth M2M credentials
LaunchDarkly connects to your warehouse as a Databricks [service principal](https://docs.databricks.com/aws/en/admin/users-groups/service-principals) using OAuth M2M authentication. Generate an OAuth secret for the service principal, then provide its client ID and secret to LaunchDarkly. To learn more, read the Databricks documentation about [OAuth machine-to-machine (M2M) authentication](https://docs.databricks.com/aws/en/dev-tools/auth/oauth-m2m).
To create a service principal, follow the Databricks instructions to [Add service principals to your account](https://docs.databricks.com/aws/en/admin/users-groups/manage-service-principals#-add-service-principals-to-your-account). Ensure that the principal has **Workspace access** and **Databricks SQL access** entitlements. To learn how, read the Databricks documentation about [Access entitlements](https://docs.databricks.com/aws/en/security/auth/entitlements).
To generate the OAuth credentials:
1. Navigate to the Databricks account console, click **Settings**, then open **Identity and access** and find "Service principals."
2. Select the service principal you created, then copy its **Application ID**. This is your client ID.
3. Open the service principal's **Secrets** tab, then click **Generate secret**.
4. Set a **Lifetime (days)** for the secret, then click **Generate**.
5. Copy the **Secret** and save it in a secure location. Databricks displays the secret only once. This is your client secret.
You provide the client ID and secret later when you [configure the LaunchDarkly integration](#step-5-configure-the-launchdarkly-integration). After you save, you cannot rotate the client ID or client secret in place.
Under SQL warehouse [permissions](https://docs.databricks.com/aws/en/compute/sql-warehouse/create#manage-a-sql-warehouse), assign the **Can use** permission to the service principal.
### Create a personal access token
Use this procedure if you want to use a **Personal access token** to set up the integration. First, create a workspace token in Databricks. Then use that token to create the PAT that LaunchDarkly uses. Finally, grant your service principal permission to use personal access tokens.
To create a workspace token:
1. Navigate to **Workspace settings** > **Developer**.
2. Click **Manage**.
3. Click **Generate new token**.
4. Copy and save the token. You use this token as the bearer token in the next request.
Create the PAT that LaunchDarkly uses by sending the following request. Replace `DATABRICKS_HOSTNAME` with your workspace hostname, `EXISTING_TOKEN` with the token from the previous steps, and `LIFETIME_SECONDS` with the token lifetime in seconds:
```bash title="API access token" lines wrap theme={null}
curl -X POST https://DATABRICKS_HOSTNAME/api/2.0/token/create \
-H "Authorization: Bearer EXISTING_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"comment": "New PAT",
"lifetime_seconds": LIFETIME_SECONDS
}'
```
Copy and save the PAT from the response. Provide this PAT to LaunchDarkly when you [configure the LaunchDarkly integration](#step-5-configure-the-launchdarkly-integration).
Then grant the service principal permission to use personal access tokens. You can use the service principal you created for [Databricks Data Export](/docs/integrations/data-export/databricks#create-a-service-principal):
1. Navigate to **Workspace settings** > **Workspace admin** > **Advanced** > **Personal access tokens** > **Permission settings**.
2. Add the service principal.
3. Assign the **Can use** permission.
## Step 3: Grant Unity Catalog permissions
LaunchDarkly connects as the service principal you created, or as the owner of the personal access token. This principal does not need workspace or account admin rights. Grant it only the Unity Catalog privileges below, before you save the integration. These grants apply to both authentication methods.
For the **Catalog** and **Schema** where LaunchDarkly writes experiment results, grant the privileges that match how you set up the schema:
* If the schema already exists: `USE CATALOG` on the catalog, and `USE SCHEMA`, `CREATE TABLE`, `MODIFY`, and `SELECT` on the schema.
* If you want LaunchDarkly to create the schema: `USE CATALOG` and `CREATE SCHEMA` on the catalog. The service principal owns the schema it creates and needs no schema-level grants.
LaunchDarkly owns the tables it creates in this schema and updates their columns as new Experimentation features ship.
For the tables your metrics read from, grant read-only access. LaunchDarkly reads only the tables that your [metric data sources](/docs/home/warehouse-native/metric-data-sources) and [assignment data sources](/docs/home/warehouse-native/assignment-data-sources) query, and never writes to them:
* If you specify a **Metrics catalog** and **Metrics schema**: `USE CATALOG` on the catalog, and `USE SCHEMA` and `SELECT` on the schema.
* If you specify a **Metrics catalog** without a **Metrics schema**: `USE CATALOG`, `USE SCHEMA`, and `SELECT` on the catalog. This allows LaunchDarkly to read every schema in it.
* If a data source queries a table in any other catalog or schema, grant `USE CATALOG`, `USE SCHEMA`, and `SELECT` for that location too.
For the catalog and schema of your [Databricks Data Export destination](/docs/integrations/data-export/databricks), LaunchDarkly needs `USE CATALOG`, `USE SCHEMA`, and `SELECT` to read exposure data. If you use the same service principal as Data Export, it already has these privileges.
Here is an example that grants the required privileges. Replace the catalog, schema, and principal names with your own:
```sql title="Grant Unity Catalog privileges" lines wrap theme={null}
-- Experiment results schema that already exists
GRANT USE CATALOG ON CATALOG results_catalog TO `SERVICE_PRINCIPAL`;
GRANT USE SCHEMA, CREATE TABLE, MODIFY, SELECT ON SCHEMA results_catalog.results_schema TO `SERVICE_PRINCIPAL`;
-- Or, to let LaunchDarkly create the experiment results schema
GRANT USE CATALOG, CREATE SCHEMA ON CATALOG results_catalog TO `SERVICE_PRINCIPAL`;
-- Metrics catalog and schema (read only)
GRANT USE CATALOG ON CATALOG metrics_catalog TO `SERVICE_PRINCIPAL`;
GRANT USE SCHEMA, SELECT ON SCHEMA metrics_catalog.metrics_schema TO `SERVICE_PRINCIPAL`;
-- Or, to read every schema in the metrics catalog
GRANT USE CATALOG, USE SCHEMA, SELECT ON CATALOG metrics_catalog TO `SERVICE_PRINCIPAL`;
-- Data Export catalog and schema (read only)
-- Only if Experimentation uses a different service principal than Data Export
GRANT USE CATALOG ON CATALOG export_catalog TO `SERVICE_PRINCIPAL`;
GRANT USE SCHEMA, SELECT ON SCHEMA export_catalog.export_schema TO `SERVICE_PRINCIPAL`;
```
To learn more, read the Databricks documentation about [Unity Catalog privileges](https://docs.databricks.com/aws/en/data-governance/unity-catalog/manage-privileges/privileges).
## Step 4: Gather your Databricks catalog and warehouse details
LaunchDarkly fills the **Workspace URL** from the Databricks Data Export destination for the environment you select. It also copies the **HTTP path** from that destination. You can change the HTTP path to query a different SQL warehouse in the same workspace.
If you use a different warehouse than Data Export, collect the [HTTP path](https://docs.databricks.com/aws/en/integrations/compute-details) from Databricks:
1. Navigate to **SQL** > **SQL Warehouses** in Databricks.
2. Select the warehouse you plan to use.
3. Select the **Connection details** tab.
4. Copy and save the HTTP path.
Then collect the catalog and schema names you enter in LaunchDarkly:
* [Catalog](https://docs.databricks.com/aws/en/catalogs/manage-catalog#view-catalog-details): The Unity Catalog catalog where LaunchDarkly writes experiment results.
* [Schema](https://docs.databricks.com/aws/en/schemas/manage-schema#find-and-view-schemas): The schema within that catalog where LaunchDarkly writes experiment results.
* Metrics catalog (optional): The catalog that holds the data your metrics query, if it differs from the catalog above.
* Metrics schema (optional): The schema that holds the data your metrics query, if it differs from the schema above.
You cannot change these values after you save the integration.
## Step 5: Configure the LaunchDarkly integration
Now, configure the Databricks native Experimentation integration in LaunchDarkly. This integration is a second LaunchDarkly integration that looks similar to the Databricks Data Export integration, but requires a separate setup process.
To set up the integration:
1. Navigate to the [**Integrations**](https://app.launchdarkly.com/settings/integrations) page.
2. Search for and select **Databricks Native Experimentation**.
3. Click **Add integration**. The [configuration page](https://app.launchdarkly.com/settings/integrations/databricks-experimentation/new) appears.
4. Enter an integration **Name**.
5. Select the LaunchDarkly **Project and environment** for this integration. Only environments with a Databricks Data Export destination are available. You cannot change this after you save the integration.
6. Confirm the **Workspace URL**. LaunchDarkly fills this from the Databricks Data Export destination for the environment you selected. You cannot edit it.
7. Confirm or update the **HTTP path**. LaunchDarkly copies this from the Data Export destination. You can change it to query a different SQL warehouse in the same workspace. You cannot change it after you save.
8. Select an **Authentication method**. LaunchDarkly selects **OAuth M2M** by default. Click **Personal access token** to authenticate with a token instead. You cannot change the method after you save.
9. Enter the credentials for the method you selected:
* **OAuth M2M**: The **Client ID** and **Client secret** you saved in [Step 2](#create-oauth-m2m-credentials).
* **Personal access token**: The PAT you saved in [Step 2](#create-a-personal-access-token).
10. Enter your Databricks **Catalog**.
11. Enter your Databricks **Schema**.
12. (Optional) Enter your Databricks **Metrics catalog**.
13. (Optional) Enter your Databricks **Metrics schema**.
14. After reading the Integration Terms and Conditions, check the **I have read and agree to the Integration Terms and Conditions** checkbox.
15. Click **Test connection and save**.
If the "Authentication method" option does not appear, skip step 8 and enter a **Personal access token** in step 9.
## Step 6: Verify the connection
Finally, verify the connection. On the integration configuration page, check the "Connection status" indicator. A green **Healthy** status confirms LaunchDarkly can connect to Databricks and has the correct permissions.
## Edit an existing integration
After you save the integration, most fields are read-only.
You can change the **Name**. If you authenticated with a personal access token, you can rotate the token. Leave the obfuscated token value unchanged to keep the current token.
You cannot change the authentication method, **Client ID**, **Client secret**, project, environment, workspace URL, HTTP path, catalog, schema, metrics catalog, or metrics schema. To use different OAuth credentials or a different authentication method, delete the integration and create a new one.
You can update the maintainer and review connection health from the side panel.
## Next steps
Read the following topics to understand how to create an experiment using a Databricks native metric and how to analyze the experiment results:
* [Creating experiments using warehouse native metrics](/docs/home/warehouse-native/creating)
* [Analyzing experiments](/docs/home/experimentation/analyze)
The results page for a warehouse native experiment displays the date and time the results were last updated from Databricks. The initial load of experiment results can take up to 60 minutes to appear, and further results updates appear within 15–30 minutes.