Skip to main content
Contact us for help configuring ClickHouse native ExperimentationAccess to ClickHouse native Experimentation requires both Data Export and Experimentation. If you do not have access or need help getting started, contact your LaunchDarkly representative or start a Support ticket.
This topic explains how to set up the ClickHouse native Experimentation integration. ClickHouse native Experimentation lets you run experiments in LaunchDarkly using data directly from your ClickHouse service. LaunchDarkly experiments read and analyze metric events stored in ClickHouse, without requiring you to send event data through LaunchDarkly SDKs. Setting up the integration requires the following steps:
  1. Configure the LaunchDarkly integration
  2. Run the setup script in ClickHouse
  3. Test the connection and save the integration
  4. Verify the connection

​
Prerequisites

Before setting up ClickHouse native Experimentation, ensure that you have:
  • The ClickHouse Data Export integration set up in LaunchDarkly for the project and environment you use. Only environments with a ClickHouse Data Export destination are available when you create this integration.
  • A LaunchDarkly role of Owner or Admin, or a custom role that allows the following actions:
  • Access to the ClickHouse SQL console as a user who can create users and grant privileges.
  • Metric event data in ClickHouse that you want to use for Experimentation.
  • If your ClickHouse service restricts inbound connections by IP address, an IP access list that includes the following LaunchDarkly IP addresses:
    • 52.21.152.96/32
    • 52.200.35.24/32
    • 52.200.50.23/32
    • 54.144.218.89/32
    • 54.221.221.197/32
    • 34.236.6.43/32
Keep the IP address that ClickHouse Data Export requires in the same IP access list. For a full list of LaunchDarkly public IP addresses, read Public IP list.

​
About the ClickHouse databases

ClickHouse native Experimentation uses the following databases in your ClickHouse service:
  • Data Export database: The database that your ClickHouse Data Export destination writes to. LaunchDarkly reads the experiment data that Data Export sends there.
  • Metrics database: The database that holds the tables your metric data sources query. LaunchDarkly fills in the Data Export database, and you can change it if needed. The metrics database cannot be the results database.
  • Results database: The database where LaunchDarkly writes experiment results. LaunchDarkly creates it the first time it connects, if it does not already exist.
LaunchDarkly connects to these databases as a ClickHouse experimentation user that is separate from the Data Export user. The setup script that LaunchDarkly generates creates this user and grants it access to all three databases.

​
ClickHouse naming conventions

When you select a project and environment, LaunchDarkly fills in default names for the ClickHouse experimentation user and the results database:
  • User: ld_experimentation_user_<projkey>__<envkey>, where <projkey> is your project key and <envkey> is your environment key, in lowercase.
  • Results database: ld_experimentation_<projkey>__<envkey>, in lowercase.
In both names, LaunchDarkly replaces each character that is not a letter or a number with an underscore. LaunchDarkly shortens a long results database name in a way that keeps it unique to the environment. You can change the user name. It must start with a letter and contain only letters, digits, and underscores, and it cannot be default. You cannot change the results database name. Each environment gets its own results database, which keeps each environment’s experiment results separate. If another ClickHouse native Experimentation integration or a ClickHouse Data Export destination on the same ClickHouse service already uses the name, LaunchDarkly adds a number to the end, such as _2, when you generate the setup script. For example, environments with the keys prod-eu and prod_eu produce the same default name. The second environment to generate its setup script gets a results database that ends in _2.

​
Step 1: Configure the LaunchDarkly integration

Configure the ClickHouse native Experimentation integration in LaunchDarkly. This integration is a second LaunchDarkly integration that looks similar to the ClickHouse Data Export integration, but requires a separate setup process. To set up the integration:
  1. Navigate to the Integrations page.
  2. Search for and select ClickHouse Native Experimentation.
  3. Click Add integration. The configuration page appears.
  4. Enter an integration Name.
  5. Choose the LaunchDarkly project and environment in Select project and environment. Only environments with a ClickHouse Data Export destination are available. You cannot change this after you save the integration. If the Data Export setup for the environment is not finished, LaunchDarkly displays a warning. Finish the Data Export setup before you continue.
  6. Review the Host and Port fields. LaunchDarkly automatically fills in these fields from your Data Export destination configuration. You cannot edit them.
  7. In the “Configure resource names” section of the form, review the Results database field. LaunchDarkly automatically fills in this field based on your selected project and environment. To learn more, read ClickHouse naming conventions.
  8. Review the User. LaunchDarkly fills in a default name, and you can change it if needed.
  9. Review the Metrics database name. LaunchDarkly fills in the Data Export database. If your metric event tables are in a different database, enter that database instead.
  10. Enter a Password for the ClickHouse experimentation user in the “Generate setup script” section of the form. Use only letters, digits, dashes (-), and underscores (_). LaunchDarkly includes this password in the setup script. Save this password. You need it if you generate the script again.
  11. Click Generate script. LaunchDarkly displays the “Setup script.”
  12. Copy the setup script.
Leave the configuration page open. You return to it after you run the setup script. After you generate the setup script, LaunchDarkly saves a draft of the integration setup. You cannot update the project or environment after LaunchDarkly saves the draft. If you open the draft later from the integrations list, you must enter the password and generate the script again before you can save the integration. To continue from a draft:
  1. Open the draft from the integrations list.
  2. Enter the Password. If you already ran a setup script for this user, enter the same password you previously used.
  3. Click Generate script. LaunchDarkly displays the setup script.
  4. Continue with Step 2 if you have not run a setup script for this user yet, or if you changed the Metrics database name since you last ran one. Otherwise, continue with Step 3.
If you change the user, the metrics database, or the password, click Generate script again before you save. Changing the integration Name does not clear the setup script. If you change the metrics database after you already ran a setup script for this user, run the GRANT statements from the new script. The user already exists, which means you can skip the CREATE USER statement.

​
Step 2: Run the setup script in ClickHouse

Run the setup script before you save the integration. When you save, LaunchDarkly tests the connection as the ClickHouse experimentation user. The user and its privileges must already exist for this test to pass. To run the setup script:
  1. Open the SQL console for your ClickHouse service as a user who can create users and grant privileges.
  2. Paste the setup script you copied in Step 1.
  3. Run the script.
The setup script makes the following changes:
  • It creates the ClickHouse experimentation user with the password you entered. If a user with that name already exists, this statement fails, and the script does not change that user’s password.
  • It grants the user privileges to create the results database, as well as to create tables in, write to, and read from that database.
  • It grants the user read access to the Data Export database.
  • It grants the user read access to the metrics database, if it is different from the Data Export database.
You can run the GRANT statements more than once. Each run produces the same result. If you run the whole script again after the user exists, the CREATE USER statement fails, and the user and its privileges from the earlier run stay in place. The setup script gives the ClickHouse experimentation user read access only to the Data Export database and the metrics database. If a metric data source reads tables from another database, grant the experimentation user SELECT on that database as well. The setup script does not restrict the experimentation user to specific hosts. If your ClickHouse service restricts inbound connections by IP address, add the LaunchDarkly IP addresses listed in the Prerequisites to its IP access list.

​
Step 3: Test the connection and save the integration

Next, return to LaunchDarkly to save the integration:
  1. Return to the ClickHouse native Experimentation configuration page.
  2. After reading the Integration Terms and Conditions, check the I have read and agree to the Integration Terms and Conditions checkbox.
  3. Click Test connection and save. LaunchDarkly connects to ClickHouse as the experimentation user. If the connection succeeds, LaunchDarkly saves the integration and opens the integration’s page.
If the connection test fails, LaunchDarkly displays an error that starts with “ClickHouse connection test failed” and does not activate the integration. The draft remains, which means you can correct the setup and try again. To resolve the most common errors:
  • If the error says “authentication failed,” check the User and Password. If you ran an earlier setup script for this user and then generated the script again with a different password, ClickHouse still has the earlier password. Enter the password you used then and click Generate script again, or choose a different User.
  • If the error says “database not found,” check that the Data Export database and the metrics database you entered both exist in ClickHouse.
  • If the error says “TLS handshake failed,” confirm that your ClickHouse service accepts native Transport Layer Security (TLS) connections on the port that the Data Export destination uses.
  • If the error says “could not reach host,” check your ClickHouse IP access list. It must include the LaunchDarkly IP addresses listed in the Prerequisites.

​
Step 4: Verify the connection

Finally, verify the connection. On the integration’s page, check “Connection status” in the details panel. The status is one of the following:
  • Connected: LaunchDarkly can connect to ClickHouse and has the privileges it needs.
  • Pending: LaunchDarkly has not finished setting up the results database yet.
  • Degraded: LaunchDarkly connected to ClickHouse earlier, but its most recent attempt failed.
  • Disconnected: LaunchDarkly could not connect to ClickHouse.
  • Not connected: LaunchDarkly has no connection information for the integration yet.
If the status is “Degraded” or “Disconnected,” confirm that the ClickHouse experimentation user still has the privileges from the setup script, and that your IP access list still includes the LaunchDarkly IP addresses. The details panel also shows “Results last updated” with the date and time LaunchDarkly last wrote experiment results to ClickHouse.

​
Edit an existing integration

After you save the integration, the connection fields are read-only. The configuration page shows the setup script that granted LaunchDarkly access, without the password. You can change the integration Name and Status. LaunchDarkly saves each change as you make it. You can also update the maintainer and review connection health from the details panel. You cannot change the project, environment, host, port, user, password, results database, or metrics database. To change any of these values, delete the integration and create a new one. To delete the integration, click Delete integration, type DELETE to confirm, and click Delete.

​
Next steps

Read the following topics to understand how to create an experiment using ClickHouse native metrics and how to analyze the experiment results: The results page for a warehouse native experiment displays the date and time the results were last updated from ClickHouse. Experiment data reaches ClickHouse through the hourly Data Export job. Because of this, the first results can take up to three hours to appear after an experiment starts receiving traffic. After that, LaunchDarkly refreshes results every 15 minutes.