Create and manage AI Config variations

Overview

This topic explains how to create and manage AI Config variations.

An AI Config is a resource that you create in LaunchDarkly. You can use AI Configs to customize, test, and roll out new large language models (LLMs) within your generative AI applications.

Within each AI Config, you define one or more variations. Each variation includes a model configuration and, optionally, one or more messages. A variation can be in one of two states: “published” or “archived.”

Create AI Config variations

You can create variations for an AI Config using the LaunchDarkly UI. Each variation defines a unique combination of model settings and prompt messages. You can create multiple variations in the UI at the same time.

To create one or more variations in the LaunchDarkly UI:

1. In LaunchDarkly, click AI Configs.

2. Click the name of the AI Config you want to edit.

3. Select the Variations tab:

   

4. Enter a variation Name. You’ll use this name to refer to the variation when setting up targeting rules.

   Import from external playgrounds

   If you have already used an external playground to set up a model, model parameters, and messages, you can import that configuration into your AI Config variation. Here’s how:

   1. Click Import from playground.
   2. In the “Import from playground” dialog, select a provider from the Which playground are you using? menu.
   3. Paste the output from your provider’s playground into the text box.
   4. Click Import.
   5. Skip to step 10 in the main procedure.

5. Click Select a model and choose a model to use.

   • LaunchDarkly provides a frequently updated list of common models.
   • You can also choose Add a model and create your own. To learn more, read Create AI model configurations.
   LaunchDarkly does not send any data to these models

   LaunchDarkly does not transmit any information you provide when creating an AI Config and its variations to external models. We do not use any of this data to fine-tune or train any models. Your model selection is solely used to suggest parameter values and help you organize which messages are associated with each model. This list of available models is updated regularly. If you want us to add a new model, click the Give feedback option to share your request.

6. After you assign a model to a variation, an Add parameters button appears. Use this button to configure model or custom parameters for the variation. To learn more, read Add parameters.

7. (Optional) Use the dropdown to select the message role for the first message in this variation. The default value is “system” but you can also choose “assistant” or “user.” Not all model providers support every role.

8. (Optional) Enter the text of the message. You can use standard Markdown formatting within the message.

   • (Optional) Use double curly braces to indicate variables that you’ll set at runtime from your application. For example, enter This is an {{ example }} message if you want to replace {{ example }} with some other value in your application code when an end user encounters this variation.
   • (Optional) Use double curly braces, the ldctx prefix, and dot (.) notation to indicate context attributes whose values you want to use at runtime. For example, enter Describe the typical weather in {{ ldctx.city }} to replace {{ ldctx.city }} with the city attribute from each context that encounters this AI Config.
   • To learn more about how variables and context attributes are inserted into messages at runtime, read Customizing AI Configs.

9. (Optional) Click + Add another message and repeat steps 7 and 8 to include another message in this variation.

10. Click Review and save to save the AI Config variation.

After creating an AI Config and its variations, you can set up targeting rules and use the AI Config in your SDK.

(Optional) Add parameters

After you assign a model to a variation, an Add parameters button appears. Use this button to configure model or custom parameters for the variation.

To add parameters:

1. Click Add parameters to see the Model parameters and Custom parameters options.
2. To add model parameters:
   1. Click Model parameters.
   2. Click + Add model parameters.
   3. Select a model parameter, such as temperature or max_tokens, from the list. The available parameters vary depending on the model you select.
   4. In the configuration panel for the selected parameter, view the “Base value” and optionally enter an “Override value” for this variation.
   5. To add more parameters, click + Add model parameters again.
3. To add custom parameters:
   1. Click Custom parameters.
   2. In the JSON editor that appears, enter your custom parameters as key-value pairs inside the curly braces ({}).

You can continue creating or editing additional variations, or complete the variation creation process:

1. Click Review and save. The “Save changes” dialog lists all the variations you’ve created. You can view your changes in a unified or split view, and expand or collapse individual variations as needed.
2. (Optional) Enter a comment describing your changes.
3. Click Save to add all variations to the AI Config. Each new variation receives a version number, and the version numbers of existing variations are incremented.

Edit AI Config variations

You can update variations for an AI Config using the LaunchDarkly UI. You can edit multiple variations in the UI at the same time.

To update one or more variations in the LaunchDarkly UI:

1. In LaunchDarkly, click AI Configs.
2. Click the name of the AI Config you want to edit.
3. Make changes to any of the AI Config’s variations. You can expand or collapse individual variations to help you focus on specific updates.
4. Click Review and save. The “Save changes” dialog lists the updates for each variation. You can view your changes in a unified or split view, and collapse or expand individual variations as needed.
5. (Optional) Enter a comment describing the changes.
6. Click Save to create a new version of the AI Config that includes all the updates. Each updated variation receives a new version number.

Duplicate an AI Config variation

Often, when you create a new variation for an AI Config, you want the new variation to be very similar to an existing variation. You may only be changing the messages slightly or adjusting the model parameters. Instead of creating a new AI Config from scratch, you can duplicate an existing AI Config variation and make changes to the copy.

To duplicate an existing AI Config variation:

1. Navigate to the detail page for the AI Config.

2. Select the Variations tab.

3. Find the variation you want to duplicate.

4. Click the three-dot overflow menu for the variation and select Duplicate:

   

5. The new variation has the same name as the one you duplicated, appended with “(Copy)”. Update the new variation as needed.

6. Click Review and save.

Working with archived AI Config variations

Each AI Config variation can be in one of two states: “published” or “archived.”

By default, all variations are published. You can only use published variations in targeting rules.

You can mark a variation as archived if you want to keep it for reference or comparison purposes, but no longer need to use it in targeting rules or running experiments.

Archive AI Config variations

To archive an AI Config variation:

1. Navigate to the detail page for the AI Config.
2. Select the Variations tab.
3. Find the variation you want to archive.
4. Click the three-dot overflow menu and choose Archive.

The variation moves to the archived view on the Variations tab, and its version number is incremented. If you compare the new variation version to the previous one, there will be no differences in the model configuration or messages. The only difference between the two variation versions is the publication state.

View archived AI Config variations

To view archived AI Config variations:

1. Navigate to the detail page for the AI Config.
2. Select the Variations tab.
3. From the Published dropdown, select Archived to open the list of archived variations.

Restore archived AI Config variations

To restore an archived AI Config variation:

1. Navigate to the detail page for the AI Config.
2. Select the Variations tab.
3. From the Published dropdown, select Archived to open the list of archived variations.
4. Find the variation you want to restore.
5. Click the three-dot overflow menu and choose Restore.

The variation moves to the published view on the Variations tab, and its version number is incremented. If you compare the new variation version to the previous one, there will be no differences in the model configuration or messages. The only difference between the two variation versions is the publication state.

Delete an AI Config variation

To delete an AI Config variation:

1. Navigate to the detail page for the AI Config.
2. Select the Variations tab.
3. Find the variation you want to delete. Use the Published dropdown to switch between the published and archived views if needed to find the variation.
4. Click the three-dot overflow menu and choose Delete variation.