Customizing AI Configs | LaunchDarkly

Overview

This topic explains how to customize an AI Config using LaunchDarkly AI SDKs.

Customizing an AI Config evaluates the configuration for a given context and returns the appropriate variation based on targeting rules. Customization requires an AI Config key, a context, a fallback value, and optional variables for substitution.

In completion mode, the customized variation includes the model configuration and messages for the context. In agent mode, the customized variation includes the instructions for the context. In both cases, you must customize the AI Config each time you generate content from your AI model.

Customize messages

Customizing an AI Config substitutes context attributes and optional variables into the messages or instructions defined in the AI Config variation. If a variation includes multiple messages, all messages are customized and returned.

If the SDK cannot retrieve the variation, such as when the AI Config key does not exist or LaunchDarkly is unreachable, the SDK returns the fallback value.

Syntax for customization

When you create a message or instruction in an AI Config variation, use the following syntax to substitute context attributes or other variables.

Use double curly braces to indicate variables provided by your application. For example:
- Enter This is an {{ example }} in the message or instruction in the LaunchDarkly UI. Variable names cannot include a period (.).
- In the SDK, pass a dictionary or key-value pair with the key example and the value to substitute.
Use double curly braces with the ldctx prefix and dot (.) notation to reference context attributes. For example:
- Enter Describe the typical weather in {{ ldctx.city }} to substitute the city attribute from each context that encounters the AI Config.
- Context attribute names cannot include a period (.). To reference nested attributes, use dot notation in the message or instruction. For example, if the context includes an address object with a city field, use {{ ldctx.address.city }}.
- In the SDK, the context is a required parameter. You do not need to pass additional variables when using ldctx.

Customization is based on context

Customization adds a context to the Contexts list if a context with the same key does not already exist. Each SDK evaluates the AI Config using only the context object you provide at runtime.

The SDK does not automatically use attributes shown on the Contexts list, and attributes are not synchronized across SDK instances. Provide all required attributes with each customization to ensure variables in messages or instructions are substituted correctly. To learn more, read Context configuration.

AI SDKs

This feature is available for the following AI SDKs:

.NET AI

Expand .NET AI SDK code sample

Use the Config function to customize an AI Config. Customization substitutes context attributes and variables into the prompt messages defined in the AI Config variation. You must call Config each time you generate content from your AI model.

The Config function takes the AI Config key, a Context, a fallback value, and optional additional variables to substitute into your prompt. It performs the evaluation, then returns the customized prompt and model along with a tracker instance for recording prompt metrics. You can pass the customized prompt directly to your AI.

The fallback value is the value of the AI Config variation that your application should use in the event of an error, for example, if the AI Config key is not valid, or if there is a problem connecting to LaunchDarkly. You can use LdAiConfig.Disabled as a fallback value and handle the disabled case in your application.

Alternatively, you can create a custom LdAiConfig object using LdAiConfig.New().

Here is an example of calling the Config method:

.NET AI SDK

1 var fallbackConfig = LdAiConfig.New()
2   .SetEnabled(false)
3   .Build()
4 
5 var tracker = aiClient.Config(
6   "ai-config-key-123abc",
7   context,
8   fallbackConfig,
9   new Dictionary<string, object> {
10     { "exampleCustomVariable", "exampleCustomValue" }
11   }
12 );
13 
14 if (tracker.Config.Enabled == true) {
15 
16   // Send a request to your AI provider, using the details from the customized Config
17 
18 } else {
19 
20   // Application path to take when the tracker.Config is disabled
21 
22 }

After the function call, you can view the context that you provided to it on the Contexts list.

To learn more, read Config.

Go AI

Expand Go AI SDK code sample

Use the Config() function to customize an AI Config. Customization substitutes context attributes and variables into the prompt messages defined in the AI Config variation.

The Config() function takes an AI Config key, a context, and a fallback value. It performs the evaluation and returns a Config object with the customized messages and model, along with a Tracker object to capture performance metrics. You can pass the customized messages directly to your AI.

The fallback value is used when an error occurs, such as when the AI Config key is invalid or LaunchDarkly is unreachable. You can use an empty, disabled Config as a fallback value by calling ldai.Disabled() and handle the disabled case in your application. Alternatively, you can create a custom Config using NewConfig.

Here is an example of calling the Config() method:

Go AI SDK

1 fallbackValue := NewConfig().Build() // by default, the Config is disabled
2 
3 cfg, tracker := aiClient.Config("ai-config-key-123abc", context, fallbackValue, map[string]interface{}{"exampleCustomVariable": "exampleCustomValue"})
4 
5 if cfg.Enabled() {
6 
7   // Send a request to your AI provider, using details from the customized cfg
8 
9 } else {
10 
11   // Application path to take when the cfg is disabled
12 
13 }

After the function call, you can view the context that you provided to it on the Contexts list.

To learn more, read Config.

Node.js (server-side) AI

Expand Node.js (server-side) AI SDK code sample

Use the config(), agent(), and agents() functions to customize AI Configs at runtime. Customization substitutes context attributes and optional variables into the messages or instructions defined in the AI Config variation.

The customization behavior depends on whether the AI Config uses completion mode or agent mode. You select the mode when you create the AI Config in the LaunchDarkly UI.

Customize AI Configs in completion mode

In completion mode, each AI Config variation includes a single set of roles and messages used to prompt your generative AI model.

Use config() to customize the AI Config. The config() function takes an AI Config key, a Context, a fallback value, and optional variables for substitution. It performs the evaluation and returns the customized messages and model along with a tracker instance for recording metrics. You can pass the customized messages directly to your AI provider.

The fallback value is used when an error occurs, such as when the AI Config key is invalid or LaunchDarkly is unreachable. You can use an empty, disabled configuration as a fallback value and handle the disabled case in your application. Alternatively, you can construct a fallback configuration using values from an AI Config variation defined in the LaunchDarkly UI.

Here is an example of calling the config() method:

Customize an AI Config in completion mode

1 const fallbackConfig = { enabled: false };
2 
3 const aiConfig: LDAIConfig = aiClient.config(
4   'ai-config-key-123abc',
5   context,
6   fallbackConfig,
7   { exampleCustomVariable: 'exampleCustomValue' },
8 );
9 
10 if (aiConfig.enabled) {
11   // Send a request to your AI provider using the customized config
12 } else {
13   // Application path to take when the AI Config is disabled
14 }

Customize AI Configs in agent mode

In agent mode, each AI Config variation includes a set of instructions that enable multi-step workflows.

Use agent() or agents() to customize AI Configs in agent mode. The agent() function customizes a single AI Config in agent mode, while the agents() function customizes multiple AI Configs in agent mode.

The instructions returned by the SDK come directly from the instructions you define for the AI Config variation in the LaunchDarkly UI. The goal or task shown in the UI is delivered unchanged as the instructions field in the SDK.

Customization requires an AI Config key, a fallback value, optional variables, and a context. Both functions perform the evaluation and return the customized instructions along with a tracker instance for recording metrics. If the SDK cannot perform the evaluation or LaunchDarkly is unreachable, it returns the fallback value.

For example, you might use an empty, disabled agent-mode configuration as a fallback value and handle the disabled case in your application.

Here is an example of customizing an AI Config in agent mode:

Customize an AI Config in agent mode

1 agent_config = LDAIAgentConfig(
2   key='ai-config-key-123abc',
3   default_value=LDAIAgentDefaults(
4     enabled=False
5   ),
6   variables={ 'example_custom_variable': 'example_custom_value' }
7 )
8 
9 agent = aiclient.agent(agent_config, context)

To learn more, read agent and agents.

Python AI

Expand Python AI SDK code sample

The customization behavior depends on whether the AI Config uses completion mode or agent mode. You select the mode when you create the AI Config in the LaunchDarkly UI.

Customize AI Configs in completion mode

In completion mode, each AI Config variation includes a single set of roles and messages used to prompt the generative AI model.

Use config() to customize the AI Config. The config() function takes an AI Config key, a context, and a fallback value. It performs the evaluation and returns the customized messages and model along with a tracker instance for recording metrics.

Here is an example of calling the config() method:

Customize an AI Config in completion mode

1 key = 'ai-config-key-123abc'
2 context = Context.builder('context-key-123abc') \
3   .kind('user') \
4   .set('name', 'Sandy') \
5   .build()
6 fallback_value = AIConfig(enabled=False)
7 variables = { 'example_custom_variable': 'example_custom_value' }
8 
9 config, tracker = aiclient.config(key, context, fallback_value, variables)
10 
11 if config.enabled:
12   # Send a request to your AI provider using the customized config
13 else:
14   # Application path to take when the config is disabled

After the function call, you can view the context that you provided to it on the Contexts list.

To learn more, read config.

Customize AI Configs in agent mode

In agent mode, each AI Config variation includes a set of instructions that enable multi-step workflows.

For example, you might use an empty, disabled agent-mode configuration as a fallback value and handle the disabled case in your application.

Here is an example of customizing an AI Config in agent mode:

Customize an AI Config in agent mode

1 agent_config = LDAIAgentConfig(
2   key='ai-config-key-123abc',
3   default_value=LDAIAgentDefaults(
4     enabled=False
5   ),
6   variables={ 'example_custom_variable': 'example_custom_value' }
7 )
8 
9 agent = aiclient.agent(agent_config, context)

Ruby AI

Expand Ruby AI SDK code sample

Use the config function to customize an AI Config. Customization substitutes context attributes and optional variables into the prompt messages defined in the AI Config variation. You must call config each time you generate content from your AI model.

The config function takes an AI Config key, a Context, a fallback value, and optional variables for substitution. It performs the evaluation and returns the customized prompt and model along with a tracker instance for recording metrics. You can pass the customized prompt directly to your AI provider.

Here is an example of calling the config method:

1 key = 'ai-config-key-123abc'
2 context = LaunchDarkly::LDContext.create({ key: 'example-user-key', kind: 'user', name: 'Sandy' })
3 fallback_value = LaunchDarkly::AI::AIConfig.new(enabled: false)
4 variables = { 'example_custom_variable' => 'example_custom_value' }
5 
6 ai_config = ai_client.config(key, context, fallback_value, variables)
7 
8 if ai_config.enabled
9   # Send a request to your AI provider using the customized config
10 else
11   # Application path to take when the AI Config is disabled
12 end