Getting started with OpenAI and AgentControl configs

Overview

This guide shows how to connect an OpenAI-powered application to LaunchDarkly AgentControl. OpenAI models are widely adopted for chat completions, function calling, and structured outputs. By the end, you will be able to manage your model configuration and prompts outside of your application code, and track metrics automatically.

AgentControl supports two modes:

• Completion mode returns messages and roles (system, user, assistant). Use it for chat-style interactions and message-oriented workflows. Completion mode supports online evaluations with judges attached in the LaunchDarkly UI.
• Agent mode returns a single instructions string. Use it when your runtime or framework expects a goal/instructions input for a structured workflow. Agent mode changes the configuration shape from messages to instructions. Your application maps these instructions into your provider or framework’s native input.

Both modes support tool calling. This guide walks through completion mode as the main path, with an optional agent config section. To learn more about when to use each mode, read When to use completion mode vs agent mode.

This guide provides examples in both Python and Node.js (TypeScript).

Prerequisites

To complete this guide, you need the following:

• A LaunchDarkly account with an SDK key for your environment and a member role that allows AgentControl actions. To learn more about LaunchDarkly roles, read Roles.
• An OpenAI API key. Create one at platform.openai.com.
• A development environment:
  • Python: Python 3.10 or higher
  • Node.js: Node.js 20 or higher
• Familiarity with LaunchDarkly contexts. To learn more, read Contexts and segments.

Concepts

Before you begin, review these key concepts.

AgentControl configs

An AgentControl config is a LaunchDarkly resource that controls how your application uses large language models. Each AgentControl config contains one or more variations. Each variation specifies:

• A model configuration, including the model name and parameters
• Messages that define the prompt

You can update these settings in LaunchDarkly at any time without changing your application code.

Contexts

A context represents the end user interacting with your application. LaunchDarkly uses context attributes to:

• Determine which variation to serve based on targeting rules
• Populate {{ ldctx.* }} placeholders in your prompts with context attribute values

Other placeholders, such as {{ topic }}, are populated from the variables argument you pass at runtime.

The tracker

When you retrieve an AgentControl config, call createTracker() (Node.js) or create_tracker() (Python) on the returned object to mint a tracker. The tracker records metrics from your OpenAI calls, including:

• Generation count
• Input and output tokens
• Latency
• Success and error rates

These metrics appear on the AI Insights dashboard in LaunchDarkly.

Step 1: Install the SDK

Install the LaunchDarkly AI SDK and the OpenAI SDK in your application. AgentControl is supported by LaunchDarkly server-side SDKs only. The Node.js examples in this guide use the server-side Node.js AI SDK.

Here is how to install the required packages:

$
pip install "launchdarkly-server-sdk-ai>=0.20.0"
$
pip install "launchdarkly-server-sdk-ai-openai>=0.6.0"
$
pip install openai
$
pip install python-dotenv

Create a .env file in your project root to store your API keys:

$
# .env
$
LAUNCHDARKLY_SDK_KEY=<your-launchdarkly-sdk-key>
$
OPENAI_API_KEY=<your-openai-api-key>

Add .env to your .gitignore to keep credentials out of version control.

Step 2: Initialize the clients

Initialize both the LaunchDarkly client and the OpenAI client. Store your API keys in environment variables.

Here is the initialization code:

1
import os
2
import json
3
import ldclient
4
from ldclient import Context
5
from ldclient.config import Config
6
from ldai import LDAIClient, AICompletionConfigDefault, AIAgentConfigDefault
7
from ldai.providers.types import LDAIMetrics
8
from ldai.tracker import TokenUsage
9
from ldai_openai import get_ai_metrics_from_response
10
from openai import OpenAI
11
from dotenv import load_dotenv
12
13
load_dotenv()
14
15
OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY")
16
SDK_KEY = os.environ.get("LAUNCHDARKLY_SDK_KEY_OPENAI")
17
CONFIG_KEY = "openai-assistant"
18
AGENT_CONFIG_KEY = "openai-agent"
19
20
openai_client = OpenAI(api_key=OPENAI_API_KEY)
21
ldclient.set_config(Config(SDK_KEY))
22
if not ldclient.get().is_initialized():
23
    exit(1)
24
25
ai_client = LDAIClient(ldclient.get())

Step 3: Create an AgentControl config in LaunchDarkly

Create an AgentControl config in the LaunchDarkly UI to store your OpenAI model settings and prompts.

To create the AgentControl config:

1. Click Create and select AgentControl config.
2. In the Create AgentControl config dialog, select Completion.
3. Enter a name, such as “OpenAI assistant”.
4. Click Create.

To create a variation:

1. On the Variations tab, replace “Untitled variation” with a name, such as “GPT-4o”.
2. Click Select a model and choose the gpt-4o OpenAI model.
3. Click Parameters and set temperature to 0.7 and max_tokens to 1024.
4. Add a system message to define your assistant’s behavior:

You are a helpful assistant. Answer questions about {{topic}}.

5. Click Review and save.

To enable targeting:

1. Select the Targeting tab.
2. In the “Default rule” section, click Edit.
3. Set the default rule to serve your variation.
4. Click Review and save.

Step 4: Get the AgentControl config in your application

Retrieve the AgentControl config from LaunchDarkly by calling the completion config function. Pass a context that represents the current user. A fallback configuration is optional — if LaunchDarkly is unreachable and no fallback is provided, the config returns with enabled: false.

Here is how to get the AgentControl config:

1
# Define the context for the current user
2
context = Context.builder("user-123") \
3
    .kind("user") \
4
    .name("Sandy") \
5
    .build()
6
7
# Pass a default for improved resiliency when the AgentControl config is unavailable
8
# or LaunchDarkly is unreachable; omit for a disabled default.
9
# Example:
10
#   from ldai import AICompletionConfigDefault
11
#   default = AICompletionConfigDefault(
12
#       enabled=True,
13
#       model={"name": "gpt-5"},
14
#       provider={"name": "openai"},
15
#       messages=[{"role": "system", "content": "You are a helpful assistant."}],
16
#   )
17
#   config = ai_client.completion_config(CONFIG_KEY, context, default, variables={"topic": "Python"})
18
19
# Get the AgentControl config
20
config = ai_client.completion_config(CONFIG_KEY, context, variables={"topic": "Python"})
21
tracker = config.create_tracker()

Check the enabled property and handle the disabled case in your application. If you want your application to serve a specific model and prompt when LaunchDarkly is unreachable, pass an AICompletionConfigDefault (Python) or a plain config object (TypeScript) as the third argument — see the commented fallback example above.

Step 5: Call OpenAI and track metrics

OpenAI’s Chat Completions API expects messages in an array, including system-role messages. Unlike Anthropic, there is no separate top-level system parameter. Instead, the system prompt is a message with role: "system" in the messages array. Use the generic track_metrics_of (Python) or trackMetricsOf (Node.js) helper with get_ai_metrics_from_response (Python) or getAIMetricsFromResponse (Node.js) from the launchdarkly-server-sdk-ai-openai / @launchdarkly/server-sdk-ai-openai package. The helper records duration, success/error, and token usage automatically.

LaunchDarkly variations store OpenAI-native parameter names (max_completion_tokens, temperature, top_p, and so on), so they pass directly to the SDK. You do need to drop tools from params before spreading them — attached tools pass through on the top-level tools argument, and leaving them in params would duplicate the argument. Define a small helper for that:

1
def drop_tools(params):
2
    """Drop `tools` from params — attached tools are passed via the top-level
3
    `tools=` argument, so leaving them in params would duplicate the argument."""
4
    return {k: v for k, v in (params or {}).items() if k != "tools"}

Then call OpenAI with the config:

1
if config.enabled:
2
    tracker = config.create_tracker()
3
    messages = config.messages or []
4
5
    chat_messages = [m.to_dict() for m in messages]
6
    chat_messages.append({"role": "user", "content": "How do I read a file in Python?"})
7
8
    ld_params = (config.model.to_dict().get("parameters") if config.model else None) or {}
9
10
    # The helper lives in the launchdarkly-server-sdk-ai-openai package.
11
    completion = tracker.track_metrics_of(
12
        get_ai_metrics_from_response,
13
        lambda: openai_client.chat.completions.create(
14
            model=config.model.name,
15
            messages=chat_messages,
16
            **drop_tools(ld_params),
17
        ),
18
    )

Step 6: Optional: Use agent mode with tool calling

Agent-mode AgentControl configs return a single instructions string instead of a message list, and they let you attach reusable tools from the LaunchDarkly tools library. This step uses OpenAI’s Responses API for agent mode. The Responses API is OpenAI’s agent-oriented surface and accepts LaunchDarkly’s flat {type, name, description, parameters} tool shape directly — no per-call conversion. It also manages conversation state server-side through previous_response_id, so each turn only sends the new input (the initial user message, or the tool outputs from the previous turn).

Create the tool in the tools library

First, define the tool in LaunchDarkly so the AgentControl config variation can reference it:

1. In the left navigation, click Library, then select the Tools tab.
2. Click Add tool.
3. Enter get_order_status as the key.
4. Enter “Look up the status of a customer order by order ID” as the description.
5. Define the schema using the JSON editor:

1
{
2
  "type": "object",
3
  "properties": {
4
    "order_id": {
5
      "type": "string",
6
      "description": "The order ID to look up"
7
    }
8
  },
9
  "required": ["order_id"]
10
}

6. Click Save.

Create the agent AgentControl config

1. Click Create and select AgentControl config.
2. Select Agent mode.

3. Enter a name:

OpenAI agent

4. Click Create.
5. On the Variations tab, name the variation (for example, “GPT-4o agent”).
6. Click Select a model and choose gpt-4o.
7. Click Parameters and set max_tokens to 1024.
8. Add the agent instructions:

You are an order status assistant. Use the get_order_status tool to look up customer orders by ID. Only call the tool when the user asks about an order.

9. Click + Attach tools and select get_order_status.

10. Click Review and save.
11. On the Targeting tab, set the default rule to serve your variation and save.

To learn more about managing tools, read Tools in AgentControl.

Retrieve the agent config and run the tool loop

Use agent_config() instead of completion_config(). The SDK returns attached tools under parameters.tools in LaunchDarkly’s flat shape, so convert them to OpenAI’s nested {type: 'function', function: {name, description, parameters}} shape before passing them on the tools= argument. Tool handler functions stay in your application code. LaunchDarkly stores the schema, while your application owns the behavior.

Because the Responses API has a different response shape than Chat Completions, use tracker.track_metrics_of (Python) / tracker.trackMetricsOf (TypeScript) with a provider-specific converter function. The converter receives the raw API response and returns an LDAIMetrics object; the tracker handles duration and success/error itself.

1
def responses_metrics(response) -> LDAIMetrics:
2
    """Convert an OpenAI Responses API result into LDAIMetrics. Passed to
3
    tracker.track_metrics_of — which handles duration + success/error itself.
4
    Parses the Responses output shape: response.output[] and response.usage.*_tokens."""
5
    usage = getattr(response, "usage", None)
6
    tokens = None
7
    if usage:
8
        tokens = TokenUsage(
9
            total=usage.total_tokens or 0,
10
            input=usage.input_tokens or 0,
11
            output=usage.output_tokens or 0,
12
        )
13
    return LDAIMetrics(success=True, tokens=tokens)
14
15
16
# Same fallback pattern as completion — omit the default for a disabled fallback.
17
agent = ai_client.agent_config(AGENT_CONFIG_KEY, context)
18
19
if agent.enabled:
20
    tracker = agent.create_tracker()
21
    ld_params = (agent.model.to_dict().get("parameters") if agent.model else None) or {}
22
    # Tools attached to the variation in LD are already in the Responses API's
23
    # flat shape — pass them straight through.
24
    tools = ld_params.get("tools", []) or []
25
26
    # Handler stays in app code; LD governs the schema.
27
    def get_order_status(order_id: str) -> str:
28
        orders = {
29
            "ORD-123": "Shipped — arrives Thursday",
30
            "ORD-456": "Processing — estimated ship date: tomorrow",
31
            "ORD-789": "Delivered on Monday",
32
        }
33
        return orders.get(order_id, f"No order found with ID {order_id}")
34
35
    tool_handlers = {"get_order_status": get_order_status}
36
37
    # Responses API agent loop: chain turns via previous_response_id so OpenAI
38
    # keeps track of conversation state for us — each turn we only send the
39
    # new input (the initial user message, or the function_call_outputs for
40
    # the previous turn's tool calls).
41
    next_input = [{"role": "user", "content": "What's the status of order ORD-123?"}]
42
    previous_response_id = None
43
44
    MAX_STEPS = 5
45
    for _ in range(MAX_STEPS):
46
        response = tracker.track_metrics_of(
47
            responses_metrics,
48
            lambda: openai_client.responses.create(
49
                model=agent.model.name,
50
                instructions=agent.instructions,
51
                input=next_input,
52
                tools=tools,
53
                previous_response_id=previous_response_id,
54
                **drop_tools(ld_params),
55
            ),
56
        )
57
        previous_response_id = response.id
58
59
        function_calls = [item for item in response.output if item.type == "function_call"]
60
        if not function_calls:
61
            break
62
63
        next_input = []
64
        for call in function_calls:
65
            args = json.loads(call.arguments)
66
            if call.name not in tool_handlers:
67
                raise ValueError(f"Unknown tool: {call.name}")
68
            result = tool_handlers[call.name](**args)
69
            tracker.track_tool_call(call.name)
70
            next_input.append({
71
                "type": "function_call_output",
72
                "call_id": call.call_id,
73
                "output": result,
74
            })

In completion mode, you can attach judges to variations in the LaunchDarkly UI for automatic evaluation. In agent mode, invoke judges programmatically through the AI SDK.

To learn more, read When to use completion mode vs agent mode and Agents.

Step 7: Monitor your AgentControl config

Use the LaunchDarkly UI to monitor how your applications are performing across all AgentControl configs and for individual configs.

To view aggregated metrics across all your AgentControl configs, navigate to Insights in the left navigation under the AI section. The Insights overview page displays cost, latency, error rate, invocation counts, and model distribution across your organization. To learn more, read AI Insights.

To view metrics for a specific AgentControl config:

1. Navigate to your AgentControl config.
2. Select the Monitoring tab.

The dashboard displays the following metrics:

• Generation count: The total number of AI generation calls tracked for this config.
• Input and output tokens: Token consumption broken down by prompt tokens sent and completion tokens received.
• Latency: The time taken for each generation call, shown as percentiles (p50, p95).
• Success and error rates: The proportion of successful versus failed generation calls.

Metrics update approximately every minute. Use these metrics to compare variations and optimize your prompts. To learn more, read Monitor AgentControl configs.

Step 8: Flush events and close the client

Always flush events before closing the LaunchDarkly client when your application shuts down — trailing events are at risk of being lost otherwise, in short-lived scripts and long-running services alike.

Here is how to flush events and close the client:

1
# Always flush events before closing — trailing events are at risk of being
2
# lost otherwise, in short-lived scripts and long-running services alike.
3
ldclient.get().flush()
4
ldclient.get().close()

Complete example

Here is a complete working example that combines all the steps.

1
import os
2
import json
3
import ldclient
4
from ldclient import Context
5
from ldclient.config import Config
6
from ldai import LDAIClient, AICompletionConfigDefault, AIAgentConfigDefault
7
from ldai.providers.types import LDAIMetrics
8
from ldai.tracker import TokenUsage
9
from ldai_openai import get_ai_metrics_from_response
10
from openai import OpenAI
11
from dotenv import load_dotenv
12
13
load_dotenv()
14
15
OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY")
16
SDK_KEY = os.environ.get("LAUNCHDARKLY_SDK_KEY_OPENAI")
17
CONFIG_KEY = "openai-assistant"
18
AGENT_CONFIG_KEY = "openai-agent"
19
20
21
def drop_tools(params):
22
    """Drop `tools` from params — attached tools are passed via the top-level
23
    `tools=` argument, so leaving them in params would duplicate the argument."""
24
    return {k: v for k, v in (params or {}).items() if k != "tools"}
25
26
27
def responses_metrics(response) -> LDAIMetrics:
28
    """Convert an OpenAI Responses API result into LDAIMetrics. Passed to
29
    tracker.track_metrics_of — which handles duration + success/error itself.
30
    Parses the Responses output shape: response.output[] and response.usage.*_tokens."""
31
    usage = getattr(response, "usage", None)
32
    tokens = None
33
    if usage:
34
        tokens = TokenUsage(
35
            total=usage.total_tokens or 0,
36
            input=usage.input_tokens or 0,
37
            output=usage.output_tokens or 0,
38
        )
39
    return LDAIMetrics(success=True, tokens=tokens)
40
41
42
def main():
43
    openai_client = OpenAI(api_key=OPENAI_API_KEY)
44
45
    ldclient.set_config(Config(SDK_KEY))
46
    if not ldclient.get().is_initialized():
47
        return
48
49
    ai_client = LDAIClient(ldclient.get())
50
    context = Context.builder("user-123").kind("user").name("Sandy").build()
51
52
    # ===================
53
    # COMPLETION MODE — Chat Completions API
54
    # ===================
55
56
    # Pass a default for improved resiliency when the AgentControl config is unavailable
57
    # or LaunchDarkly is unreachable; omit for a disabled default.
58
    # Example:
59
    #   default = AICompletionConfigDefault(
60
    #       enabled=True,
61
    #       model={"name": "gpt-5"},
62
    #       provider={"name": "openai"},
63
    #       messages=[{"role": "system", "content": "You are a helpful assistant."}],
64
    #   )
65
    #   config = ai_client.completion_config(CONFIG_KEY, context, default, variables={"topic": "Python"})
66
    config = ai_client.completion_config(CONFIG_KEY, context, variables={"topic": "Python"})
67
68
    if config.enabled:
69
        tracker = config.create_tracker()
70
        chat_messages = [m.to_dict() for m in (config.messages or [])]
71
        chat_messages.append({"role": "user", "content": "How do I read a file in Python?"})
72
73
        ld_params = (config.model.to_dict().get("parameters") if config.model else None) or {}
74
75
        # The helper lives in the launchdarkly-server-sdk-ai-openai package.
76
        tracker.track_metrics_of(
77
            get_ai_metrics_from_response,
78
            lambda: openai_client.chat.completions.create(
79
                model=config.model.name,
80
                messages=chat_messages,
81
                **drop_tools(ld_params),
82
            ),
83
        )
84
85
    # ===================
86
    # AGENT MODE — Responses API (OpenAI's preferred agent surface;
87
    # flat tool shape matches LD's native tool format, no conversion needed)
88
    # ===================
89
90
    # Same fallback pattern as completion — omit the default for a disabled fallback.
91
    agent = ai_client.agent_config(AGENT_CONFIG_KEY, context)
92
93
    if agent.enabled:
94
        tracker = agent.create_tracker()
95
        ld_params = (agent.model.to_dict().get("parameters") if agent.model else None) or {}
96
        # Tools attached to the variation in LD are already in the Responses API's
97
        # flat shape — pass them straight through.
98
        tools = ld_params.get("tools", []) or []
99
100
        def get_order_status(order_id: str) -> str:
101
            orders = {
102
                "ORD-123": "Shipped — arrives Thursday",
103
                "ORD-456": "Processing — estimated ship date: tomorrow",
104
                "ORD-789": "Delivered on Monday",
105
            }
106
            return orders.get(order_id, f"No order found with ID {order_id}")
107
108
        tool_handlers = {"get_order_status": get_order_status}
109
110
        # Responses API agent loop: chain turns via previous_response_id so OpenAI
111
        # keeps track of conversation state for us.
112
        next_input = [{"role": "user", "content": "What's the status of order ORD-123?"}]
113
        previous_response_id = None
114
115
        MAX_STEPS = 5
116
        for _ in range(MAX_STEPS):
117
            response = tracker.track_metrics_of(
118
                responses_metrics,
119
                lambda: openai_client.responses.create(
120
                    model=agent.model.name,
121
                    instructions=agent.instructions,
122
                    input=next_input,
123
                    tools=tools,
124
                    previous_response_id=previous_response_id,
125
                    **drop_tools(ld_params),
126
                ),
127
            )
128
            previous_response_id = response.id
129
130
            function_calls = [item for item in response.output if item.type == "function_call"]
131
            if not function_calls:
132
                break
133
134
            next_input = []
135
            for call in function_calls:
136
                args = json.loads(call.arguments)
137
                if call.name not in tool_handlers:
138
                    raise ValueError(f"Unknown tool: {call.name}")
139
                result = tool_handlers[call.name](**args)
140
                tracker.track_tool_call(call.name)
141
                next_input.append({
142
                    "type": "function_call_output",
143
                    "call_id": call.call_id,
144
                    "output": result,
145
                })
146
147
    # Always flush events before closing — trailing events are at risk of being
148
    # lost otherwise, in short-lived scripts and long-running services alike.
149
    ldclient.get().flush()
150
    ldclient.get().close()
151
152
153
if __name__ == "__main__":
154
    main()

What to explore next

After you have the basic integration working, you can extend it with:

• Tools for calling external functions from your workflows
• Online evaluations to score response quality automatically
• Experiments to compare AgentControl config variations statistically
• Agents for multi-step workflows

For more AgentControl guides, read the other guides in the AgentControl guides section.

Troubleshooting

Metrics not appearing

If metrics do not appear on the AI Insights dashboard:

• Verify that you are calling tracker.track_metrics_of() (Python) or tracker.trackMetricsOf() (Node.js) with the correct metrics converter.
• Ensure you call flush() before closing the client, especially for short-lived scripts.
• Wait at least one minute for metrics to process.

SDK initialization failures

If the LaunchDarkly SDK fails to initialize:

• Verify your SDK key is correct and matches the environment you are targeting.
• Check that your network can reach LaunchDarkly servers.
• Review the SDK logs for specific error messages.

Config returns fallback value

If you always receive the fallback configuration:

• Verify targeting is enabled for your AgentControl config.
• Check that the AgentControl config key in your code matches the key in LaunchDarkly.
• Ensure your context matches the targeting rules.

OpenAI API errors

If you receive OpenAI API errors:

• Verify your OPENAI_API_KEY is set correctly.
• Check that your API key has sufficient permissions.
• Ensure the model name in your AgentControl config matches an available OpenAI model.

Conclusion

In this guide, you connected an OpenAI-powered application to LaunchDarkly AgentControl. You can now:

• Manage prompts and model settings in LaunchDarkly without code changes
• Track token usage, latency, and success rates automatically
• Use template variables to customize prompts per user