Using tools in agent optimization

This topic explains how to pass tool definitions through to your agent and judge calls during optimization runs.

One of the strengths of using an SDK-based optimization method is being able to run your LLMs exactly as you would in a real-world scenario. This includes being able to use your tool definitions and have them use live, real-world implementations of those tools. Whether this is using built-in tools like web search that can return different results for each query, or endpoints that always return the latest data, an LLM can interpret and process the shape of that data in different ways depending on exactly what gets returned.

How tools work in optimization runs

LaunchDarkly includes attached tools as part of the config, which you can pass into your agent calls. Your instructions and tool_use_behavior parameters, where applicable, largely determine whether or not a tool gets called as part of its execution.

The optimization process does not remove or add tools to your agents. If you think your LLM might need access to different tools, or to optimize against calling different tools, include all tool definitions. The instructions will be optimized toward the use of one or the other depending on how well they suit the optimization goals.

Tools available to your agents are also provided to the evaluation. Acceptance statement criteria receive the same list of tools that the invocation does in case the evaluator thinks it needs to check on underlying data to confirm validity.

Tool definitions

Tool definitions in LaunchDarkly are decorated functions. To learn more, read Tools.

Here’s an example from OpenAI:

1
from agents import function_tool
2
3
# ...
4
@dataclass
5
class GetPreferencesResponse:
6
    some_property: str
7
8
@function_tool
9
async def get_preferences(user_id: str) -> GetPreferencesResponse:
10
    user_preferences = some_function_that_gets_preferences(user_id)
11
    return user_preferences
12
13
# ... OpenAI setup ...
14
root = Agent(
15
    name="my-agent",
16
    instructions="...instructions...",
17
    handoffs=[],
18
    tools=[get_preferences],
19
    model="gpt-5",
20
)
21
22
result = Runner.run(root, "... some user input ...")

Agent config objects from LaunchDarkly can have tools attached and pass the tool payload through with the configuration. To retrieve the tools, you can access them off of the model.parameters.

Here’s how:

1
tools = config.model.get_parameter("tools") if config.model else []

This returns an object containing the tool definitions. Each of these tool definitions has a name as set in the LaunchDarkly UI.

One way to map tools to the definitions is to use a dict containing a mapping of the LaunchDarkly tool keys to the defined handler functions. Here’s an example:

1
tool_handlers = {
2
    "get-preferences": get_preferences
3
}

You can then pass your tools during handle_agent_call and handle_judge_call. Here’s how:

1
def resolve_tools(config: LLMCallConfig) -> list["Tool"]:
2
    tools_param = config.model.get_parameter("tools") if config.model else []
3
    return [tool_handlers[tool["name"]] for tool in tools_param if tool["name"] in tool_handlers]
4
5
def handle_agent_call(
6
    key: str,
7
    config: LLMCallConfig,
8
    context: LLMCallContext,
9
):
10
    root = Agent(
11
        name="my-agent",
12
        instructions="...instructions...",
13
        tools=resolve_tools(config),
14
        model="gpt-5",
15
    )
16
17
    result = Runner.run(root, context.user_input)

This ensures the entire map of tool definitions is available to all of the invocation and evaluation agent calls that have them registered on the LaunchDarkly tools object.