> ## Documentation Index
> Fetch the complete documentation index at: https://launchdarkly.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Expected output mode

<View title="Developer" />

<View title="Federal docs" />

<View title="EU docs" />

This topic explains how to use Expected output mode to optimize an agent against known input/output pairs. This lets you optimize an agent against its expected outputs.

These optimizations compare the output of a given invocation against both the acceptance criteria and judges as well as the provided output to ensure it conforms to expectations.

## How Expected output mode works

When you run an optimization on ground truth responses, it's important to ensure that the optimization result takes into account and scores all expected input/variable/output pairs.

To accomplish this, Expected output mode does the following things:

1. The optimization scores each triplet of input, output, and variables against each of the judges.
2. The system collects judge responses. If any results fail, it generates a new candidate variation or agent.
3. The system scores the new candidate against each of the inputs again.

This process repeats until either:

* A successful candidate passes all input triplets.
* Or the run reaches the maximum number of attempts.

After a candidate succeeds, the system returns the candidate variation or agent. If you use the `optimize_from_config` method or `auto_commit` is true for the `*_from_options` methods, the system pushes that candidate to the agent config as a new variation.

<Info>
  **Expected output mode can use more resources.**

  Because it mode requires complete executions of all available tests and may require multiple runs to develop a variation that passes, Expected output mode has a much higher ceiling for the number of possible LLM provider calls the system makes, which scales with your number of inputs.
</Info>

## Code examples

There are two ways to use Expected output mode:

* With the [`optimize_from_ground_truth_options` method](#from-options).
* Or [From an optimization config](#from-an-optimization-config).

### From options

To use the Expected output mode, use the `optimize_from_ground_truth_options` method and include a `ground_truth_responses` array that collates the inputs together rather than the raw lists we use otherwise.

Here's how:

<CodeGroup>
  ```python title="Python" expandable lines wrap theme={null}
  # ...
  options = GroundTruthOptimizationOptions(
          context_choices=[ld_context],
          max_attempts=5,
          model_choices=["claude-opus-4-5", "claude-haiku-4-5"],
          judge_model="claude-opus-4-5",
          handle_agent_call=handle_llm_call,
          handle_judge_call=handle_llm_call,
          judges={
              "relevance": OptimizationJudge(
                  judge_key="my-relevance-judge",
                  threshold=0.8,
              ),
          },
          ground_truth_responses=[
              GroundTruthSample(
                  user_input="I'm going to tokyo next week, where should I stay near Shinjuku?",
                  expected_response="Recommend hotels in the Shinjuku area of Tokyo.",
                  variables={"city": "Tokyo", "region": "Shinjuku"},
              ),
              GroundTruthSample(
                  user_input="where to eat in anchorage",
                  expected_response="Recommend restaurants in Anchorage, Alaska.",
                  variables={"city": "Anchorage", "region": "Downtown"},
              ),
              GroundTruthSample(
                  user_input="airbnbs near tahoe",
                  expected_response="Recommend Airbnb rentals near Lake Tahoe.",
                  variables={"city": "Lake Tahoe", "region": "South Shore"},
              ),
              GroundTruthSample(
                  user_input="what are some food options in sf near the airport",
                  expected_response="Recommend restaurants near SFO in San Francisco.",
                  variables={"city": "San Francisco", "region": "SFO"},
              ),
          ],
      )

  # ...

  result = await optimization_client.optimize_from_ground_truth_options("your-agent-key", options)
  ```
</CodeGroup>

### From an optimization config

When operating from a config directly, LaunchDarkly automatically infers the type of invocation you wish to use. The implementation configuration is the same regardless of which invocation occurs.

Here's how:

<CodeGroup>
  ```python title="Python" lines wrap theme={null}
  result = await optimization_client.optimize_from_config(
      "my-optimization-config-key",
      OptimizationFromConfigOptions(
          project_key="my-project",
          context_choices=[ld_context],
          handle_agent_call=handle_llm_call,
          handle_judge_call=handle_llm_call,
      ),
  )
  ```
</CodeGroup>