About annotations
An annotation records your review of a span. You can add an annotated span to a dataset to turn real production agent behavior into dataset rows. An annotation can include the following:- A correctness verdict that marks the span as correct or incorrect
- A comment that explains your verdict
- An expected output, which is the answer the span should have produced
Prerequisites
Before you annotate a span, you need the following:- The LaunchDarkly Observability plugin, set up so spans reach LaunchDarkly. To learn more, read LLM observability.
- A role that includes the
viewTraceandupdateTraceAnnotationactions. To learn more, read Role actions.
Spans you can annotate
You can annotate spans for model calls, tool calls, and agent runs that belong to an environment. You cannot annotate embedding spans or LLM judge spans. A verdict on a whole trace is an annotation on the root span of the trace. You cannot add a trace-level verdict when the root span is a type you cannot annotate.Review a span
To review a span:- Click Observability > Explore.
- Click the Traces tab.
- In the Conversations tab, click a conversation name. The conversation’s side panel appears.
- Click the thumbs-up icon if the span is correct.
- Click the thumbs-down icon if the span is incorrect, then edit the expected output. The expected output field accepts up to 1 MiB.
- (Optional) Click the comment icon and enter why you marked a span correct or incorrect.
- Click Add to dataset to save your review and add the span to a dataset you choose. Reviews are only saved when you add them to a dataset.
Comments length limitA comment can contain up to 4,000 characters.