Skip to main content
This topic explains how to review a span in a trace by marking it correct or incorrect, adding a comment, and writing the expected output.

About annotations

An annotation records your review of a span. You can add an annotated span to a dataset to turn real production agent behavior into dataset rows. An annotation can include the following:
  • A correctness verdict that marks the span as correct or incorrect
  • A comment that explains your verdict
  • An expected output, which is the answer the span should have produced
To learn how to add a reviewed span to a dataset, read Datasets from traces.

Prerequisites

Before you annotate a span, you need the following:
  • The LaunchDarkly Observability plugin, set up so spans reach LaunchDarkly. To learn more, read LLM observability.
  • A role that includes the viewTrace and updateTraceAnnotation actions. To learn more, read Role actions.

Spans you can annotate

You can annotate spans for model calls, tool calls, and agent runs that belong to an environment. You cannot annotate embedding spans or LLM judge spans. A verdict on a whole trace is an annotation on the root span of the trace. You cannot add a trace-level verdict when the root span is a type you cannot annotate.
Some spans with gen_ai attributes cannot be annotatedNot every span with gen_ai attributes can be annotated. You cannot annotate embedding spans or LLM judge spans.

Review a span

To review a span:
  1. Click Observability > Explore.
  2. Click the Traces tab.
  3. In the Conversations tab, click a conversation name. The conversation’s side panel appears.
    • Click the thumbs-up icon if the span is correct.
    • Click the thumbs-down icon if the span is incorrect, then edit the expected output. The expected output field accepts up to 1 MiB.
    • (Optional) Click the comment icon and enter why you marked a span correct or incorrect.
  4. Click Add to dataset to save your review and add the span to a dataset you choose. Reviews are only saved when you add them to a dataset.
Comments length limitA comment can contain up to 4,000 characters.
To learn more, read Datasets from traces.

Annotate with the REST API

You can also save an annotation with the LaunchDarkly REST API without adding the span to a dataset. To learn more, read the following pages: