Skip to main content
This topic explains how to add a reviewed span from a trace to a dataset and what the resulting dataset row contains.

How adding a span works

When you add a span to a dataset, LaunchDarkly saves your review and creates a dataset row in one action. LaunchDarkly does not keep the review in the UI when you do not add the span to a dataset. To learn more about reviewing a span, read Span annotations. To learn more about using rows after you add them, read Datasets.

Prerequisites

Before you add a span to a dataset, you need the following:
  • The LaunchDarkly Observability plugin, set up so spans reach LaunchDarkly. To learn more, read LLM observability.
  • A reviewed span. To learn more, read Span annotations.
  • The updateAIDataset action on the destination dataset.
  • The viewTrace action on the source trace.
Adding a span requires two permissionsGranting only the AI dataset actions is not enough. Adding a span to a dataset fails when a member does not have viewTrace action on the source trace. To learn more, read Role actions.

Add a span to a dataset

To add a span to a dataset:
  1. Click Observability > Explore.
  2. Click the Traces tab.
  3. In the Conversations tab, click a conversation name. The conversation’s side panel appears.
    • Click the thumbs-up icon if the span is correct.
    • Click the thumbs-down icon if the span is incorrect, then edit the expected output. The expected output field accepts up to 1 MiB.
    • (Optional) Click the comment icon and enter why you marked a span correct or incorrect.
  4. Click Add to dataset to save your review and add the span to a dataset you choose. Reviews are only saved when you add them to a dataset.

View a dataset row’s metadata

To view a dataset row’s metadata:
  1. Click Agent management > Library.
  2. Click the Datasets tab.
  3. Choose the dataset you want to review.
  4. Click the relevant row in the dataset. The “Edit row” side panel appears with the metadata listed in the panel.
A row’s metadata can include the following lineage keys:
  • source_trace_id: The trace that contains the span you added.
  • source_span_id: The span you added.
  • environment_key: The environment you added the span from.
  • source_config_key: The config that ran. Present only when the trace records it.
  • source_variation_key: The variation that ran. Present only when the trace records it.
LaunchDarkly flattens nested values into dotted keys, such as the following:
  • annotation.annotation_id: The annotation applied to the span. Present only when the span has an annotation.
  • annotation.annotated_by: The member who reviewed the span. Present only when the span has an annotation.
  • annotation.annotated_at: When the member reviewed the span. Present only when the span has an annotation.
  • annotation.correctness: Whether the reviewer marked the span correct or incorrect. Present only when the reviewer set a verdict.
  • annotation.explanation: The reviewer’s comment. Present only when the reviewer added a comment.
When a span makes tool calls, the metadata also includes a tool_calls key. This key is not part of the lineage, but it appears alongside the lineage keys.

Limitations

Adding a span to a dataset has the following limitations:
  • LaunchDarkly does not support variable overrides. Adding a span stores the rendered prompt as the input and does not store template variables.
  • You can add only spans that you can annotate. To learn more, read Annotate spans.

Troubleshoot adding a span

When adding a span to a dataset fails, LaunchDarkly shows an error with a Retry button. The error reads “Couldn’t add this span to,” followed by the dataset name and the reason the add failed. If you have the AI dataset actions but can’t add a span, check that your role also includes viewTrace on the source trace.

Errors you can fix

This table explains the errors you can fix and how to resolve them:
ErrorCauseResolution
CONTENT_NOT_CAPTUREDThe instrumentation recorded the model and tool calls, but not the prompts and replies. Most OpenTelemetry generative AI instrumentation turns off message content capture by default.Turn on message content capture, then add a span from a new trace. Retrying with the same trace does not work.
TRACE_ENVIRONMENT_MISMATCHThe trace belongs to a different environment than the one you are adding it from.Add the span from the trace’s own environment.
TRACE_TELEMETRY_NOT_FOUNDLaunchDarkly Observability can’t return the trace. Usually, the trace is older than your retention period. The trace might also be in another project, or have a timestamp outside the lookup window.Add a span from a trace that is within your retention period, in the current project.
DATASET_NOT_FOUNDSomeone deleted the dataset while you were adding the span.Choose another dataset.

Traces you can’t add

Some errors mean you can’t add the trace to a dataset, and changing your setup does not resolve them. This table explains the errors for traces you can’t add:
ErrorCause
INPUT_MISSINGThe trace has no user prompt to use as the row’s input. For example, caching can drop the prompt from a trace.
UNSUPPORTED_TELEMETRY_SOURCELaunchDarkly can’t parse the spans because they come from a framework that LaunchDarkly does not support yet.
MALFORMED_TELEMETRYLaunchDarkly can’t parse the spans because they are malformed.
OUTPUT_TOO_LARGEThe expected output is larger than 1 MiB. The UI blocks expected outputs over this limit before you submit, so this error is rare.

LaunchDarkly errors

The following errors come from LaunchDarkly, not from your trace or setup:
  • TRANSIENT_ERROR
  • OBSERVABILITY_AUTH_FAILED
  • OBSERVABILITY_BAD_REQUEST
  • OBSERVABILITY_INVALID_ARGUMENT
  • TRANSFORM_FAILED
To resolve these errors, click Retry. If the error continues, start a Support ticket. The following messages can also appear when you add a span:
  • “Couldn’t save the review, so nothing was added to [DATASET NAME],”: If LaunchDarkly can’t save your review, it doesn’t add the span to the dataset.
  • “This span is already in [DATASET NAME]”: If the span is already in the dataset, LaunchDarkly asks whether to replace the existing row instead of showing an error.

Add traces with the REST API

You can also add a span to a dataset with the LaunchDarkly REST API. Adding a span is a job that runs in the background. You start the job, then poll it until it finishes. The REST API requires the same permissions as the UI. To learn more, read Prerequisites. This table explains the endpoints:
MethodEndpointDescription
POST/api/v2/projects/PROJECT_KEY/environments/ENVIRONMENT_KEY/datasets/DATASET_KEY/trace-promotionsStarts adding a span to the dataset. Returns a 202 status code and a job to poll.
GET/api/v2/projects/PROJECT_KEY/environments/ENVIRONMENT_KEY/datasets/DATASET_KEY/trace-promotions/JOB_IDReturns the status of a job. A failed job returns a 200 status code with status: failed and an error, not a 5xx status code.
The POST request body accepts the following fields:
  • traceId: The ID of the trace that contains the span.
  • timestamp: The timestamp of the trace.
  • spanId: (Optional) The ID of the span to add.
  • annotationId: (Optional) The ID of the annotation to apply to the row.
  • overwriteExisting: (Optional) Set to true to replace an existing row for the span.
In addition to the errors in Troubleshoot adding a span, the REST API can return the following errors:
  • DATASET_ROW_ALREADY_EXISTS: The span already has a row in the dataset, and overwriteExisting is set false.
  • ANNOTATION_SPAN_MISMATCH: The annotationId belongs to a different span.
  • ANNOTATION_NOT_FOUND: The annotation was deleted after the job started.
To learn more, read the following pages: