How adding a span works
When you add a span to a dataset, LaunchDarkly saves your review and creates a dataset row in one action. LaunchDarkly does not keep the review in the UI when you do not add the span to a dataset. To learn more about reviewing a span, read Span annotations. To learn more about using rows after you add them, read Datasets.Prerequisites
Before you add a span to a dataset, you need the following:- The LaunchDarkly Observability plugin, set up so spans reach LaunchDarkly. To learn more, read LLM observability.
- A reviewed span. To learn more, read Span annotations.
- The
updateAIDatasetaction on the destination dataset. - The
viewTraceaction on the source trace.
Add a span to a dataset
To add a span to a dataset:- Click Observability > Explore.
- Click the Traces tab.
- In the Conversations tab, click a conversation name. The conversation’s side panel appears.
- Click the thumbs-up icon if the span is correct.
- Click the thumbs-down icon if the span is incorrect, then edit the expected output. The expected output field accepts up to 1 MiB.
- (Optional) Click the comment icon and enter why you marked a span correct or incorrect.
- Click Add to dataset to save your review and add the span to a dataset you choose. Reviews are only saved when you add them to a dataset.
View a dataset row’s metadata
To view a dataset row’s metadata:- Click Agent management > Library.
- Click the Datasets tab.
- Choose the dataset you want to review.
- Click the relevant row in the dataset. The “Edit row” side panel appears with the metadata listed in the panel.
source_trace_id: The trace that contains the span you added.source_span_id: The span you added.environment_key: The environment you added the span from.source_config_key: The config that ran. Present only when the trace records it.source_variation_key: The variation that ran. Present only when the trace records it.
annotation.annotation_id: The annotation applied to the span. Present only when the span has an annotation.annotation.annotated_by: The member who reviewed the span. Present only when the span has an annotation.annotation.annotated_at: When the member reviewed the span. Present only when the span has an annotation.annotation.correctness: Whether the reviewer marked the span correct or incorrect. Present only when the reviewer set a verdict.annotation.explanation: The reviewer’s comment. Present only when the reviewer added a comment.
tool_calls key. This key is not part of the lineage, but it appears alongside the lineage keys.
Limitations
Adding a span to a dataset has the following limitations:- LaunchDarkly does not support variable overrides. Adding a span stores the rendered prompt as the input and does not store template variables.
- You can add only spans that you can annotate. To learn more, read Annotate spans.
Troubleshoot adding a span
When adding a span to a dataset fails, LaunchDarkly shows an error with a Retry button. The error reads “Couldn’t add this span to,” followed by the dataset name and the reason the add failed. If you have the AI dataset actions but can’t add a span, check that your role also includesviewTrace on the source trace.
Errors you can fix
This table explains the errors you can fix and how to resolve them:| Error | Cause | Resolution |
|---|---|---|
CONTENT_NOT_CAPTURED | The instrumentation recorded the model and tool calls, but not the prompts and replies. Most OpenTelemetry generative AI instrumentation turns off message content capture by default. | Turn on message content capture, then add a span from a new trace. Retrying with the same trace does not work. |
TRACE_ENVIRONMENT_MISMATCH | The trace belongs to a different environment than the one you are adding it from. | Add the span from the trace’s own environment. |
TRACE_TELEMETRY_NOT_FOUND | LaunchDarkly Observability can’t return the trace. Usually, the trace is older than your retention period. The trace might also be in another project, or have a timestamp outside the lookup window. | Add a span from a trace that is within your retention period, in the current project. |
DATASET_NOT_FOUND | Someone deleted the dataset while you were adding the span. | Choose another dataset. |
Traces you can’t add
Some errors mean you can’t add the trace to a dataset, and changing your setup does not resolve them. This table explains the errors for traces you can’t add:| Error | Cause |
|---|---|
INPUT_MISSING | The trace has no user prompt to use as the row’s input. For example, caching can drop the prompt from a trace. |
UNSUPPORTED_TELEMETRY_SOURCE | LaunchDarkly can’t parse the spans because they come from a framework that LaunchDarkly does not support yet. |
MALFORMED_TELEMETRY | LaunchDarkly can’t parse the spans because they are malformed. |
OUTPUT_TOO_LARGE | The expected output is larger than 1 MiB. The UI blocks expected outputs over this limit before you submit, so this error is rare. |
LaunchDarkly errors
The following errors come from LaunchDarkly, not from your trace or setup:TRANSIENT_ERROROBSERVABILITY_AUTH_FAILEDOBSERVABILITY_BAD_REQUESTOBSERVABILITY_INVALID_ARGUMENTTRANSFORM_FAILED
- “Couldn’t save the review, so nothing was added to [DATASET NAME],”: If LaunchDarkly can’t save your review, it doesn’t add the span to the dataset.
- “This span is already in [DATASET NAME]”: If the span is already in the dataset, LaunchDarkly asks whether to replace the existing row instead of showing an error.
Add traces with the REST API
You can also add a span to a dataset with the LaunchDarkly REST API. Adding a span is a job that runs in the background. You start the job, then poll it until it finishes. The REST API requires the same permissions as the UI. To learn more, read Prerequisites. This table explains the endpoints:| Method | Endpoint | Description |
|---|---|---|
POST | /api/v2/projects/PROJECT_KEY/environments/ENVIRONMENT_KEY/datasets/DATASET_KEY/trace-promotions | Starts adding a span to the dataset. Returns a 202 status code and a job to poll. |
GET | /api/v2/projects/PROJECT_KEY/environments/ENVIRONMENT_KEY/datasets/DATASET_KEY/trace-promotions/JOB_ID | Returns the status of a job. A failed job returns a 200 status code with status: failed and an error, not a 5xx status code. |
POST request body accepts the following fields:
traceId: The ID of the trace that contains the span.timestamp: The timestamp of the trace.spanId: (Optional) The ID of the span to add.annotationId: (Optional) The ID of the annotation to apply to the row.overwriteExisting: (Optional) Set totrueto replace an existing row for the span.
DATASET_ROW_ALREADY_EXISTS: The span already has a row in the dataset, andoverwriteExistingis setfalse.ANNOTATION_SPAN_MISMATCH: TheannotationIdbelongs to a different span.ANNOTATION_NOT_FOUND: The annotation was deleted after the job started.