Skip to main content
This topic documents how to get started with the Go AI SDK, and links to reference information on all of the supported features. The Go AI SDK is designed for use with AgentControl. It retrieves a config variation, calls your model provider through a handler, and records metrics, traces, and judge evaluations around each call.
This SDK replaces the previous Go AI SDKVersion 1.0.0 of the Go AI SDK is a new module, github.com/launchdarkly/go-ai-sdk, with a different API from the previous Go AI SDK, github.com/launchdarkly/go-server-sdk-ai/ldai. The two modules are not interchangeable. If your application uses the previous SDK, its reference is available at Go AI SDK reference (legacy). To plan your move to this SDK, read Migrate from the legacy AI SDKs.
SDK quick linksThe SDKs are open source. In addition to this reference guide, we provide source, API reference documentation, and sample applications.
ResourceLocation
SDK API documentationSDK API docs
GitHub repositorygo-ai-sdk
Published modulepkg.go.dev
Sample applicationsexamples

Get started

LaunchDarkly AI SDKs interact with AgentControl configs. Configs are the LaunchDarkly resources that manage model configurations and messages for your generative AI applications.
Try the QuickstartThis reference guide describes working specifically with the Go AI SDK. For a complete introduction to LaunchDarkly AI SDKs and how they interact with configs, read Quickstart for AgentControl.
You can use the Go AI SDK to customize your config based on the context that you provide. This means both the messages and the model evaluation in your generative AI application are specific to each end user, at runtime. You can also use the AI SDK to call your model provider, record metrics from the generation, and evaluate model output with judges. Follow these instructions to start using the Go AI SDK in your application.

Prerequisites

To use the Go AI SDK, you need:

Install the SDK

First, install the AI SDK as a dependency in your application. The SDK requires Go 1.26 or later. How you install it depends on what dependency management system you are using.
  • If you are using the standard Go modules system, import the SDK packages in your code and go build automatically downloads them.
  • Otherwise, use the go get command and specify the SDK version, such as go get github.com/launchdarkly/go-ai-sdk/aiserver@v1.0.0.
All of the packages are in one Go module and share one version. The aiserver package is the core client. Add one handler package for each model provider you use. Installing aiserver also installs the LaunchDarkly Go SDK and the OpenTelemetry modules that it depends on. Here is how:
Shell
Then, import the packages you use. Here is how:
Go AI SDK

Choose a provider handler

A handler calls a model provider on behalf of the SDK. The provider name and mode on the config variation decide which handler runs. A config in completion mode uses a messages handler, and a config in agent mode uses an agent handler.
PackageProviderModeDescription
claudemessagesAnthropicmessagesAnthropic Messages API. Runs function tools in a local tool loop.
claudeagentsAnthropicagentIn-process agent loop on the Anthropic Messages API, with the Claude Code built-in tools.
openaimessagesOpenAImessagesOpenAI Responses API.
openaiagentsOpenAIagentIn-process agent loop on the OpenAI Responses API, with handoffs.
bedrockmessagesAmazon BedrockmessagesAmazon Bedrock Converse API.
bedrockagentsAmazon BedrockagentAmazon Bedrock agent handler.
langchainmessagesAny provider (*)messagesLangChain-style handler. The fallback when no other handler names the provider.
langchainagentsAny provider (*)agentLangChain-style agent handler. The fallback when no other handler names the provider.
The SDK also includes the following packages.
PackageDescription
aiserverThe core client. Includes Config, Graph, ResolveGraph, InspectConfig, the registry, lifecycle functions, judges, and telemetry.
skillsAgent skills. Reads skills that configs reference, verifies them, and writes them to disk.
evaluationsEvaluations from code. Runs a dataset through a handler, scores each output, and reports the verdict.
llmsThe chat-model interface, plus built-in OpenAI, Anthropic, and Bedrock models for the LangChain handlers.
Each handler reads the usual environment variables for its provider, such as ANTHROPIC_API_KEY, OPENAI_API_KEY, or the AWS credential chain and AWS_BEARER_TOKEN_BEDROCK. You can also inject your own provider client through the handler options.
The agent handlers run their own agent loopGo has no official OpenAI Agents SDK or Claude Agent SDK. openaiagents and claudeagents run their own agent loop on the official provider SDKs. LangChain publishes no official Go SDK. langchainmessages and langchainagents use the llms package and state-graph runner that ship with this SDK.
The claudeagents built-in tools run on your hostThe claudeagents built-in tools, such as ClaudeBash, ClaudeRead, and ClaudeEdit, run in your process with the permissions of the process user. File tools are confined to Options.WorkDir, but ClaudeBash is not sandboxed and inherits the full process environment, including secrets such as API keys. Enable these tools only for configs and inputs you trust. Run the application in a container or virtual machine with a dedicated working directory and least-privilege credentials.

Initialize the client

After you install and import the SDK, initialize the AI client. The AI client is how you interact with configs. Specify the SDK key to authorize your application to connect to a particular environment within LaunchDarkly.
The Go AI SDK uses an SDK keyThe Go AI SDK uses an SDK key. Keys are specific to each project and environment. They are available on the SDK keys page under Settings. To learn more about key types, read Keys.
Here is how:
Go AI SDK
You can initialize the client in three ways.
  • Lazy initialization. The first call to Config, Graph, ResolveGraph, or InspectConfig calls InitClient with empty options, which means every setting comes from environment variables.
  • Explicit initialization. Call InitClient(ctx, opts) to pass options or to fail fast at startup.
  • Bring your own client. Call InitWithClient(ctx, ldClient, opts) with any value that satisfies aiserver.LDClient. Any *ld.LDClient satisfies the requirement. To build one, import github.com/launchdarkly/go-server-sdk/v7 as ld and call ld.MakeClient.
If a client is already installed, InitClient and InitWithClient return the installed client. Call Shutdown first to switch clients. Shutdown exports pending spans, flushes events, and closes the LaunchDarkly client, including a client you passed to InitWithClient. InitOptions.WaitFor sets the initialization timeout. A zero or negative value means the 10 second default. A failed initialization is not cached. The next call tries again. If you rely on lazy initialization while LaunchDarkly is unreachable, each call can wait up to the timeout before it fails. To keep an outage from adding that wait to every request, call InitClient at startup and handle its error there.
Initializing the client also sets up OpenTelemetry tracingInitClient and InitWithClient register a global OpenTelemetry tracer provider. To learn what the SDK exports, read Record metrics and traces.

Configure the context

Next, configure the context that will use the config, that is, the context that will encounter generated AI content in your application. The context attributes determine which variation of the config LaunchDarkly serves to the end user, based on the targeting rules in your config. If you are using template variables in the messages in your config’s variations, the context attributes also fill in values for the template variables. Here is how:
Go AI SDK
This example uses the ldcontext and ldvalue packages from github.com/launchdarkly/go-sdk-common/v3.

Protect private attributes

Anonymous contexts and private attributes work the same way as in the Go server SDK. The AI SDK sends its events through the Go server SDK, which applies its usual handling of anonymous contexts and private attributes. Spans carry only context kinds and keys, never attribute values. The SDK records them as feature_flag.context.id, feature_flag.contextKeys, and context.contextKeys.<kind>. Prompt templates are the exception. The full context, including private attributes, is available to templates as ldContext.
Templates can expose private attributesIf a template in your config’s messages references a private attribute, the SDK puts the attribute’s value into the prompt. With content capture on, which is the default, the value also appears in span text. To turn content capture off, read Control what the SDK records on spans.

Customize a config and call your model

Then, use Config to customize a config and call your model. Customization means that any variables you include in the messages when you define the config variation have their values set to the context attributes and variables you pass in. You set the mode for a config when you create it in the LaunchDarkly UI. The customization process within the AI SDK is similar to evaluating flags in one of the LaunchDarkly client-side, server-side, or edge SDKs, in that the SDK completes the customization without a separate network call. After it customizes the config, the SDK routes the call to the handler that matches the variation’s provider and mode, and returns the response. Config takes a config key and the handlers that can serve it. Invoke takes the input, the context, and optional call options. Use WithVariables to fill template variables in the config’s messages, and WithHistory to pass earlier messages in the conversation. Here is how:
Go AI SDK
resp.Text() returns the response as text, whether the model returned a string or structured output.

Use a convenience function

If you use one provider in one mode, a convenience function builds the handler and makes the call in one step. Each handler package has one, such as openaimessages.OpenAIMessages, claudemessages.ClaudeMessages, and openaiagents.OpenAIAgents. Pass the config key, the user input, the context, and optional InvokeOptions. The options hold Variables for template variables, ToolHandlers, Registry, SkipJudges, and Handler, which sets the handler options. Here is how:
Go AI SDK
If a config can serve more than one provider, use Config and Invoke with several handlers instead.

Handle a disabled or unavailable config

A call can fail before it reaches a model. The SDK has no fallback value. Instead, Invoke and Stream return an error that you match with errors.Is, and your application decides what to do next.
  • ErrConfigDisabled means the config variation is turned off.
  • ErrConfigNotFound means the flag returned no config. The flag is missing, has the wrong type, or could not be evaluated, for example because the client is not ready. The error wraps the reason from the LaunchDarkly Go SDK, so errors.Is can match that cause too.
To see every sentinel error, read Errors.

Stream a response

Use Stream to receive the response as the model generates it. Stream returns an iterator. Each event has a type, and chunk events carry text. Here is how:
Go AI SDK
If you break out of the loop, the SDK abandons the stream. The SDK marks the spans with launchdarkly.stream.abandoned and tracks neither a success nor an error. If a handler panics, the SDK tracks a generation error and then the panic continues to your code. A panic in your own loop body counts as abandonment.

Use structured output

A config can return structured output that conforms to a JSON schema, rather than plain text. Use structured output when another system, database, or function consumes the model’s response, or when you need to extract specific information, such as a severity or category. Set the output format on the config variation in the LaunchDarkly UI. The variation’s OutputFormat holds the schema. When a variation has an output format, resp.Text() returns the structured output as a JSON string, which you can unmarshal into your own type. Here is how:
Go AI SDK

Use more than one provider

Pass several handlers to serve variations that use different providers. The variation’s provider name and mode pick the handler. A handler with the provider *, such as the LangChain handlers, is the fallback when no other handler names the provider. Here is how:
Go AI SDK
If no handler serves the variation’s provider and mode, the call returns ErrNoHandler.

Write a custom handler

If you use a provider that has no handler package, or you want to call an internal endpoint, use aiserver.CreateHandler to build your own handler. A custom handler works with Config routing and registries in the same way as the built-in handlers. CreateHandler takes four arguments:
  • A ProvidesFor value, which holds the provider name and mode that the handler serves.
  • An invoke function, which receives a HandlerRequest and returns a HandlerResult.
  • An optional stream function. If you pass nil, Stream falls back to the invoke function.
  • A ContentCapture setting for the handler’s spans.
The HandlerRequest holds the customized config, the user input, the tool handlers, the variables, and the conversation history. Return the model’s answer in the result’s Output, and its token counts in Usage, with keys such as input_tokens and output_tokens. Here is how:
Go AI SDK
A custom handler with the provider * serves any provider that no other handler names, which makes it a fallback.

Share handlers and tools with a registry

If your application calls several configs, build the handlers and tools once, then share them with a registry. Tool keys must match the keys in the variation’s tools map exactly. The SDK never offers a tool with no registered function to the model. Here is how:
Go AI SDK
Handlers and tools that you pass directly on a call take precedence over the registry’s. Unlike the JavaScript and Python AI SDKs, the Go AI SDK has no global registry. Build a *Registry and pass it explicitly. If you want one registry for the whole process, keep it in a variable in your own package.

Use built-in provider tools

Some handlers include built-in tools that run natively, rather than a function you write. Built-in tools are *aiserver.NativeTool values. To use one, assign it as the implementation for a LaunchDarkly tool key in place of a function. The SDK tracks a built-in tool call like any other tool call. The claudeagents package exports these built-in tools:
  • ClaudeWebSearch and ClaudeWebFetch, which map to Anthropic server tools. Anthropic runs them.
  • ClaudeBash, ClaudeRead, ClaudeEdit, ClaudeWrite, ClaudeGlob, ClaudeGrep, ClaudeTodoWrite, and ClaudeNotebookEdit, which run on your host. To learn about the risks, read Choose a provider handler.
Here is how to enable Claude web search:
Go AI SDK

Read a config without calling a model

Use InspectConfig to read the variation that a context receives, without calling a model. This is useful if you want to call your provider yourself. InspectConfig returns an InspectResult and an error. A turned-off variation returns Enabled as false and a nil error. Initialization, evaluation, and parse failures return an error and a zero result. The InspectResult has the following fields:
  • Enabled: Whether the variation is turned on.
  • Config: The config variation, as an *AiConfig. This is nil when the variation is turned off.
  • Meta: Metadata about the variation, as a *VariationMeta. It has VariationKey, Version, Mode, ModelKey, and ModelVersion.
The Config holds everything you need to call a provider yourself. It has Instructions, Messages, Model, Provider, Tools, JudgeConfiguration, EvaluationMetricKey, and OutputFormat. Each message has a Role and Text. The model has a Name, Parameters, and Custom values, and the provider has a Name. Consider the following before you call a provider directly:
  • The SDK returns messages as templates, without filling in their variables. Use aiserver.ParseTemplate to fill them.
  • LaunchDarkly records no metrics for a provider call that you make yourself.
Here is how:
Go AI SDK

Run agent graphs

An agent graph links multiple agent configs together into nodes and edges. Use Graph to run a graph. The SDK routes each node to the handler that matches the node’s provider and mode. Pass agent handlers. Here is how:
Go AI SDK
Invoke and Stream on a graph take optional CallOption values. Use WithVariables to pass template variables. If the graph, or a node that it references, is turned off, the call returns ErrGraphDisabled. Other failures, such as an initialization or evaluation error, return a different error. Stream on a graph yields node start, chunk, node done, handoff, and done events. A node’s own judge scores are on its node done event, and the done event holds only the graph-level judge scores. To collect every score, read each node done event. Here is how:
Go AI SDK
Each agent handler package has a graph convenience wrapper with the handler already bound, such as openaiagents.OpenAIGraph, claudeagents.ClaudeGraph, bedrockagents.BedrockGraph, and langchainagents.LangChainGraph. Here is how:
Go AI SDK
If you want to run the graph with a framework-native runner, use ResolveGraph to read the topology and node configs. If the graph is turned off, ResolveGraph returns a disabled definition and a nil error. Check def.Enabled before you run it. The agent handler packages include adapters that take the result, such as ToClaudeAgents, ToOpenAIAgents, and ToLangGraph. This example reuses the tools map from the previous example. Here is how:
Go AI SDK

Evaluate output with judges

Judges are AgentControl configs that score the output of another config. You attach judges to a variation in the LaunchDarkly UI, and each attachment has a sampling percentage. To learn more, read Judges. The SDK runs the attached judges after the call returns, and includes the results in the response. In an agent graph, each node can have its own judges, and a graph-level judge scores the final answer. A judge needs a handler for its own provider. If no handler covers the judge’s provider, the SDK skips that judge. If the judge’s own config is missing or turned off, the SDK skips that judge and logs the skip. If a judge’s provider call fails, Invoke returns both the response and the error. The answer is still usable. Check the response before you treat the error as fatal.

Run judges later

By default, Invoke returns the judge results with the response. To keep response time low, set SkipJudges on ConfigArgs, or on the InvokeOptions of a convenience function. The response then carries JudgeTasks instead of results. Each task is JSON-serializable, which means you can queue it. Pass each task and your handlers to RunJudge when you are ready to run it, for example on your own goroutine. RunJudge returns a *JudgeRunResult and an error. The result has Score, Response, Usage, and TrackData. RunJudge returns an error only when the judge’s provider call fails. It returns a nil result and a nil error when no handler serves the judge’s provider, or when the judge returns no numeric score. RunJudge does not send the evaluation metric. To record the score, call aiserver.Track with the task’s evaluation metric key, the task’s context, the result’s TrackData, and the score. Here is how:
Go AI SDK
Start the worker for each task with go runJudgeWorker(ctx, task, handlers). Pass handlers that cover each judge’s provider. If you use a registry, pass registry.Handlers().

Record metrics and traces

The SDK records metrics around each call without extra code. It sends LaunchDarkly $ld:ai:* events for token usage, latency, and errors, and it creates OpenTelemetry spans for handler calls, tool calls, and judge evaluations. Version 1.0 of the Go AI SDK does not record user feedback, such as a thumbs up or down on a response. It has no feedback event and no resumption tokens.

Configure tracing

InitClient registers a global OpenTelemetry tracer provider. It exports spans over OpenTelemetry Protocol (OTLP)/HTTP with gzip compression. The SDK uses the endpoint from InitOptions.OTLPEndpoint, then OTEL_EXPORTER_OTLP_ENDPOINT, then the default endpoint. The tracer provider sets these resource attributes.
AttributeValue
launchdarkly.project_idSet from the SDK key, from InitOptions.SDKKey or LD_SDK_KEY.
service.nameLD_SERVICE_NAME, or go-sdk if unset.
deployment.environmentLD_ENVIRONMENT, when set.
The tracer provider sends your SDK key as a resource attributeThe launchdarkly.project_id resource attribute carries your SDK key. If you point OTEL_EXPORTER_OTLP_ENDPOINT or InitOptions.OTLPEndpoint at your own collector, the SDK sends the key to that collector with every export.
To group the spans from one conversation, bind a conversation ID to the context. The SDK adds gen_ai.conversation.id to every span that starts under that context. Here is how:
Go AI SDK

Control what the SDK records on spans

By default, spans include content. Handler spans carry the prompt and completion text, tool spans carry the tool arguments and results, and judge spans carry the judge’s explanation. The SDK always records the model, provider, token usage, status, and tool names.
Spans include prompt and completion text by defaultIf your prompts or completions can contain sensitive data, turn content capture off before the first handler call.
You can turn content capture off for one handler, or for the whole process. A handler setting takes precedence over the process setting, and the process setting takes precedence over the default, which is on. Here is how:
Go AI SDK
After the first handler call, Configure accepts only options that have the same effect, and otherwise returns ErrConfigureLocked. Shutdown resets this setting.

Use Agent skills

Agent skills are skill files that AgentControl configs reference. The skills package reads those skills, verifies their content, and writes them to disk as <root>/<key>/SKILL.md. Your agent can then load them. Writing skills to disk works on POSIX systems only. Install the package with go get github.com/launchdarkly/go-ai-sdk/skills. The following example keeps a directory in sync with LaunchDarkly. When you revoke a skill, the SDK’s watcher removes it from disk within 500 milliseconds. Add .launchdarkly-skills.json to your .gitignore file, because the SDK writes a manifest with that name. Here is how:
Go AI SDK
To learn more, read Agent skills.

Run evaluations from code

The evaluations package runs a dataset through your own handler, scores each output with AgentControl judges or local scorers, and reports the results to LaunchDarkly. Use it as a gate in continuous integration. Install the package with go get github.com/launchdarkly/go-ai-sdk/evaluations, plus the handler package that runs your generation step. Running evaluations from code needs more credentials than the other features. In LaunchDarkly, the API access token is called the Eval API key. To generate your credentials, click Evaluations in the side navigation, click Run evaluations, then choose From code. If you create the token yourself, assign a role that allows reading datasets and tools, and creating evaluations and evaluation runs, in the target project. The Writer role works.
VariableRequiredDescription
LD_PROJECT_KEYYes, unless you set InitEvaluationsOptions.ProjectKeyThe project that holds the evaluations, runs, tools, datasets, and judges.
LD_API_TOKENYes, unless you set InitEvaluationsOptions.APITokenA LaunchDarkly API access token. It authorizes every management API request.
LD_SDK_KEYYes, unless you set InitEvaluationsOptions.SDKKey or already initialized a clientThe server-side SDK key for the client that sends result events.
LD_API_BASE_URINoThe management API host. The default is https://app.launchdarkly.com. The SDK never uses LD_BASE_URI for this.
LD_UI_BASE_URINoThe host of the run link in the result. The default is https://app.launchdarkly.com.
Here is how:
Go AI SDK
The SDK never computes a verdict. It reports scores, and LaunchDarkly rules on them against the thresholds that you pin on the evaluation, and returns its verdict in res.Verdict as EvalVerdictPass or EvalVerdictFail. When LaunchDarkly returns a verdict, Passed follows it. Otherwise, Passed is true only if no row errored or failed. To learn more, read Evaluations from code.

Reference

This section lists the environment variables that configure the SDK, the errors that it returns, and its main functions.

Configuration

Each environment variable fills the matching InitOptions field only if that field is empty. A field that you set wins over its variable, and a blank value counts as unset.
VariableInitOptions fieldRequiredDescription
LD_SDK_KEYSDKKeyYes, unless you set the field or use InitWithClientThe server-side SDK key.
LD_BASE_URIBaseURINoThe polling base URI, for non-production environments.
LD_STREAM_URIStreamURINoThe streaming URI.
LD_EVENTS_URIEventsURINoThe events URI.
OTEL_EXPORTER_OTLP_ENDPOINTOTLPEndpointNoThe OTLP/HTTP trace endpoint. The SDK appends /v1/traces. The default is https://otel.observability.app.launchdarkly.com.
LD_SERVICE_NAMEServiceNameNoThe service.name resource attribute. The default is go-sdk.
LD_ENVIRONMENTEnvironmentNoThe deployment.environment resource attribute. Omitted if unset.
LD_ENVIRONMENT_IDNoneNoThe LaunchDarkly environment ID. The SDK sends it as environmentId on every $ld:ai:* event and as feature_flag.set.id on handler spans, so that AgentControl monitoring can correlate traces. Omitted if unset.

Errors

Match errors with errors.Is. Each error is wrapped with the key or provider that it concerns, and keeps the LaunchDarkly SDK’s own reason if there is one.
ErrorMeaning
ErrNotInitializedYou called GetClient before InitClient or InitWithClient.
ErrConfigNotFoundThe flag returned no config. It is missing, the wrong type, or could not be evaluated, for example because the client is not ready.
ErrConfigDisabledThe config variation is turned off.
ErrNoHandlerNo handler serves the variation’s provider and mode.
ErrGraphDisabledThe graph, or a node that it references, is turned off.
ErrConfigureLockedYou called Configure with a different effect after the first handler call.

API reference

For the full API, read the SDK API docs. The following table lists the main functions in the aiserver package, and the convenience functions in the handler packages.
FunctionPurpose
Config(ConfigArgs)Binds a config key. Use Invoke and Stream on the result.
Graph(key, GraphOptions)Binds an agent graph. Use Invoke and Stream on the result.
ResolveGraph(ctx, key, ldctx, GraphOptions)Returns the topology and node configs for framework-native runners, and an error for resolution failures.
InspectConfig(ctx, key, ldctx)Reads a variation without calling a model. Returns an error for initialization, evaluation, and parse failures.
Configure(SDKOptions)Sets the process-wide content capture default.
RunJudgeRuns a judge task from a call with SkipJudges set, for example on your own goroutine.
InitClient, InitWithClient, GetClient, ShutdownManage the client lifecycle.
SetLoggerSets the SDK logger. The default is slog.Default().
OpenAIMessages, ClaudeMessages, OpenAIAgents, and othersConvenience functions in the handler packages. Build the handler and call Config().Invoke in one step.
OpenAIGraph, ClaudeGraph, BedrockGraph, LangChainGraphGraph wrappers in the agent handler packages. Bind a handler to Graph.

Supported features

This SDK supports the following features:

What’s next

Now that you can call models with the Go AI SDK, explore these next steps: