> ## Documentation Index > Fetch the complete documentation index at: https://launchdarkly.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Java AI SDK reference **This SDK is not yet generally available** The Java AI SDK is still undergoing testing and active development toward a 1.0 release. Its functionality may change without notice, including changes that break backward compatibility. This topic documents how to get started with the Java AI SDK, and links to reference information on all of the supported features. The Java AI SDK is designed for use with AgentControl. You can follow development or contribute on [GitHub](https://github.com/launchdarkly/java-core/tree/main/lib/sdk/server-ai). **SDK quick links** We provide open source SDKs. In addition to this reference guide, we provide source, API reference documentation, and sample applications:
Resource Location
SDK API documentation [SDK API docs](https://launchdarkly.github.io/java-core/lib/sdk/server-ai/)
GitHub repository [server-ai](https://github.com/launchdarkly/java-core/tree/main/lib/sdk/server-ai)
Published module [Maven Central](https://central.sonatype.com/artifact/com.launchdarkly/launchdarkly-java-server-sdk-ai)
**For use in server-side applications only** This SDK is intended for use in multi-user Java server applications. To learn more about our different SDK types, read [Choosing an SDK type](/docs/sdk/concepts/client-side-server-side). ## Get started LaunchDarkly AI SDKs interact with [AgentControl configs](/docs/home/getting-started/vocabulary#agentcontrol-config). Configs are the LaunchDarkly resources that manage model configurations and messages for your generative AI applications. **Try the Quickstart** This reference guide describes working specifically with the Java AI SDK. For a complete introduction to LaunchDarkly AI SDKs and how they interact with configs, read [Quickstart for AgentControl](/docs/home/agentcontrol/quickstart). You can use the Java AI SDK to customize your config based on the [context](/docs/home/getting-started/vocabulary#context) you provide. This means both the messages and the model evaluation in your generative AI application are specific to each end user, at runtime. You can also use the Java AI SDK to record metrics from your AI model generation, including duration and tokens. Follow these instructions to start using the Java AI SDK in your application. ### Version compatibility The Java AI SDK requires a minimum Java version of 8. ### Install the SDK This module is part of the [java-core](https://github.com/launchdarkly/java-core) monorepo and is published to Maven Central as `com.launchdarkly:launchdarkly-java-server-sdk-ai`. The Java AI SDK is built on top of the [Java (server-side) SDK](/docs/sdk/server-side/java), so you need to install both packages. To find the latest version, refer to the [SDK releases page](https://github.com/launchdarkly/java-core/releases). Here is how: ```xml title="Maven" lines wrap theme={null} com.launchdarkly launchdarkly-java-server-sdk com.launchdarkly launchdarkly-java-server-sdk-ai ``` **Internal API convention** Public, supported types live directly under `com.launchdarkly.sdk.server.ai`, including its documented subpackages. Anything under `com.launchdarkly.sdk.server.ai.internal` is an implementation detail. It is not part of the supported API, it is excluded from the published Javadoc and sources jars, and it may change without notice. ### Initialize the client After you install the SDK, create a single, shared instance of `LDClient`. When the `LDClient` is initialized, use it to initialize the `LDAIClient`. The `LDAIClient` is how you interact with configs. Here is how: ```java title="Java AI SDK" lines wrap theme={null} LDClient ldClient = new LDClient(sdkKey); LDAIClient aiClient = new LDAIClientImpl(ldClient); ``` ### Configure the context Next, configure the context that will encounter generated AI content in your application. The context attributes determine which variation of the config LaunchDarkly serves to the end user, based on the targeting rules in your config. If you use template variables in the messages or instructions in your config's variations, the context attributes also provide values for those variables. Within a prompt message or agent instruction, the evaluation context is available as `{{ldctx}}`, for example `{{ldctx.key}}`. Here is how: ```java title="Java AI SDK" lines wrap theme={null} LDContext context = LDContext.builder("example-context-key") .set("firstName", "Sandy") .set("lastName", "Smith") .set("email", "sandy@example.com") .build(); ``` ### Customize a config Customize your config to set values for variables used in messages or instructions, based on the context attributes and variables you provide. The details of customizing a config depend on whether you are using configs in completion mode or agent mode. You set the mode when you [create the config](/docs/home/agentcontrol/create) in the LaunchDarkly UI. #### Customize configs in completion mode In completion mode, each variation in your config includes a single set of roles and messages used to prompt your AI model. Use `completionConfig` to customize the config. This method takes the following variables: * AgentControl config key * Context * Fallback value * Optional variables It performs the evaluation, then returns an `AICompletionConfig` object that includes the customized model, provider, and messages, with variables already interpolated. Call `createTracker` on the returned object to get a tracker instance you can use to record metrics. If `completionConfig` cannot perform the evaluation or LaunchDarkly is unreachable, the SDK returns the fallback configuration you provided. For example, you might use an empty, disabled `AICompletionConfigDefault` as a fallback value, or a fully configured default. Either way, check `config.isEnabled()` before using the result, and skip generation for the disabled case. Here is how: ```java title="Customize a config in completion mode" lines wrap theme={null} Map variables = new HashMap<>(); variables.put("username", "Sandy"); AICompletionConfig config = aiClient.completionConfig( "example-config-key", context, AICompletionConfigDefault.disabled(), variables ); LDAIConfigTracker tracker = config.createTracker(); ``` After you retrieve a config, `config.getModel()` and `config.getProvider()` are ready to pass to your model provider as-is. The `config.getMessages()` method returns the messages with any placeholders already interpolated. #### Customize configs in agent mode In agent mode, each variation in your config includes a set of instructions that enable multi-step workflows. In agent mode, use `agentConfig` or `agentConfigs` to customize the config. `agentConfig` customizes a single agent config. `agentConfigs` customizes a list of them and returns a map keyed by agent key. The SDK returns the instructions as you defined them for the variation in the LaunchDarkly UI. Call `createTracker` on the returned object to get a tracker instance for recording metrics. If the method cannot perform the evaluation or LaunchDarkly is unreachable, it returns the fallback value. For example, you might use an empty, disabled `AIAgentConfigDefault` as a fallback value, or a fully configured default. Either way, check `agent.isEnabled()` before using the result, and skip generation for the disabled case. Here is how: ```java title="Customize a config in agent mode" lines wrap theme={null} Map variables = new HashMap<>(); variables.put("topic", "climate change"); AIAgentConfig agent = aiClient.agentConfig( "example-config-key", context, AIAgentConfigDefault.disabled(), variables ); ``` To learn more, read [Customizing AgentControl configs](/docs/sdk/features/agentcontrol-config). #### Retrieve a judge config Use `judgeConfig` to retrieve the judge configuration for a config variation. The method takes an AgentControl config key, a context, a fallback value, and optional variables. It returns an `AIJudgeConfig` object that includes the judge's customized messages, model configuration, and the evaluation metric key to associate scores with. To learn more about using a judge to score model output, read [Evaluate output manually with a judge](#evaluate-output-manually-with-a-judge) below. #### Retrieve a config without interpolating placeholders Use `completionConfigTemplate`, `agentConfigTemplate`, or `judgeConfigTemplate` to retrieve a config the same way as their non-template counterparts, but with Mustache placeholders, including `{{ldctx.*}}`, left intact. These methods omit the `variables` parameter, since interpolation is skipped. This is useful for displaying prompt previews, storing templates for later rendering, or auditing prompt content. ### Evaluate output manually with a judge The Java AI SDK does not automatically invoke judges on completion or agent calls. To evaluate output yourself: 1. Retrieve the judge config with `judgeConfig`. 2. Implement `Runner`, a single-method interface that wraps whatever model provider SDK you use, so the SDK can invoke your model when it runs a judge prompt. The SDK does not ship provider-specific `Runner` implementations, so you write your own for whatever provider you use. 3. Construct a `Judge` by passing the judge config, your `Runner` implementation, and a logger to the `Judge` constructor. 4. Call `evaluate` on the `Judge` to score an input and output pair. The `evaluate` method takes the input, the output, and an optional sampling rate, and returns a `JudgeResult` with the score and reasoning. 5. Record the result with the tracker's `trackJudgeResult` method. Here is how: ```java title="Evaluate output manually with a judge" lines wrap theme={null} AIJudgeConfig judgeConfig = aiClient.judgeConfig( "example-judge-key", context, AIJudgeConfigDefault.disabled(), variables ); if (judgeConfig.isEnabled()) { Judge judge = new Judge(judgeConfig, runner, logger); JudgeResult result = judge.evaluate(inputText, outputText); // Record the result against the tracker for the completion or agent config // whose output you are evaluating, so the score is associated with that run. tracker.trackJudgeResult(result); } ``` Manual judge evaluation does not automatically emit metrics for the config's **Monitoring** tab. It also does not attach judges to variations in the LaunchDarkly UI. To learn more, read [Online evaluations in AgentControl](/docs/home/agentcontrol/online-evaluations) and [Judges](/docs/home/agentcontrol/judges). ### Call the provider and record metrics from AI model generation To make a request to your generative AI provider and record metrics from the response: 1. Check `config.isEnabled()` before generating, and skip generation for the disabled case. 2. Create a tracker with `config.createTracker()`. 3. Wrap your provider call in `trackDurationOf` to record its duration. Alternatively, call the individual `track*` methods directly, such as `trackTimeToFirstToken`, `trackToolCall`, or `trackFeedback`. 4. Record token usage with `trackTokens`. 5. Record the outcome with `trackSuccess` or `trackError`. Here is how: ```java title="Java AI SDK, any model" lines wrap theme={null} if (config.isEnabled()) { LDAIConfigTracker tracker = config.createTracker(); Response response = tracker.trackDurationOf(() -> { // Make a request to your generative AI provider, using details from // the config. For example, pass config.getModel() and config.getMessages(). return callProvider(); }); tracker.trackTokens(new TokenUsage(response.getTotalTokens(), response.getInputTokens(), response.getOutputTokens())); tracker.trackSuccess(); } ``` Call `completionConfig` or `agentConfig` each time you generate content from your AI model, and call `createTracker` each time to start a new run. To learn more, read [Tracking AI metrics](/docs/sdk/features/ai-metrics). ### Agent graphs Agent graphs let you define a directed graph of agent configs in the LaunchDarkly UI, then traverse the graph at runtime to orchestrate multi-step workflows. Each node in the graph is an agent config. Each edge defines a handoff from one agent to the next. Use `agentGraph` to retrieve a graph by its key. The graph key populates automatically when you create a graph in the LaunchDarkly UI. After the graph exists, the key is in the agent graph details panel. To learn more, read [Initialize an agent graph](/docs/home/agentcontrol/agent-graphs#initialize-an-agent-graph). To set up a graph invocation: 1. Call `agentGraph` with the graph key, a context, and any variables. The method returns an `AgentGraphDefinition` that exposes the graph's topology and a tracker for recording invocation metrics. 2. Check `graph.isEnabled()` before walking the graph. 3. Create a tracker with `graph.createTracker()` to record invocation metrics. Then walk the graph manually or with `traverse`/`reverseTraverse`, as described below. Here is how: ```java title="Java AI SDK" lines wrap theme={null} Map variables = new HashMap<>(); variables.put("topic", "climate change"); AgentGraphDefinition graph = aiClient.agentGraph("example-graph-key", context, variables); if (graph.isEnabled()) { AIGraphTracker tracker = graph.createTracker(); // Walk the graph and invoke each agent's config. } ``` You can navigate the graph manually using `rootNode`, `getNode`, `getChildNodes`, `getParentNodes`, and `terminalNodes`. Each `AgentGraphNode` exposes: * `getKey()`: The agent config key for this node. * `getConfig()`: The `AIAgentConfig` for the node, with its instructions, model, tools, and other agent config properties. * `getEdges()`: The outgoing `GraphEdge` list from this node. Each edge includes the target node key and a handoff map describing how to pass state to the next node. * `isTerminal()`: Whether this node has no outgoing edges. Alternatively, use `traverse` or `reverseTraverse` to walk the graph with a callback that the SDK invokes for each node. `traverse` walks the graph in topological order starting from the root, visiting a node only after all of its reachable predecessors have been visited. `reverseTraverse` walks in topological order starting from the terminal nodes, visiting a node only after all of its reachable descendants have been visited, so the root is visited last. Both methods take a callback with the signature `BiFunction, Object>` and an initial context map. The SDK calls the callback one time for each node, passing the node and a context map scoped to that node's dependency results. The callback's return value is stored under the node's key for downstream nodes to read. Here is how: ```java title="Java AI SDK" lines wrap theme={null} graph.traverse((node, ctx) -> { // Use node.getConfig() to call your AI provider, passing instructions and tools. // Read upstream results from ctx as needed. Object output = callModelForNode(node.getConfig(), ctx); // The return value is stored under node.getKey() for downstream nodes. return output; }, new HashMap<>()); ``` The SDK does not automatically record handoff or invocation metrics during traversal. Call the appropriate `AIGraphTracker` methods from your callback to record what happens at each node and edge. Use the `AIGraphTracker` returned by `createTracker` to record metrics for the full graph invocation as well as per-edge handoffs: * `trackInvocationSuccess` and `trackInvocationFailure` record whether the graph completed successfully. * `trackDuration` records total invocation duration. * `trackTotalTokens` records aggregate token usage across all nodes. * `trackPath` records the sequence of nodes visited. * `trackHandoffSuccess` and `trackHandoffFailure` record per-edge handoff outcomes. * `trackRedirect` records when a node redirects to a different target than its declared edges. You can call the provider for each node the same way you do in agent mode. To learn more, read [Customize configs in agent mode](#customize-configs-in-agent-mode). To learn more about defining agent graphs in the LaunchDarkly UI, read [Agent graphs](/docs/home/agentcontrol/agent-graphs). ### Resume tracking across processes If a single AI workflow spans multiple processes, you can resume tracking on the same config or graph invocation in a different process. For example, a request handler might hand work off to a background job: 1. In the original process, create the config and tracker as usual. 2. Call `getResumptionToken` on the tracker to get a token. 3. Persist or pass the token to the next process, through a job queue, database row, or message envelope. 4. In the receiving process, call `aiClient.createTracker` for a completion or agent tracker, or `aiClient.createGraphTracker` for an agent graph tracker, passing the token and a context. Both methods return a tracker that records to the same invocation as the original. **Keep resumption tokens server-side** Resumption tokens embed flag-evaluation details such as the variation key and config version. Keep tokens server-side and do not round-trip them through untrusted clients where they could leak flag-targeting information. Here is how: ```java title="Original process" lines wrap theme={null} AICompletionConfig config = aiClient.completionConfig("example-config-key", context, fallbackConfig, variables); LDAIConfigTracker tracker = config.createTracker(); // Hand off work to another process. Persist the resumption token so the // receiving process can reconstruct the tracker. String token = tracker.getResumptionToken(); enqueueBackgroundJob(token, context); ``` ```java title="Receiving process" lines wrap theme={null} LDAIConfigTracker tracker = aiClient.createTracker(token, context); ``` For agent graphs, use `createGraphTracker` in place of `createTracker`. The token comes from `AIGraphTracker.getResumptionToken()` on the original graph tracker. ## Supported features This SDK supports the following features: * [Anonymous contexts](/docs/sdk/features/anonymous#java-ai) * [Context configuration](/docs/sdk/features/context-config#java-ai) * [Customizing AgentControl configs](/docs/sdk/features/agentcontrol-config#java-ai) * [Private attributes](/docs/sdk/features/private-attributes#java-ai) * [Tracking AI metrics](/docs/sdk/features/ai-metrics#java-ai)