Node.js (server-side) AI SDK reference

This SDK is in open betaThe Node.js (server-side) AI SDK is in open beta. Its APIs are subject to change. We do not recommend beta SDKs for production use. You can follow development or contribute on GitHub.

This topic explains how to get started with the Node.js (server-side) AI SDK, and links to reference information on all of the supported features. This SDK is the most current AI SDK, but it is not a drop-in replacement for the legacy Node.js (server-side) AI SDK, which is now in maintenance mode. We recommend that new applications use this SDK. If you are migrating from the legacy SDK, read Migrate from the legacy AI SDKs. You can use either JavaScript or TypeScript when working with the Node.js (server-side) AI SDK. This SDK is intended for use in multi-user Node.js server applications. To learn more about the different LaunchDarkly SDK types, read Choosing an SDK type. The Node.js (server-side) AI SDK is designed for use with AgentControl. The Node.js AI SDK provides an opinionated, higher-level entry to LaunchDarkly AgentControl. Register your handlers and tools one time, then call a model, run a judge, or execute an agent graph. You do not have to wire up provider clients, metrics tracking, or message conversion by yourself.

SDK quick linksLaunchDarkly SDKs are open source. In addition to this reference guide, we provide source, sample applications, and provider-specific packages:

Resource	Location
GitHub repository	js-ai-sdk
Sample applications	Examples directory
Published modules	npm
Provider-specific packages	OpenAI, Anthropic, and LangChain handler packages. Read Install the SDK.

Get started

LaunchDarkly AI SDKs interact with AgentControl configs. Configs are LaunchDarkly resources that manage model configurations and messages for your generative AI applications. When you invoke a config with the SDK, LaunchDarkly controls which model, provider, prompt, and tools are used at runtime. By using AgentControl, you can change providers, models, or prompts in LaunchDarkly without deploying code.

Try the quickstartThis reference guide describes working with the Node.js (server-side) AI SDK. For a complete introduction to LaunchDarkly AI SDKs and how they interact with configs, read Quickstart for AgentControl.

The SDK follows a three-tier architecture:

A core client (@launchdarkly/ai-server) manages the LaunchDarkly lifecycle, telemetry, and the config(), graph(), and resolveGraph() entry points.
Handler packages integrate a specific provider and mode, such as OpenAI in messages mode or Anthropic in agent mode.
Your application registers the handlers and tools it needs, then makes calls.

Here’s how to start using the Node.js (server-side) AI SDK in your application.

Install the SDK

First, install the core convenience package and one or more handler packages using your application’s dependency manager. For standard Node.js applications, install @launchdarkly/ai-node. This package re-exports the core client and bundles the Node.js (server-side) SDK as a dependency, so the client initializes automatically with no peer dependency wiring. Then, install a handler package for each provider and mode you want to use. The following handler packages are available:

Package	Provider	Mode	Convenience function
`@launchdarkly/ai-openai-messages`	OpenAI	`messages`	`openaiMessages`
`@launchdarkly/ai-openai-agents`	OpenAI	`agent`	`openaiAgents`
`@launchdarkly/ai-claude-messages`	Anthropic	`messages`	`claudeMessages`
`@launchdarkly/ai-claude-agents`	Anthropic	`agent`	`claudeAgents`
`@launchdarkly/ai-langchain-messages`	Any (`*`)	`messages`	`langchainMessages`
`@launchdarkly/ai-langchain-agents`	Any (`*`)	`agent`	`langchainAgents`

Here’s how to install the core package and the OpenAI messages handler:

npm install @launchdarkly/ai-node @launchdarkly/ai-openai-messages

To export traces to LaunchDarkly observability, also install the @launchdarkly/ai-otel package. This package bundles the OpenTelemetry SDK dependencies that the client discovers at runtime. To learn more, read Observability. Here’s how to install the OpenTelemetry (OTel) package:

npm install @launchdarkly/ai-node @launchdarkly/ai-otel @launchdarkly/ai-openai-messages

Edge and custom runtimesFor edge or custom runtimes, such as Vercel or Cloudflare, install the core @launchdarkly/ai-server package instead of @launchdarkly/ai-node, along with the LaunchDarkly SDK for your platform. Then call initClient(preInitializedClient) with your platform’s LaunchDarkly client before your first AI call. To learn more, read Manage the client lifecycle.

Configure the environment

The SDK reads your LaunchDarkly SDK key and your provider API keys from environment variables. Set LD_SDK_KEY and the API key for each provider you use, such as OPENAI_API_KEY or ANTHROPIC_API_KEY.

The Node.js SDKs use an SDK keyThe Node.js (server-side) AI SDK uses an SDK key. Keys are specific to each project and environment. They are available on the SDK keys page under Settings. To learn more about key types, read Keys.

Here’s how to load environment variables from a .env file at the top of your application:

import 'dotenv/config';

Call a model

There are two ways to call models. You can:

Call a model with a convenience function.
Call a model with a handler, and optionally write a custom handler.

The two options vary in how much configuration they allow. Convenience functions are the most opinionated entry point to the SDK. They are designed to trade configurability for speed so you can swap a provider SDK call for the LaunchDarkly equivalent and get the same behavior from the provider, while also getting targeting, metrics, and prompt management from LaunchDarkly. For more control, call a model with a handler by using config() from the core package. config() accepts a config key and one or more handlers, then routes to the correct handler at invocation time based on the config variation’s provider and mode. It returns an object with invoke() and stream() methods. You can also pass an array of handlers. When you do this, the SDK delegates each call to the handler whose provider and mode match the config variation. This lets you support multiple providers with a single call site. If you pass a single handler and it does not match the variation, invoke() throws an error.

Call a model with a convenience function

Each handler package exports a convenience function that is the shortest path to a working call. The convenience function lets you import one function and call it with the config key, the user input, and an LDContext, in that order. The SDK evaluates the config, selects the provider, calls the model, and records metrics. To interpolate values into your config’s messages, pass a variables property in the optional fourth argument. Here’s how to call an OpenAI config:

import 'dotenv/config';
import { openaiMessages } from '@launchdarkly/ai-openai-messages';

const result = await openaiMessages(
  'my-ai-config-flag',
  'What is feature flagging?',
  { kind: 'user', key: 'user-123' },
);

console.log(result.response);

Call a model with a handler

LangChain handlers match any providerLangChain handler packages register with the wildcard provider '*'. The router treats '*' as a fallback that matches any provider when no explicit provider name exists in the handler. Handlers with an explicit provider name always take precedence over a wildcard.

Here’s how to route between multiple providers:

import 'dotenv/config';
import { config, shutdown } from '@launchdarkly/ai-server';
import { createOpenAIHandler } from '@launchdarkly/ai-openai-messages';
import { createOpenAIAgentHandler } from '@launchdarkly/ai-openai-agents';
import { createClaudeMessagesHandler } from '@launchdarkly/ai-claude-messages';
import { createClaudeAgentsHandler } from '@launchdarkly/ai-claude-agents';

const result = await config({
  key: 'my-ai-config-flag',
  handler: [
    createOpenAIHandler(),
    createOpenAIAgentHandler(),
    createClaudeMessagesHandler(),
    createClaudeAgentsHandler(),
  ],
}).invoke(
  'What is feature flagging?',
  { kind: 'user', key: 'user-123' },
  { user_name: 'Ada' },
);

console.log(result.response);

// Flush pending LaunchDarkly events and OpenTelemetry spans, then close the connection.
await shutdown();

The argument to invoke() is an optional map of variables. The SDK uses these variables to fill in {{variable}} placeholders in the config’s messages or instructions. The SDK also injects the LDContext under the ldContext key, so templates can reference context attributes with {{ldContext.key}} or {{ldContext.email}}.

invoke() throws on a disabled or unreachable configThere is no fallback argument in this SDK. invoke() throws an error if the variation has targeting off, if the config key does not exist, or if the SDK cannot reach LaunchDarkly. To prevent this, wrap the call sites to return a fallback response.To read the state without throwing, call inspectConfig(key, context). It returns the enabled flag, the parsed config, and the variation metadata. It returns a result even for a config with targeting off or one the SDK cannot reach.

Write a custom handler

This step is optional. If you use a provider that does not have a pre-built package, or you want an internal endpoint, you can use createHandler() to build your own. createHandler(providesFor, fn) attaches routing metadata to a function so it works with config() routing and registries. You can also use a custom handler as a fallback for providers or models you do not support. Register a handler that returns an unsupported-provider response, and config() routes unmatched calls to it. Here’s how to write a custom handler:

import { config, createHandler } from '@launchdarkly/ai-server';

const internalHandler = createHandler(['InternalProvider', 'messages'], async (cfg, userInput) => {
  const response = await fetch('https://models.internal.example.com/generate', {
    method: 'POST',
    body: JSON.stringify({ model: cfg.model.name, prompt: userInput }),
  });
  const { text, usage } = await response.json();
  return { output: text, usage };
});

const result = await config({
  key: 'my-ai-config-flag',
  handler: internalHandler,
}).invoke('What is feature flagging?', { kind: 'user', key: 'user-123' });

Create a registry

A Registry holds a set of handlers and a map of tool implementations so you can register handlers and tools one time and reuse them, rather than passing them with every call. You then pass the registry as the registry option to make the registered tools and handlers available to the call site. Tool keys in the registry must match the tool names defined in the config variation exactly, including case. A mismatch causes a runtime error when the model requests the tool. You can also pass handler and toolHandlers inline on a single call, even when you supply a registry. Inline options always take precedence over the registry, so you can override a handler or tool for one call without changing shared configuration.

Use a scoped registry

We recommend that you use scoped registries where possible because they follow the principle of least privilege. Each registry exposes only the handlers and tools a given call site needs. Here’s how to create and use a scoped registry:

import { Registry, config } from '@launchdarkly/ai-server';
import { createOpenAIHandler } from '@launchdarkly/ai-openai-messages';
import { createOpenAIAgentHandler } from '@launchdarkly/ai-openai-agents';

const openaiRegistry = new Registry({
  handlers: [createOpenAIHandler(), createOpenAIAgentHandler()],
  tools: { 'get-prefs': getPreferencesFn },
});

const result = await config({ key: 'my-ai-config-flag', registry: openaiRegistry })
  .invoke(userInput, context);

To merge two registries without mutating either one, use compose(a, b). When a handler key or tool name appears in both, the second registry takes precedence.

Use the global registry

globalRegistry is a process-wide singleton exported from the core package. Populate it once at startup, then reference it from any call site. Because the global registry is process-wide, use it only when a single set of handlers and tools is appropriate for your whole application. Here’s how to populate and use the global registry:

import { globalRegistry, config } from '@launchdarkly/ai-server';
import { createOpenAIHandler } from '@launchdarkly/ai-openai-messages';
import { createClaudeAgentsHandler, ClaudeWebSearch } from '@launchdarkly/ai-claude-agents';

// Call once at application startup.
globalRegistry.register({
  handlers: [createOpenAIHandler(), createClaudeAgentsHandler()],
  tools: {
    'web-search': ClaudeWebSearch,
    'get-prefs': getPreferencesFn,
  },
});

// Reference the registry from any call site.
const result = await config({ key: 'my-ai-config-flag', registry: globalRegistry })
  .invoke(userInput, context);

Use provider tools

Some provider SDKs expose built-in capabilities that the provider handles natively rather than dispatching to a function you write. For example, the Claude Agent SDK has a built-in tool for web search. To opt in, use a NativeTool sentinel as the tool implementation. Assign LaunchDarkly tool keys to the sentinels so you can use the provider’s built-in tools with LaunchDarkly features such as tool-call tracking. Here’s how to enable the Claude web search tool:

import { config } from '@launchdarkly/ai-server';
import { ClaudeWebSearch, createClaudeAgentsHandler } from '@launchdarkly/ai-claude-agents';

const result = await config({
  key: 'my-ai-config-flag',
  toolHandlers: {
    'web-search': ClaudeWebSearch,          // built-in sentinel, no function needed
    'get-prefs': async ({ id }) => { /* your function */ },
  },
  handler: [createClaudeAgentsHandler()],
}).invoke('What are the latest LaunchDarkly release notes?', { kind: 'user', key: 'user-123' });

The @launchdarkly/ai-claude-agents package exports sentinels for the Claude Agent SDK built-in tools, including ClaudeBash, ClaudeRead, ClaudeEdit, ClaudeWrite, ClaudeGlob, ClaudeGrep, ClaudeWebFetch, ClaudeWebSearch, ClaudeTodoWrite, and ClaudeNotebookEdit.

Run an agent graph

An agent graph is a multi-agent workflow defined in a LaunchDarkly graph object as a root agent with directed edges. Use graph() to run one. The SDK uses a model-driven router. It starts at the root node, presents the outgoing edges as handoff choices to the model, and follows the edge the model selects, threading the original user request and the previous node’s response into each subsequent node. The loop ends when the model produces a final answer, reaches a leaf node, detects a cycle, or hits the step cap. Each node runs through the same path as config(), so every node emits its own telemetry and runs its own judges tagged with the graph key. Because routing is per node, you can build mixed graphs where one node uses OpenAI and another uses Anthropic. Here’s how to run a graph:

import 'dotenv/config';
import { graph, shutdown } from '@launchdarkly/ai-server';
import { createClaudeAgentsHandler } from '@launchdarkly/ai-claude-agents';

const result = await graph('support-graph', {
  handlers: [createClaudeAgentsHandler()],
}).invoke('I was double charged', { kind: 'user', key: 'user-123' }, { account_tier: 'pro' });

console.log(result.response); // final output
console.log(result.usage);    // aggregate token counts

await shutdown();

To evaluate the final graph output with a judge, pass a judge config key as the graphJudge option. The handler packages also export single-provider convenience functions, claudeGraph, openaiGraph, and langchainGraph, that pre-bind their own handler. For mixed-provider graphs, use the base graph() function and pass multiple handlers.

Run a graph on a native framework runner

If you want a provider’s own framework to manage handoffs, tool loops, and conversation state, use resolveGraph() to resolve the topology without executing it, then pass the result to a native runner. A native runner is a function that takes your LaunchDarkly-defined agent graph and hands it off to an AI framework’s own orchestration engine to execute, instead of letting the LaunchDarkly SDK run it. Each handler package ships a runner that converts the resolved graph into the provider’s own multi-agent primitives. Native runners bypass the SDK’s model-driven router, but they still accept the same toolHandlers map as graph(), emit the same $ld:ai:* tracking events, and wrap execution in a parent OpenTelemetry span. The following native runners are available:

Runner	Package	Framework
`toOpenAIAgents`	`@launchdarkly/ai-openai-agents`	OpenAI Agents SDK
`toLangGraph`	`@launchdarkly/ai-langchain-agents`	LangGraph `StateGraph`
`toClaudeAgents`	`@launchdarkly/ai-claude-agents`	Claude Agent SDK

Here’s how to run a graph on LangGraph:

import { resolveGraph } from '@launchdarkly/ai-server';
import { toLangGraph } from '@launchdarkly/ai-langchain-agents';

const result = await toLangGraph(
  resolveGraph('support-graph', { context }),
  { toolHandlers: registry.tools, context },
).invoke('I was double charged');

Use structured output

You can configure a config to return structured output that conforms to a JSON schema. Specify the output format when you view a config in the LaunchDarkly UI. When an output format is set, the SDK applies it to your call automatically, and invoke() returns a parsed object as response instead of a string. First-class providers, such as OpenAI in messages mode, enforce the schema with the provider’s built-in structured-output feature. For providers without built-in support, the SDK enforces the schema on a best-effort basis by appending schema instructions to the prompt. If the model returns output that cannot be parsed, invoke() returns the raw string rather than throwing an error.

Structured output and streaming are mutually exclusiveYou cannot use structured output and streaming at the same time. Streaming ignores any output format set on the config. To receive structured output, use invoke() rather than stream().

Stream a response

To receive a response token by token, use stream() instead of invoke(). stream() returns an async generator that yields a chunk event for each text delta, followed by a single done event that carries the full response, normalized token usage, and any judge results. Here’s how to stream a response:

import { config, globalRegistry } from '@launchdarkly/ai-server';

const stream = config({
  key: 'my-ai-config-flag',
  registry: globalRegistry,
}).stream('What is feature flagging?', { kind: 'user', key: 'user-123' });

for await (const event of stream) {
  if (event.type === 'chunk') {
    process.stdout.write(event.text);
  } else {
    // Final event: full response, usage, and judge results.
    console.log('\n', event.usage);
  }
}

Run judges asynchronously

Judges evaluate a model’s output against criteria you define in a judge config. By default, judges configured on a config run inline, which adds their evaluation time to your response latency. To run judges without increasing latency, evaluate them on a worker thread. Pass skipJudges: true to config() to suppress inline evaluation. invoke() then returns judgeTasks, an array of pre-packaged, serializable tasks, alongside the model response. Spawn a worker thread for each task and call runJudge(task, handlers) inside the worker. Because each task carries everything the judge needs, the worker handles the AI call and the LaunchDarkly tracking event on its own, and your main thread does not block. Here’s how to return judge tasks and hand them to workers:

import { config, globalRegistry } from '@launchdarkly/ai-server';
import { Worker } from 'node:worker_threads';

const { invoke } = config({ key: 'my-ai-config-flag', registry: globalRegistry, skipJudges: true });

const { response, judgeTasks } = await invoke(userInput, context);

// Fire and forget: the main thread continues immediately.
for (const task of judgeTasks ?? []) {
  new Worker(new URL('./judge-worker.js', import.meta.url), { workerData: task });
}

import 'dotenv/config';
import { workerData } from 'node:worker_threads';
import type { JudgeTask } from '@launchdarkly/ai-server';
import { getClient, initClient, runJudge, shutdown } from '@launchdarkly/ai-server';
import { createOpenAIHandler } from '@launchdarkly/ai-openai-messages';

const task = workerData as JudgeTask;

await initClient();
const result = await runJudge(task, [createOpenAIHandler()]);

if (result && task.evaluationMetricKey) {
  getClient().track(task.evaluationMetricKey, task.userContext, result.trackData, result.score);
}

await shutdown();

Configure observability

The SDK records AI metrics, such as duration and token counts, automatically on every call. It also creates OpenTelemetry spans that follow the Gen AI semantic conventions. You do not need to initialize a tracker or wrap your model calls in spans yourself. To export traces to LaunchDarkly observability, make the OpenTelemetry SDK packages available at runtime. You have two options for how to do this:

Install @launchdarkly/ai-otel, which bundles the OpenTelemetry packages the client needs. The client discovers them automatically.
Configure your own OpenTelemetry setup. You are not required to install a LaunchDarkly observability plugin.

At runtime, the client detects whether the OpenTelemetry packages are present. If they are, it configures a tracer provider with a GZIP-compressed OTLP HTTP exporter and W3C trace-context propagators. If they are absent, it logs a one-time warning with the install command and continues. In this case, feature flags and AI calls work normally, and your code still “creates” and “ends” spans on every call with no errors, but nothing is recorded or sent anywhere. The tracing instrumentation is present but inert. The following environment variables configure telemetry:

Variable	Description
`LD_SERVICE_NAME`	The OpenTelemetry `service.name` resource attribute. Defaults to `nodejs-sdk`.
`LD_ENVIRONMENT`	The `deployment.environment` resource attribute, such as `production` or `staging`.
`OTEL_EXPORTER_OTLP_ENDPOINT`	An override for the OTLP endpoint. Defaults to the LaunchDarkly Observability backend.

To learn more, read LLM observability and Monitor configs.

Manage the client lifecycle

The SDK manages a single shared LaunchDarkly client for you. In most applications, the client initializes lazily on your first AI call, so you do not need to call initClient() explicitly. Call initClient() yourself in these cases:

You want to pass custom initialization options.
You run in an edge or custom runtime and supply a pre-initialized client. Call initClient(preInitializedClient) with a client from your platform’s LaunchDarkly SDK before your first AI call.
You want to pre-warm the client at startup rather than on the first request.

Call shutdown() when your application exits to flush pending LaunchDarkly events and OpenTelemetry spans and close the connection. shutdown() is safe to call more than once.

Supported features

The Node.js (server-side) AI SDK builds on the Node.js (server-side) SDK. It supports the same LaunchDarkly features as the legacy Node.js (server-side) AI SDK, but because it is more opinionated, you configure some features differently. The following features work the same way as in the legacy Node.js (server-side) AI SDK. Each one is a matter of constructing an LDContext, which the SDK passes through to config evaluation and metrics unchanged:

Anonymous contexts: Construct a context with anonymous: true.
Context configuration: Construct single-context or multi-context objects.
Private attributes: Mark attributes private with _meta.privateAttributes on the context.

Set SDK-wide private attributes with a pre-initialized clientContext-level private attributes, set with _meta.privateAttributes, work with the SDK without any extra setup. However, the SDK-wide allAttributesPrivate and privateAttributes options belong to the underlying LaunchDarkly SDK, and initClient() does not expose them. To set them, initialize @launchdarkly/node-server-sdk yourself with those options, then pass the client to initClient(). To learn more, read Manage the client lifecycle.

You configure the following features differently in the SDK. For a full walkthrough with before and after examples, read Migrate from the legacy AI SDKs.

Feature	In the legacy Node.js (server-side) AI SDK	In the SDK
Customizing AgentControl configs	You call `completionConfig()`, `agentConfig()`, or `agentConfigs()` to get a customized config, then call the model provider yourself.	You call `config()`, `graph()`, or a provider convenience function, and pass customization variables as the third argument to `invoke()`. To learn more, read Call a model.
Tracking AI metrics	You obtain a tracker from the customized config and record metrics yourself with the tracker’s methods.	The SDK records duration, token, generation, and tool-call metrics automatically on every call, and its handlers create OpenTelemetry spans for you. To learn more, read Configure observability.

​Get started

​Install the SDK

​Configure the environment

​Call a model

​Call a model with a convenience function

​Call a model with a handler

​Write a custom handler

​Create a registry

​Use a scoped registry

​Use the global registry

​Use provider tools

​Run an agent graph

​Run a graph on a native framework runner

​Use structured output

​Stream a response

​Run judges asynchronously

​Configure observability

​Manage the client lifecycle

​Supported features

Get started

Install the SDK

Configure the environment

Call a model

Call a model with a convenience function

Call a model with a handler

Write a custom handler

Create a registry

Use a scoped registry

Use the global registry

Use provider tools

Run an agent graph

Run a graph on a native framework runner

Use structured output

Stream a response

Run judges asynchronously

Configure observability

Manage the client lifecycle

Supported features