Getting started with LangChain and AI Configs

Overview

This guide shows how to use a LangChain chat model with LaunchDarkly AI Configs. LangChain is a framework for building LLM-powered applications that abstracts away provider differences — choose it when you want to swap between OpenAI, Anthropic, Gemini, and other providers without changing your application code. By the end, you will have a working integration that retrieves model configuration and prompts from LaunchDarkly, invokes a LangChain model, and reports metrics back automatically.

This guide uses the official LaunchDarkly LangChain provider packages, which handle model creation, provider mapping, and metrics extraction. This guide uses the low-level model-creation flow so you can see how the pieces fit together. For higher-level helper methods such as direct provider creation and structured-output support, read the Python or Node.js LangChain provider package documentation.

AI Configs support two modes:

• Completion mode returns messages and roles (such as “system,” “user,” “assistant”). Use it for chat-style interactions and message-oriented workflows. Completion mode supports online evaluations with judges attached in the LaunchDarkly user interface (UI).
• Agent mode returns a single instructions string. Use it when your runtime or framework expects a goal/instructions input for a structured workflow. Agent mode changes the configuration shape from messages to instructions. Your application maps these instructions into your provider or framework’s native input.

Both modes support tool calling. LangChain’s provider abstraction works with both modes, letting you swap models without changing your application code. This guide walks through completion mode as the main path, with an optional agent config section. To learn more about when to use each mode, read When to use prompt-based vs agent mode.

This guide provides examples in both Python and Node.js (TypeScript).

Prerequisites

To complete this guide, you need the following:

• A LaunchDarkly account with a SDK key for your environment and a member role that allows AI Config actions. To learn more about LaunchDarkly roles, read Roles.
• An API key for your chosen model provider (OpenAI, Anthropic, or another supported provider).
• A development environment:
  • Python: Python 3.10 or higher
  • Node.js: Node.js 20 or higher
• The LangChain integration package for your model provider installed locally. For example, langchain-openai for OpenAI models, langchain-anthropic for Anthropic models, or langchain-google-genai for Gemini. LangChain raises an ImportError if the provider package is not installed when the model is created.
• Familiarity with LaunchDarkly contexts. To learn more, read Contexts and segments.

Concepts

Before you begin, review these key concepts.

AI Configs

An AI Config is a LaunchDarkly resource that controls how your application uses large language models. Each AI Config contains one or more variations. Each variation specifies:

• A model configuration, including the model name and parameters
• Messages that define the prompt

You can update these settings in LaunchDarkly at any time without changing your application code.

The LaunchDarkly LangChain provider

LaunchDarkly publishes official LangChain provider packages for Python and Node.js. These packages handle model creation, message conversion, and metrics extraction so you do not need to write custom integration code.

The provider creates a LangChain chat model directly from an AI Config, passing through all model parameters and mapping provider names automatically. It also extracts token usage from LangChain responses for tracking in LaunchDarkly.

Contexts

A context represents the end user interacting with your application. LaunchDarkly uses context attributes to:

• Determine which variation to serve based on targeting rules
• Populate {{ ldctx.* }} placeholders in your prompts with context attribute values

Other placeholders, such as {{ topic }}, are populated from the variables argument you pass at runtime.

The tracker

When you retrieve an AI Config, the SDK returns a config object with a tracker property. The tracker records metrics from your LangChain calls, including:

• Generation count
• Input and output tokens
• Latency
• Success and error rates

These metrics appear on the Monitoring tab in LaunchDarkly.

Step 1: Install the SDK

Install the LaunchDarkly AI SDK and the official LangChain provider package. AI Configs are supported by LaunchDarkly server-side SDKs only. The Node.js examples in this guide use the server-side Node.js AI SDK.

Here is how to install the required packages:

$
pip install launchdarkly-server-sdk>=9.4.0
$
pip install launchdarkly-server-sdk-ai>=0.18.0
$
pip install launchdarkly-server-sdk-ai-langchain>=0.5.0
$
pip install langchain>=1.0.0
$
pip install langchain-openai>=0.1.0
$
pip install python-dotenv>=1.0.0

If your AI Config targets a provider whose LangChain package is not installed, the model creation step raises an error at runtime. For Anthropic models, also install langchain-anthropic (Python) or @langchain/anthropic (Node.js). For Gemini models, install langchain-google-genai (Python) or @langchain/google-genai (Node.js).

Create a .env file in your project root with your credentials:

$
# .env
$
LAUNCHDARKLY_SDK_KEY=<your-launchdarkly-sdk-key>
$
OPENAI_API_KEY=<your-openai-api-key>

Step 2: Initialize the clients

Initialize the LaunchDarkly client and the AI client. Store your SDK key in an environment variable.

Here is the initialization code:

1
import os
2
import ldclient
3
from ldclient import Context
4
from ldclient.config import Config
5
from ldai.client import LDAIClient
6
from ldai_langchain import create_langchain_model, convert_messages_to_langchain, get_ai_metrics_from_response
7
from langchain_core.messages import HumanMessage
8
from dotenv import load_dotenv
9
10
load_dotenv()
11
12
SDK_KEY = os.environ.get("LAUNCHDARKLY_SDK_KEY")
13
14
ldclient.set_config(Config(SDK_KEY))
15
ai_client = LDAIClient(ldclient.get())

Replace LAUNCHDARKLY_SDK_KEY with your LaunchDarkly SDK key. You can find your SDK key from the Environments list for your LaunchDarkly project. To learn how, read SDK credentials.

Step 3: Create an AI Config in LaunchDarkly

Create an AI Config to store your model settings and prompts. You can do this through the LaunchDarkly UI, or programmatically using the LaunchDarkly MCP server from an AI coding assistant such as Claude Code or Cursor.

To create the AI Config in the UI:

1. Click Create and select AI Config.
2. In the Create AI Config dialog, select Completion.
3. Enter a name, such as “LangChain assistant”.
4. Click Create.

To create a variation:

1. On the Variations tab, replace “Untitled variation” with a name, such as “GPT-4o Detailed”.
2. Click Select a model and choose the gpt-4o OpenAI model.
3. Click Parameters and set temperature to 0.7 and max_tokens to 2000.
4. Add a system message to define your assistant’s behavior:

You are an expert assistant for {{topic}}. Provide detailed answers with examples.

5. Click Review and save.

To enable targeting:

1. Select the Targeting tab.
2. In the “Default rule” section, click Edit.
3. Set the default rule to serve your variation.
4. Click Review and save.

Step 4: Get the AI Config in your application

Retrieve the AI Config from LaunchDarkly by calling the completion config function. Pass a context that represents the current user.

Here is how to get the AI Config:

1
# Get AI Config
2
context = Context.builder("user-123").kind("user").name("Sandy").build()
3
4
# Pass a default for improved resiliency when the AI config is unavailable
5
# or LaunchDarkly is unreachable. Omit for a disabled default.
6
# Example:
7
#   from ldai.client import AICompletionConfigDefault
8
#   default = AICompletionConfigDefault(
9
#       enabled=True,
10
#       model={"name": "gpt-5"},
11
#       provider={"name": "openai"},
12
#       messages=[{"role": "system", "content": "You are a helpful assistant."}],
13
#   )
14
#   config = ai_client.completion_config(CONFIG_KEY, context, default, variables={"topic": "Python"})
15
config = ai_client.completion_config(CONFIG_KEY, context, variables={"topic": "Python"})
16
17
if not config.enabled:
18
    raise RuntimeError("AI Config is disabled")

The fallback argument is optional. If omitted (Python) or passed as undefined (Node.js), the SDK returns a disabled config as the fallback. Check the enabled property and handle the disabled case in your application.

Step 5: Create a LangChain model from the AI Config

Use the LaunchDarkly LangChain provider to create a chat model from the AI Config. The provider reads the model name, provider, and all parameters (temperature, max tokens, and others) from the variation, maps LaunchDarkly provider names to LangChain equivalents, and returns a configured chat model.

Here is how to create the model:

1
# create_langchain_model reads config.model.name / .parameters and picks the
2
# right chat model class (OpenAI, Anthropic, …) with no per-provider branching.
3
llm = create_langchain_model(config)
4
5
# Convert AI Config messages to LangChain format and add the user turn
6
messages = convert_messages_to_langchain(config.messages or [])
7
messages.append(HumanMessage(content="How do I read a file in Python?"))

The provider maps LaunchDarkly provider names to LangChain equivalents — for example, "gemini" maps to "google_genai". All other provider names pass through lowercased. LangChain raises an ImportError (Python) or Error (Node.js) at model creation time if the corresponding provider package is not installed.

Step 6: Call the model and track metrics

Combine the messages from your AI Config with user input, then use the tracker to call the model and record metrics automatically.

Here is how to make the API call:

1
# Track metrics automatically with track_metrics_of_async
2
tracker = config.create_tracker()
3
4
completion = await tracker.track_metrics_of_async(
5
    lambda: llm.ainvoke(messages),
6
    get_ai_metrics_from_response
7
)
8
print(completion.content)

The tracker wraps the model call and automatically records duration, token usage, and success or error status.

Step 7: Monitor your AI Config

View metrics for your AI Configs in the LaunchDarkly UI.

To view aggregated metrics across all your AI Configs, navigate to Insights in the left navigation under the AI section. The Insights overview page displays cost, latency, error rate, invocation counts, and model distribution across your organization. To learn more, read about AI insights.

To view metrics for a specific AI Config:

1. Navigate to your AI Config.
2. Select the Monitoring tab.

The Monitoring tab displays:

• Generation count: number of successful model invocations
• Token usage: input and output tokens per variation
• Time to generate: average latency per generation
• Error rate: percentage of failed invocations
• Costs: estimated spend based on token usage and model pricing

Metrics update approximately every minute. Use these metrics to compare variations and optimize your prompts. To learn more, read Monitor AI Configs.

To run statistical comparisons between variations, read Run experiments with AI Configs. To score response quality automatically using judges, read Online evaluations in AI Configs.

For multi-agent workflows, the LangChain-based LangGraph framework handles agent orchestration natively. To learn more, read Compare AI orchestrators.

Step 8: Close the client

Close the LaunchDarkly client when your application shuts down to flush pending events. Always flush before closing. Trailing events are at risk of being lost otherwise, in short-lived scripts and long-running services alike.

Here is how to close the client:

1
# Always flush events before closing. Trailing events are at risk of being
2
# lost otherwise, in short-lived scripts and long-running services alike.
3
ldclient.get().flush()
4
ldclient.get().close()

Complete example

Here is a complete working example that combines all the steps.

1
import os
2
import asyncio
3
import ldclient
4
from ldclient import Context
5
from ldclient.config import Config
6
from ldai.client import LDAIClient
7
from ldai_langchain import create_langchain_model, convert_messages_to_langchain, get_ai_metrics_from_response
8
from langchain_core.messages import HumanMessage
9
from dotenv import load_dotenv
10
11
load_dotenv()
12
13
SDK_KEY = os.environ.get("LAUNCHDARKLY_SDK_KEY")
14
CONFIG_KEY = "langchain-assistant"
15
16
17
async def async_main():
18
    ldclient.set_config(Config(SDK_KEY))
19
    if not ldclient.get().is_initialized():
20
        return
21
22
    ai_client = LDAIClient(ldclient.get())
23
24
    context = Context.builder("user-123").kind("user").name("Sandy").build()
25
26
    # LangChain's value: one AI Config key can serve OpenAI, Anthropic, or any
27
    # other provider-backed variation. create_langchain_model picks the right
28
    # chat model class and applies all parameters from the variation automatically.
29
    #
30
    # Pass a default for improved resiliency when the AI config is unavailable
31
    # or LaunchDarkly is unreachable. Omit for a disabled default.
32
    # Example:
33
    #   from ldai.client import AICompletionConfigDefault
34
    #   default = AICompletionConfigDefault(
35
    #       enabled=True,
36
    #       model={"name": "gpt-5"},
37
    #       provider={"name": "openai"},
38
    #       messages=[{"role": "system", "content": "You are a helpful assistant."}],
39
    #   )
40
    #   config = ai_client.completion_config(CONFIG_KEY, context, default, variables={"topic": "Python"})
41
    config = ai_client.completion_config(CONFIG_KEY, context, variables={"topic": "Python"})
42
43
    if not config.enabled:
44
        return
45
46
    tracker = config.create_tracker()
47
48
    llm = create_langchain_model(config)
49
50
    messages = convert_messages_to_langchain(config.messages or [])
51
    messages.append(HumanMessage(content="How do I read a file in Python?"))
52
53
    try:
54
        completion = await tracker.track_metrics_of_async(
55
            lambda: llm.ainvoke(messages),
56
            get_ai_metrics_from_response,
57
        )
58
        print(completion.content)
59
    except Exception as e:
60
        print(f"Error: {e}")
61
62
    # Always flush events before closing. Trailing events are at risk of being
63
    # lost otherwise, in short-lived scripts and long-running services alike.
64
    ldclient.get().flush()
65
    ldclient.get().close()
66
67
68
def main():
69
    asyncio.run(async_main())
70
71
72
if __name__ == "__main__":
73
    main()

What to explore next

After you have the basic integration working, you can extend it with:

• Tools for calling external functions from your workflows
• Online evaluations to score response quality automatically
• Experiments to compare AI Config variations statistically
• Agents for multi-step workflows with LangChain or LangGraph

For structured outputs and higher-level helper methods, read the Python or Node.js LangChain provider package documentation.

Troubleshooting

Some solutions for common problems are outlined below.

Provider package not installed

If you receive ImportError (Python) or Error (Node.js) when creating the model, verify that the LangChain integration package for your provider is installed. For example, if your AI Config uses an OpenAI model, install langchain-openai (Python) or @langchain/openai (Node.js). The LaunchDarkly LangChain provider creates models using LangChain’s init_chat_model, which requires the correct provider package at runtime.

Metrics not appearing

If metrics do not appear on the Monitoring tab:

• Verify that you are calling tracker.track_metrics_of_async() (Python) or tracker.trackMetricsOf() (Node.js) with the LangChain metrics extractor.
• Ensure you call flush() before closing the client, especially for short-lived scripts.
• Wait at least one minute for metrics to process.

SDK initialization failures

If the LaunchDarkly SDK fails to initialize:

• Verify your SDK key is correct and matches the environment you are targeting.
• Check that your network can reach LaunchDarkly servers.
• Review the SDK logs for specific error messages.

Config returns fallback value

If you always receive the fallback configuration:

• Verify targeting is enabled for your AI Config.
• Check that the AI Config key in your code matches the key in LaunchDarkly.
• Ensure your context matches the targeting rules.

Package import errors

If you receive ModuleNotFoundError (Python) or Cannot find module (Node.js) for the LaunchDarkly packages:

• Verify the LangChain provider package is installed. The Python package is launchdarkly-server-sdk-ai-langchain (imported as ldai_langchain). The Node.js package is @launchdarkly/server-sdk-ai-langchain.
• Ensure you are using compatible current versions of launchdarkly-server-sdk-ai, launchdarkly-server-sdk-ai-langchain, and your LangChain provider package. Check the package documentation or release notes if you encounter import or runtime errors.

Conclusion

In this guide, you connected a LangChain chat model to LaunchDarkly AI Configs using the official LangChain provider packages. You can now:

• Manage prompts and model settings in LaunchDarkly without code changes
• Track token usage, latency, and success rates automatically
• Swap providers and models without changing application code