> ## Documentation Index
> Fetch the complete documentation index at: https://launchdarkly.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# When to use completion mode vs agent mode in LaunchDarkly for AI applications: a guide for LangGraph, OpenAI, and Multi-Agent Systems

<View title="Developer" />

<View title="Federal docs" />

<View title="EU docs" />

The broader tech industry can't agree on what the term "agents" even means. [Anthropic defines agents](https://www.anthropic.com/research/building-effective-agents) as systems where "LLMs dynamically direct their own processes," while Vercel's AI SDK enables [multi-step agent loops with tools](https://sdk.vercel.ai/docs/concepts/tools), and [OpenAI provides an Agents SDK](https://platform.openai.com/docs/guides/agents-sdk) with built-in orchestration. So when you're creating an AgentControl config in LaunchDarkly and see "completion mode" vs. "agent mode," you might reasonably expect this choice to determine whether you get automatic tool execution loops, server-side state management, or some other fundamental capability difference.

But LaunchDarkly's distinction is different and more practical. Understanding it will save you from confusion and help you ship AI features faster.

## TL;DR

LaunchDarkly's "completion mode vs. agent mode" choice is about **input schemas and framework compatibility**, not execution automation. **Completion mode** returns a messages array (perfect for chat UIs), while **agent mode** returns an instructions string (optimized for LangGraph/CrewAI frameworks). Both provide the same core benefits: provider abstraction, A/B testing, metrics tracking, and the ability to change AI behavior without deploying code.

<Tip>
  **Ready to start?** [Sign up for a free trial](http://app.launchdarkly.com/signup) → [create your first AgentControl config](/docs/home/agentcontrol/create) → Choose your mode → Configure and ship.
</Tip>

## The fragmented AI landscape

LaunchDarkly supports 20+ AI providers: OpenAI, Anthropic, Gemini, Azure, Bedrock, Cohere, Mistral, DeepSeek, Perplexity, and more. Each has their own interpretation of "completions" vs "agents," creating a chaotic ecosystem with different API endpoints, execution behaviors, state management approaches, and capability limitations. This fragmentation makes it difficult to switch providers or even understand what capabilities you're getting. That's where LaunchDarkly's abstraction layer comes in.

## LaunchDarkly's approach: provider-agnostic input schemas

LaunchDarkly's AgentControl configs are a **configuration layer** that abstracts provider differences. When you choose completion mode or agent mode, you're selecting an **input schema** (either a messages array or an instructions string), not execution behavior. LaunchDarkly provides the configuration, and you handle orchestration with your own code or frameworks like LangGraph. This gives you provider abstraction, A/B testing, metrics tracking, and online evals in completion mode and agent mode, without locking you into any specific provider's execution model.

<br />

<Frame caption="Creating an AgentControl config: choosing between completion and agent-based modes.">
  <img src="https://mintcdn.com/launchdarkly/WYiCaDy82H6mz_dK/images/auto/agentcontrol-create.auto.png?fit=max&auto=format&n=WYiCaDy82H6mz_dK&q=85&s=93a944bf7a41e41bf3a674211a3d51ff" alt="AgentControl config mode selection" width="920" height="920" data-path="images/auto/agentcontrol-create.auto.png" />
</Frame>

<br />

### Completion mode: messages-based

Completion mode uses a **messages array** format with system/user/assistant roles (some providers like OpenAI also support a "developer" role for more granular control). This is the traditional chat format that works across all AI providers.

**UI Input**: "Messages" section with role-based messages

**SDK Method**: `aiclient.config()`

**Returns**: Customized prompt + model configuration

**Documentation**: [AgentControl docs](/docs/sdk/features/agentcontrol-config)

```python expandable lines wrap theme={null}
# Retrieve completion mode config
config = aiclient.config(
    key="customer-support",
    context=context,
    default_value=default_config
)

# What you get back: messages array
print(config.messages)
# [
#   {
#     "role": "system",
#     "content": "You are a helpful customer support agent for Acme Corp."
#   },
#   {
#     "role": "user",
#     "content": "How can I reset my password?"
#   }
# ]

# Use with provider SDKs that expect message arrays
response = openai.chat.completions.create(
    model=config.model.name,
    messages=config.messages  # Standard message format
)
```

**When to use completion mode:**

1. **You're building chat-style interactions**: Traditional message-based conversations where you construct system/user/assistant messages
2. **You want granular control of workflows**: Discrete steps that need to be accomplished in a specific order, or multi-step asynchronous processes where each step executes independently
3. **One-off evaluations**: Issue individual evaluations of your prompts and completions (not online evals)
4. **Simple processing tasks**: Summarization, name suggestions, or other non-context-exceeding data processing

<br />

<Frame caption="Completion mode variation showing the Messages section with role-based prompts.">
  <img src="https://mintcdn.com/launchdarkly/EyN0Ggc2IAGaGMne/images/tutorials/agent-vs-completion/completion-mode-messages.png?fit=max&auto=format&n=EyN0Ggc2IAGaGMne&q=85&s=46e6c94bf027c4b8e891f9d266f17cef" alt="Completion Mode Messages UI" width="1794" height="1272" data-path="images/tutorials/agent-vs-completion/completion-mode-messages.png" />
</Frame>

<br />

### Agent mode: goal/instructions-based

Agent mode uses a **single instructions string** format that describes the agent's goal or task. This format is optimized for agent orchestration frameworks that expect high-level objectives rather than conversational messages.

**UI Input**: "Goal or task" field with instructions

**SDK Method**: `aiclient.agent()`

**Returns**: Customized instructions + model configuration

**Examples**: [hello-python-ai examples](https://github.com/launchdarkly/hello-python-ai/tree/main)

```python expandable lines wrap theme={null}
# Retrieve agent-based config
agent_config = aiclient.agent(
    key="research-assistant",
    context=context,
    default_value=default_config
)

# What you get back: instructions string
print(agent_config.instructions)
# "You are a research assistant. Your goal is to gather comprehensive
# information on the requested topic using available search tools.
# Search multiple sources, synthesize findings, and provide a detailed
# summary with citations."

# Use with agent frameworks that expect instructions
from langgraph.prebuilt import create_react_agent
from langchain_openai import init_chat_model

llm = init_chat_model(
    model=agent_config.model.name,
    model_provider=agent_config.provider.name
)

agent = create_react_agent(
    llm,
    tools=[search_tool, citation_tool],
    prompt=agent_config.instructions  # Goal/task instructions
)

# Execute and track
response = agent.invoke({"messages": [{"role": "user", "content": "..."}]})
```

**When to use agent mode:**

1. **You're using agent frameworks**: LangGraph, LangChain, CrewAI, AutoGen, or LlamaIndex Workflows expect goal/instruction-based inputs
2. **Goal-oriented tasks**: "Research X and create Y" rather than conversational message exchange
3. **Tool-driven workflows**: While both modes support tools, agent mode's format is optimized for frameworks that orchestrate tool usage
4. **Open-ended exploration**: The output is open-ended and you don't know the actual answer you're trying to get to
5. **Data as an application**: You want to treat your data as an application to feed in arbitrary data and ask questions about it
6. **Provider agent endpoints**: LaunchDarkly may route to provider-specific agent APIs when available (note: not all models support agent mode; check your model's capabilities)

**See example:** [Build a LangGraph Multi-Agent System with LaunchDarkly](/docs/tutorials/agents-langgraph)

## Quick comparison

<table>
  <thead>
    <tr>
      <th>Feature</th>
      <th>Completion Mode</th>
      <th>Agent Mode</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>Input format</td>
      <td>Messages (system/user/assistant)</td>
      <td>Goal/task + instructions</td>
    </tr>

    <tr>
      <td>Tools support</td>
      <td>✅ Yes</td>
      <td>✅ Yes</td>
    </tr>

    <tr>
      <td>SDK method</td>
      <td><code>config()</code></td>
      <td><code>agent()</code></td>
    </tr>

    <tr>
      <td>Automatic execution loop</td>
      <td>❌ No (you orchestrate)</td>
      <td>❌ No (you orchestrate)</td>
    </tr>

    <tr>
      <td>Online evals</td>
      <td>✅ Available</td>
      <td>✅ Available</td>
    </tr>

    <tr>
      <td>Best for</td>
      <td>Chat-style prompting, single completions</td>
      <td>Agent frameworks, goal-oriented tasks</td>
    </tr>

    <tr>
      <td>Provider endpoint</td>
      <td>Standard endpoint</td>
      <td>May use provider-specific agent endpoint if available</td>
    </tr>

    <tr>
      <td>Model support</td>
      <td>All models</td>
      <td>Most models (check model card for "Agent mode" capability)</td>
    </tr>
  </tbody>
</table>

<Note>
  **Model compatibility**: Not all models support agent mode. When selecting a model in LaunchDarkly, check the model card for "Agent mode" capability. Models like GPT-4.1, GPT-5 mini, Claude Haiku 4.5, Claude Sonnet 4.5, Claude Sonnet 4, Grok Code Fast 1, and Raptor mini support agent mode, while models focused on reasoning (like GPT-5, Claude Opus 4.1) may only support completion mode.
</Note>

## How providers handle "completion vs agent"

To understand why LaunchDarkly's abstraction is valuable, let's look at how major AI providers handle the distinction between basic completions and advanced agent capabilities. The table below shows how different providers implement "advanced" modes; generally these are ADDITIVE, including all basic capabilities plus extras. For example, OpenAI's Responses API includes all Chat Completions features plus additional capabilities.

<table>
  <thead>
    <tr>
      <th>Provider</th>
      <th>"Basic" Mode</th>
      <th>"Advanced" Mode</th>
      <th>Key Difference</th>
      <th>Link</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td><strong>OpenAI</strong></td>
      <td>Chat Completions API</td>
      <td>Responses API</td>
      <td>Responses adds built-in tools (web\_search, file\_search, computer\_use, code\_interpreter, remote MCP), server-side conversation state with stored IDs, and improved streaming. Chat Completions remains supported.</td>
      <td><a href="https://platform.openai.com/docs/guides/responses-vs-chat-completions">Docs</a></td>
    </tr>

    <tr>
      <td><strong>Anthropic</strong></td>
      <td>Tool Use (client tools)</td>
      <td>Tool Use (client + server tools)</td>
      <td>Server tools (web\_search, web\_fetch) execute on Anthropic's servers. You can use both client and server tools together</td>
      <td><a href="https://docs.claude.com/en/docs/agents-and-tools/tool-use/overview">Docs</a></td>
    </tr>

    <tr>
      <td><strong>Google Gemini</strong></td>
      <td>Manual function calling</td>
      <td>Automatic function calling (Python SDK)</td>
      <td>Python SDK auto-converts functions to schemas, runs the execution loop, and supports compositional multi-step calls. Manual mode: full control, all platforms</td>
      <td><a href="https://ai.google.dev/gemini-api/docs/function-calling">Docs</a></td>
    </tr>

    <tr>
      <td><strong>Vercel AI SDK</strong></td>
      <td><code>generateText()</code></td>
      <td><code>generateText()</code> with multi-step loop</td>
      <td>Multi-step agent loops with tools; SDK continues until complete; <code>maxSteps</code> provides loop control to limit steps</td>
      <td><a href="https://sdk.vercel.ai/docs/concepts/tools">Docs</a></td>
    </tr>

    <tr>
      <td><strong>Azure OpenAI</strong></td>
      <td>Assistants API (deprecated)</td>
      <td>AI Agent Services</td>
      <td>Enterprise agent runtime with threads, tool orchestration, safety, identity, networking, and observability; includes Responses API and Computer-Using Agent in Azure</td>
      <td><a href="https://azure.microsoft.com/en-us/blog/announcing-the-responses-api-and-computer-using-agent-in-azure-ai-foundry/">Docs</a></td>
    </tr>

    <tr>
      <td><strong>AWS Bedrock (Nova)</strong></td>
      <td>Converse API (tool use)</td>
      <td>Bedrock Agents</td>
      <td>Agents: managed service with automatic orchestration + state management + multi-agent collaboration. Converse: manual tool orchestration, full control</td>
      <td><a href="https://docs.aws.amazon.com/nova/latest/userguide/agents-use-nova.html">Docs</a></td>
    </tr>

    <tr>
      <td><strong>Cohere</strong></td>
      <td>Standard chat</td>
      <td>Command A</td>
      <td>Command A: enhanced multi-step tool use, REACT agents, \~150% higher throughput</td>
      <td><a href="https://docs.cohere.com/docs/command-a">Docs</a></td>
    </tr>
  </tbody>
</table>

This fragmentation across providers is exactly why LaunchDarkly's approach matters: you configure once (messages vs. goals), and LaunchDarkly handles the provider-specific translation. Want to switch from OpenAI to Anthropic? Just change the provider in your AgentControl config. Your application code stays the same.

<Info>
  **Note on OpenAI's ecosystem (Nov 2025)**: The [Agents SDK](https://platform.openai.com/docs/guides/agents-sdk) is OpenAI's production-ready orchestration framework. It uses the Responses API by default, and via a built-in LiteLLM adapter it can run against other providers with an OpenAI-compatible shape. Chat Completions is still supported, but OpenAI recommends Responses for new work. The [Assistants API is deprecated](https://platform.openai.com/docs/assistants/whats-new) and scheduled to shut down on August 26, 2026.
</Info>

## Common misconceptions

Now that you understand the modes and how they differ from provider-specific implementations, let's clear up some common points of confusion:

**❌ "Agent mode provides automatic execution"**
No. Both modes require you to orchestrate. Agent mode just provides a different input schema.

**❌ "Agent mode is for complex tasks, completion mode is for simple ones"**
Not quite. It's about input format and framework compatibility, not task complexity.

**❌ "I can only use tools in agent mode"**
False. Both modes support tools. The difference is how you specify your task (messages vs. goal).

**❌ "LaunchDarkly is an agent framework like LangGraph"**
No. LaunchDarkly is configuration management for AI. Use it WITH frameworks like LangGraph, not instead of them.

## Why LaunchDarkly's abstraction matters

Now that you've seen how fragmented the provider landscape is, let's explore the practical value of LaunchDarkly's abstraction layer.

### Switching providers without code changes

**Without LaunchDarkly:**

```python lines wrap theme={null}
# Hardcoded provider and prompts in your application
openai_client = openai.OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
response = openai_client.chat.completions.create(
    model="gpt-4",  # Want to switch to Claude? Need to deploy new code
    messages=[
        {"role": "system", "content": "You are helpful"},  # Want to A/B test prompts? Deploy again
        {"role": "user", "content": "Hello"}
    ]
)

# To switch providers, you need to:
# 1. Write new code for different provider API
# 2. Deploy to production
# 3. Hope nothing breaks
```

**With LaunchDarkly:**

```python expandable lines wrap theme={null}
# Get config from LaunchDarkly
config = aiclient.config(key="my-ai-config", context=context)

# You still write provider-specific code, but only once
if config.provider.name == "openai":
    client = openai.OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
    response = client.chat.completions.create(
        model=config.model.name,      # Comes from LaunchDarkly
        messages=config.messages,      # Normalized schema across providers
        temperature=config.model.parameters.get('temperature')
    )
elif config.provider.name == "bedrock":
    client = boto3.client('bedrock-runtime', region_name='us-east-1')
    response = client.converse(
        modelId=config.model.name,    # Comes from LaunchDarkly
        messages=_convert_to_bedrock_format(config.messages),  # LaunchDarkly normalizes, you convert
        inferenceConfig={'temperature': config.model.parameters.get('temperature')}
    )

# Now you can switch providers via LaunchDarkly UI without deployment
# Change prompts, A/B test models, roll out gradually - all via configuration
```

**The real value:** Once your code is set up to handle different providers, you can switch between them, change prompts, A/B test models, and roll out changes gradually - all through the LaunchDarkly UI without deploying code. You write the provider handlers once; you manage AI behavior forever.

### Security and risk management

AI agents can be powerful and potentially risky. With LaunchDarkly AgentControl, you can:

* **Instantly disable problematic models or tools** without deploying code
* **Gradually roll out new agent capabilities** to a small percentage of users first
* **Quickly roll back** if an agent behaves unexpectedly
* **Control access by user tier** (limit powerful tools to trusted users)
* **Target specific individuals in production** to test experimental AI behavior in real environments without affecting other users

When you're not directly coupled to provider APIs, responding to security issues becomes a configuration change instead of an emergency deployment.

## Advanced: Provider-specific packages (JavaScript/TypeScript)

For JavaScript/TypeScript developers looking to reduce boilerplate even further, LaunchDarkly offers optional provider-specific packages. These work with **both completion and agent modes** and are purely additive - you don't need them to use LaunchDarkly AgentControl effectively.

**Available packages:**

* [`@launchdarkly/server-sdk-ai-openai`](/docs/sdk/ai/node-js) - OpenAI provider
* [`@launchdarkly/server-sdk-ai-langchain`](/docs/sdk/ai/node-js) - LangChain provider (works with both LangChain and LangGraph)
* [`@launchdarkly/server-sdk-ai-vercel`](/docs/sdk/ai/node-js) - Vercel AI SDK provider

**What they provide:**

* **Model creation helpers**: One-line functions like `createLangChainModel(aiConfig)` that return fully-configured model instances
* **Automatic metrics tracking**: Integrated metrics collection
* **Format conversion utilities**: Helper functions to translate between schemas

**Example with LangGraph:**

```javascript lines wrap theme={null}
// Get agent config from LaunchDarkly
const agentConfig = await ldClient.aiAgent('research-assistant', context);

// Create LangChain model - config already applied
const model = await LangChainProvider.createLangChainModel(agentConfig);

// Use with LangGraph
const agent = createReactAgent(model, tools, agentConfig.instructions);
const response = await agent.invoke({ messages: [{ role: "user", content: "Research X" }] });
```

<Note>
  **Production readiness:** These packages are in **early development** and not recommended for production. They may change without notice.
</Note>

**Python approach:** The Python SDK takes a different path with built-in convenience methods like `track_openai_metrics()` in the single `launchdarkly-server-sdk-ai` package. See [Python AI SDK reference](/docs/sdk/ai/python).

## Start building with LaunchDarkly AgentControl

You now understand how LaunchDarkly's completion and agent modes provide provider-agnostic configuration for your AI applications. Whether you're building chat interfaces or complex multi-agent systems, LaunchDarkly gives you the flexibility to experiment, iterate, and ship AI features without the complexity of managing multiple provider APIs.

**Choosing your mode:**

**Start with completion mode if:**

* You're building a chat interface or conversational UI
* You want precise control over multi-step workflows
* You're uncertain which mode fits your use case (it's the more flexible starting point)

**Choose agent mode if:**

* You're integrating with LangGraph, LangChain, CrewAI, or similar frameworks
* Your task is goal-oriented rather than conversational ("Research X and create Y")
* You're feeding arbitrary data and asking open-ended questions about it

**Remember:** Both modes give you the same core benefits: provider abstraction, A/B testing, and runtime configuration changes. The choice is about input format, not capabilities.

**Get started:**

1. [Sign up for a free LaunchDarkly account](https://app.launchdarkly.com/signup)
2. [Create your first AgentControl config](/docs/home/agentcontrol/create): Takes less than 5 minutes
3. [Explore example implementations](https://github.com/launchdarkly/hello-python-ai/tree/main): Learn from working code
4. Start with completion mode unless you're specifically using an agent framework

## Further reading

**LaunchDarkly resources:**

* [AgentControl quickstart guide](/docs/home/agentcontrol/quickstart)
* [Online evaluations in AgentControl](/docs/home/agentcontrol/online-evaluations)
* [Python AI SDK reference](/docs/sdk/ai/python)

**Provider documentation:**

* [Anthropic Building Effective Agents](https://www.anthropic.com/research/building-effective-agents)
* [Google Gemini Function Calling](https://ai.google.dev/gemini-api/docs/function-calling)
* [OpenAI Responses API vs Chat Completions](https://platform.openai.com/docs/guides/responses-vs-chat-completions)