Building a chatbot with multiple AI providers using AgentControl configs

This guide shows you how to build a simple AI-powered chatbot using LaunchDarkly AgentControl with multiple AI providers, including Anthropic, OpenAI, and Google. Using AgentControl, you can manage models and prompts outside of code, switch providers without redeploying, and monitor performance in real time. You’ll learn how to:

Create a basic chatbot application
Configure AI models dynamically without code changes
Create and manage multiple AgentControl config variations
Apply user contexts for personalizing AI behavior
Switch between different AI providers seamlessly
Monitor and track AI performance metrics

By the end of this tutorial, you’ll have a working chatbot that demonstrates LaunchDarkly’s AgentControl capabilities across multiple providers. The complete code for this tutorial is available in the simple-chatbot repository. For additional code examples and implementations, check out the LaunchDarkly Python AI Examples repository, which includes practical examples of AgentControl configs with various providers and use cases.

Prerequisites

Before you begin, you need the following:

Required accounts

Access to the following accounts:

A LaunchDarkly account: Sign up at app.launchdarkly.com
At least one AI provider account:
- Anthropic: console.anthropic.com
- OpenAI: platform.openai.com
- Google AI: ai.google.dev

Development environment requirements

A development environment with:

Python 3.8 or later
pip package manager
Basic Python knowledge
A code editor, such as VS Code or PyCharm

API and SDK keys

The following keys:

An active LaunchDarkly SDK key
An API key from at least one AI provider

Before you start

This guide builds a chatbot using completion-based AgentControl configs in a messages array format. If you use LangGraph or CrewAI, you may want to use agent mode instead. The following sections include best practices to help you avoid common issues and reduce debugging time.

Do not cache configs across users

Reusing configs across users breaks targeting. Instead, fetch a fresh config for each request:

# This breaks targeting - all users get the first user's config
config = ai_client.completion_config("my-key", first_user_context, fallback)
for user in users:
    response = generate(config, user.message)  # Wrong!

# Fresh config per request
for user in users:
    config = ai_client.completion_config("my-key", user.context, fallback)
    response = generate(config, user.message)

Provide a fallback config

Provide a fallback so your application does not crash when unexpected issues occur, such as LaunchDarkly being unavailable or API keys being incorrect:

fallback = AICompletionConfigDefault(
    enabled=True,
    model=ModelConfig(name="claude-3-haiku-20240307"),
    messages=[LDMessage(role="system", content="You are helpful")]
)

config = ai_client.completion_config("my-key", context, fallback)

Check if the config is enabled

Check if the config is enabled before using it:

if config.enabled:
    response = call_ai_provider(config)
else:
    response = "AI is temporarily unavailable"

Do not include personally identifiable information (PII) in contexts

Never send PII to LaunchDarkly. Here’s a bad and a good example:

# Bad - do not include PII
context = Context.builder(user.email)

# Good - opaque ID
context = Context.builder(user.id)  # "usr_abc123"
    .set("tier", "premium")  # Non-PII attributes are fine

Limit conversation history

Your chat history grows with every turn. After 50 exchanges, each request may include thousands of tokens. Here’s how to limit it:

MAX_TURNS = 20

def add_to_history(history, role, content):
    history.append({"role": role, "content": content})
    if len(history) > MAX_TURNS:
        history = [history[0]] + history[-(MAX_TURNS-1):]  # Keep system prompt
    return history

Track token usage

Without tracking, it is difficult to understand how token usage affects cost. Here’s how to track token usage:

import time
from ldai.tracker import TokenUsage

# Track duration
start = time.time()
response = client.generate(messages)  # Get full response object
tracker.track_duration(time.time() - start)

# Track tokens
if hasattr(response, 'usage'):
    tracker.track_tokens(TokenUsage(
        input=response.usage.input_tokens,
        output=response.usage.output_tokens,
        total=response.usage.input_tokens + response.usage.output_tokens
    ))

# Track success/error
tracker.track_success()  # or tracker.track_error("error message")

# Extract text for display
text = response.content[0].text  # Anthropic
# or response.choices[0].message.content  # OpenAI
# or response.text  # Google

Your provider methods should return the full response object, not just text, so you can access usage metadata. The code examples here return full responses where tracking is needed.

Example 1: Your first chatbot

Start by building a minimal chatbot application using a LaunchDarkly AgentControl config with Anthropic’s Claude.

Step 1.1: Project setup

First, create a new directory for your project:

mkdir simple-ai-chatbot
cd simple-ai-chatbot

Then, create a virtual environment and activate it:

python3 -m venv venv
source venv/bin/activate
# On Windows: venv\Scripts\activate

Step 1.2: Install dependencies

Install the required packages:

pip install launchdarkly-server-sdk \
            launchdarkly-server-sdk-ai \
            anthropic \
            openai \
            google-genai \
            python-dotenv

Create a requirements.txt file:

launchdarkly-server-sdk>=9.0.0
launchdarkly-server-sdk-ai>=0.20.0
anthropic>=0.25.0
openai>=1.0.0
google-genai>=0.1.0
python-dotenv>=1.0.0

Step 1.3: Environment configuration

First, add .env to your .gitignore file to keep credentials secure:

echo ".env" >> .gitignore

Now create a .env file in your project root:

# LaunchDarkly Configuration
LD_PROJECT_KEY=simple-chatbot
LD_SDK_KEY=your-launchdarkly-sdk-key
LAUNCHDARKLY_AGENT_CONFIG_KEY=simple-config

# AI Provider API Keys (add the ones you plan to use)
ANTHROPIC_API_KEY=your-anthropic-api-key
OPENAI_API_KEY=your-openai-api-key
GEMINI_API_KEY=your-google-api-key

Step 1.4: Create the basic chatbot

Create a file called simple_chatbot.py and add the following:

Click to expand the complete simple_chatbot.py

"""
Simple AI Chatbot
Multi-provider support: Anthropic, OpenAI, and Google
Direct API integration with automatic provider selection
"""

import os
import logging
from typing import Dict, List, Optional
from abc import ABC, abstractmethod
import dotenv

# AI Provider imports
import anthropic
import openai
import google.genai as genai

# Set up logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)

# Suppress HTTP request logs from libraries
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("httpcore").setLevel(logging.WARNING)
logging.getLogger("openai").setLevel(logging.WARNING)
logging.getLogger("anthropic").setLevel(logging.WARNING)

# Load environment variables
dotenv.load_dotenv()


class BaseAIProvider(ABC):
    """Base class for AI providers"""

    def __init__(self, api_key: Optional[str] = None):
        self.api_key = api_key
        self.client = self._initialize_client() if api_key else None

    @abstractmethod
    def _initialize_client(self):
        """Initialize the provider's client"""
        pass

    @abstractmethod
    def send_message(self, model: str, messages: List[Dict], system_prompt: str, params: Dict) -> str:
        """Send message to the AI provider"""
        pass

    def format_messages(self, messages: List[Dict], system_prompt: str) -> List[Dict]:
        """Default message formatting (can be overridden by providers)"""
        formatted = [{"role": "system", "content": system_prompt}] if system_prompt else []
        formatted.extend([{"role": msg["role"], "content": msg["content"]} for msg in messages])
        return formatted

    def extract_params(self, params: Dict) -> Dict:
        """Extract common parameters"""
        return {
            "temperature": params.get("temperature", 0.7),
            "max_tokens": params.get("max_tokens", 500)
        }


class AnthropicProvider(BaseAIProvider):
    """Anthropic Claude provider"""

    def _initialize_client(self):
        return anthropic.Anthropic(api_key=self.api_key)

    def send_message(self, model: str, messages: List[Dict], system_prompt: str, params: Dict) -> str:
        if not self.client:
            raise ValueError("Anthropic API key not configured")

        extracted_params = self.extract_params(params)

        response = self.client.messages.create(
            model=model,
            max_tokens=extracted_params["max_tokens"],
            temperature=extracted_params["temperature"],
            system=system_prompt,
            messages=messages
        )

        return response.content[0].text


class OpenAIProvider(BaseAIProvider):
    """OpenAI GPT provider"""

    def _initialize_client(self):
        return openai.OpenAI(api_key=self.api_key)

    def send_message(self, model: str, messages: List[Dict], system_prompt: str, params: Dict) -> str:
        if not self.client:
            raise ValueError("OpenAI API key not configured")

        formatted_messages = self.format_messages(messages, system_prompt)
        extracted_params = self.extract_params(params)

        response = self.client.chat.completions.create(
            model=model,
            messages=formatted_messages,
            **extracted_params
        )

        return response.choices[0].message.content


class GoogleProvider(BaseAIProvider):
    """Google Gemini provider"""

    def _initialize_client(self):
        # New SDK uses client instantiation with API key
        # The environment variable GEMINI_API_KEY is automatically picked up
        if self.api_key:
            import os
            os.environ['GEMINI_API_KEY'] = self.api_key
        return genai.Client()

    def send_message(self, model: str, messages: List[Dict], system_prompt: str, params: Dict) -> str:
        if not self.client:
            raise ValueError("Google API key not configured")

        extracted_params = self.extract_params(params)

        # Format conversation with system prompt
        contents = []

        # Add system prompt as context
        if system_prompt:
            contents.append(f"{system_prompt}\n")

        # Add conversation history
        for msg in messages:
            role = "User" if msg["role"] == "user" else "Assistant"
            contents.append(f"{role}: {msg['content']}")

        full_prompt = "\n".join(contents)

        # Use the new client API
        response = self.client.models.generate_content(
            model=model,
            contents=full_prompt,
            config={
                "temperature": extracted_params["temperature"],
                "max_output_tokens": extracted_params["max_tokens"],
            }
        )

        return response.text


class AIProviderRegistry:
    """Registry for AI providers with automatic initialization"""

    def __init__(self):
        self.providers = {
            "anthropic": AnthropicProvider(os.getenv("ANTHROPIC_API_KEY")),
            "openai": OpenAIProvider(os.getenv("OPENAI_API_KEY")),
            "google": GoogleProvider(os.getenv("GEMINI_API_KEY"))
        }

    def send_message(self, provider: str, model_id: str, messages: List[Dict],
                     system_prompt: str, parameters: Dict) -> str:
        """Route message to appropriate provider"""
        provider_name = provider.lower()

        if provider_name not in self.providers:
            raise ValueError(f"Unsupported provider: {provider}")

        provider_instance = self.providers[provider_name]
        return provider_instance.send_message(model_id, messages, system_prompt, parameters)

    def get_available_providers(self) -> List[str]:
        """Get list of configured providers"""
        return [name for name, provider in self.providers.items() if provider.api_key]

    def get_default_provider(self) -> tuple:
        """Get the default provider based on available API keys"""
        if os.getenv("ANTHROPIC_API_KEY"):
            return "anthropic", "claude-3-haiku-20240307"
        elif os.getenv("OPENAI_API_KEY"):
            return "openai", "chatgpt-4o-latest"
        elif os.getenv("GEMINI_API_KEY"):
            return "google", "gemini-2.5-flash-lite"
        else:
            raise ValueError("No AI provider API keys found")


def run_chatbot():
    """Main chatbot loop"""
    print("=" * 70)
    print("  Simple AI Chatbot")
    print("=" * 70)
    print("\nSupporting: Anthropic Claude, OpenAI GPT, Google Gemini")
    print("Type 'exit' or 'quit' to end the conversation\n")

    # Initialize AI provider registry
    try:
        ai_registry = AIProviderRegistry()
        available = ai_registry.get_available_providers()

        if not available:
            logger.error("No AI provider API keys found. Please configure at least one provider.")
            return

        # Get default provider
        provider, model_id = ai_registry.get_default_provider()
        logger.info(f"✓ Using {provider} with model {model_id}")
        logger.info(f"Available providers: {', '.join(available)}")

    except Exception as e:
        logger.error(f"Failed to initialize AI providers: {e}")
        return

    # Default system prompt
    system_prompt = "You are a helpful AI assistant. Provide clear, concise, and friendly responses."

    # Default parameters
    parameters = {
        "temperature": 0.7,
        "max_tokens": 500
    }

    conversation_history = []

    # Main chat loop
    while True:
        try:
            user_input = input("You: ").strip()

            if user_input.lower() in ['exit', 'quit', 'q']:
                print("\nGoodbye! Thanks for chatting.")
                break

            if not user_input:
                continue

            # Add user message to history
            conversation_history.append({"role": "user", "content": user_input})

            # Send to AI provider
            print("\nAssistant: ", end="", flush=True)

            response = ai_registry.send_message(
                provider=provider,
                model_id=model_id,
                messages=conversation_history,
                system_prompt=system_prompt,
                parameters=parameters
            )

            print(response)

            # Add assistant response to history
            conversation_history.append({"role": "assistant", "content": response})

        except KeyboardInterrupt:
            print("\n\nInterrupted. Goodbye!")
            break
        except Exception as e:
            logger.error(f"Error in chat loop: {e}")
            print(f"\nError: {e}")

            # Provide helpful guidance for common errors
            if "API key not valid" in str(e) and "googleapis.com" in str(e):
                print("\n💡 Tip: For Google Gemini, you need an API key from Google AI Studio:")
                print("   1. Go to https://aistudio.google.com/app/apikey")
                print("   2. Click 'Get API Key' and create a new key")
                print("   3. Add it to your .env file as GEMINI_API_KEY=your-key-here")
            elif "API key" in str(e).lower():
                print("\n💡 Tip: Check that your API key is correct and has the necessary permissions.")


if __name__ == "__main__":
    # Check for at least one AI provider key
    provider_keys = ["ANTHROPIC_API_KEY", "OPENAI_API_KEY", "GEMINI_API_KEY"]
    if not any(os.getenv(key) for key in provider_keys):
        logger.error("No AI provider API keys found. Please add at least one:")
        for key in provider_keys:
            logger.error(f"  - {key}")
        exit(1)

    # Run the chatbot
    run_chatbot()

Step 1.5: Run your basic chatbot

Run the chatbot:

python simple_chatbot.py

You should see output like this:

======================================================================
  Simple AI Chatbot
======================================================================

Supporting: Anthropic Claude, OpenAI GPT, Google Gemini
Type 'exit' or 'quit' to end the conversation

2026-01-14 11:48:03,603 - INFO - ✓ Using anthropic with model claude-3-haiku-20240307
2026-01-14 11:48:03,603 - INFO - Available providers: anthropic, openai

You: Hello! What can you do?

Try asking questions and chatting with the AI. The chatbot automatically uses whichever AI provider you configured. You now have a working chatbot with multi-provider support.

Part 2: Creating your first AgentControl config with two variations

Now, create an AgentControl config in LaunchDarkly with two variations to demonstrate how to dynamically control AI behavior. For detailed guidance on creating AgentControl configs, read the AgentControl Quickstart.

Step 2.1: Create an AgentControl config in LaunchDarkly

To create an AgentControl config:

Log in to LaunchDarkly.
Navigate to app.launchdarkly.com/settings/projects.
Click Create project.
Name it simple-chatbot.

Click Create project.
Click Project settings, then Environments.
Click the three-dot overflow menu next to “Production.”
Copy the SDK key.

Copying the SDK key from project settings

Update your .env file with this key:
LD_SDK_KEY=your-copied-sdk-key

Then, create a new AgentControl config:

In the left sidebar, open the AI section and click AgentControl configs.
Click Create AgentControl config.
Name it simple-config.
Configure the default variation:
- Variation Name: friendly
- Model Provider: Anthropic, or your preferred provider
- Model: claude-3-haiku-20240307
- System Prompt:
  You are a friendly and casual AI assistant. Use a warm, conversational tone. Keep responses concise (2-3 sentences) and approachable. Feel free to use occasional emojis to add personality.
- Parameters:
  - temperature: 0.8 (more creative)
  - max_tokens: 500

Setting up the friendly variation in LaunchDarkly

Save the AgentControl config.

Step 2.2: Add your AgentControl config key to your .env file

To copy and add your AgentControl config key:

At the top of your AgentControl config page, copy the Config Key.

Update your .env file with this key:

LAUNCHDARKLY_AGENT_CONFIG_KEY=simple-config

Step 2.3: Edit targeting

To edit the AgentControl config’s targeting:

Click Targeting at the top.
Click Edit.
Select friendly from the dropdown menu.
Click Review and save.
Enter update in the Comment field and Production in the Confirm field.
Click Save changes.

Changing the default variation to friendly

Now create a new file called simple_chatbot_with_targeting.py that adds persona selection capabilities and LaunchDarkly integration.

Click to expand the complete simple_chatbot_with_targeting.py code

"""
Simple AI Chatbot with LaunchDarkly Targeting
Dynamic configuration and feature flagging
Supports user context-based provider and model selection
"""

import os
import logging
from typing import Dict, List, Any, Tuple, Optional
from abc import ABC, abstractmethod
import dotenv

# LaunchDarkly imports
import ldclient
from ldclient import Context
from ldclient.config import Config
from ldai import LDAIClient, AICompletionConfig, AICompletionConfigDefault, ModelConfig, LDMessage, ProviderConfig

# AI Provider imports
import anthropic
import openai
import google.genai as genai

# Set up logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)

# Suppress HTTP request logs from libraries
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("httpcore").setLevel(logging.WARNING)
logging.getLogger("openai").setLevel(logging.WARNING)
logging.getLogger("anthropic").setLevel(logging.WARNING)

# Load environment variables
dotenv.load_dotenv()


class LaunchDarklyAIClient:
    """Manages LaunchDarkly AgentControl configuration"""

    def __init__(self, sdk_key: str, agent_config_key: str):
        """Initialize LaunchDarkly client"""
        self.sdk_key = sdk_key
        self.agent_config_key = agent_config_key
        self.ld_client = None
        self.ai_client = None

        # Only initialize if we have a valid SDK key
        if sdk_key and sdk_key != "your-launchdarkly-sdk-key" and not sdk_key.startswith("your-"):
            try:
                ldclient.set_config(Config(sdk_key))
                self.ld_client = ldclient.get()
                self.ai_client = LDAIClient(self.ld_client)
                # Check if client initialized successfully
                if not self.ld_client.is_initialized():
                    logger.info("LaunchDarkly client not initialized, will use fallback configuration")
                    self.ld_client = None
                    self.ai_client = None
            except Exception as e:
                logger.info(f"LaunchDarkly initialization skipped: {e}")
                self.ld_client = None
                self.ai_client = None
        else:
            logger.info("No valid LaunchDarkly SDK key provided, using fallback configuration")

    def get_ai_config(self, user_context: Context, variables: Dict[str, Any] = None) -> AICompletionConfig:
        """Get AgentControl configuration for a specific user context"""
        fallback_config = self._get_fallback_config()

        if not self.ai_client:
            return fallback_config

        config = self.ai_client.completion_config(
            self.agent_config_key,
            user_context,
            fallback_config,
            variables or {}
        )
        return config

    def _get_fallback_config(self) -> AICompletionConfigDefault:
        """Fallback configuration when LaunchDarkly is unavailable"""
        # Detect which provider is available
        provider_name = "anthropic"  # default
        model_name = "claude-3-haiku-20240307"

        if os.getenv("ANTHROPIC_API_KEY"):
            provider_name = "anthropic"
            model_name = "claude-3.5-haiku-20241022"
        elif os.getenv("OPENAI_API_KEY"):
            provider_name = "openai"
            model_name = "chatgpt-4o-latest"
        elif os.getenv("GEMINI_API_KEY"):
            provider_name = "google"
            model_name = "gemini-2.5-flash-lite"
        else:
            logger.warning("No AI provider API keys found for fallback configuration")

        return AICompletionConfigDefault(
            enabled=True,
            model=ModelConfig(
                name=model_name,
                parameters={"temperature": 0.7, "max_tokens": 500}
            ),
            messages=[LDMessage(
                role="system",
                content="You are a helpful AI assistant. Provide clear, concise, and friendly responses."
            )],
            provider=ProviderConfig(name=provider_name)
        )


class BaseAIProvider(ABC):
    """Base class for AI providers"""

    def __init__(self, api_key: Optional[str] = None):
        self.api_key = api_key
        self.client = self._initialize_client() if api_key else None

    @abstractmethod
    def _initialize_client(self):
        """Initialize the provider's client"""
        pass

    @abstractmethod
    def send_message(self, model: str, messages: List[Dict], system_prompt: str, params: Dict) -> str:
        """Send message to the AI provider"""
        pass

    def format_messages(self, messages: List[Dict], system_prompt: str) -> List[Dict]:
        """Default message formatting (can be overridden by providers)"""
        formatted = [{"role": "system", "content": system_prompt}] if system_prompt else []
        formatted.extend([{"role": msg["role"], "content": msg["content"]} for msg in messages])
        return formatted

    def extract_params(self, params: Dict) -> Dict:
        """Extract common parameters"""
        return {
            "temperature": params.get("temperature", 0.7),
            "max_tokens": params.get("max_tokens", 500)
        }


class AnthropicProvider(BaseAIProvider):
    """Anthropic Claude provider"""

    def _initialize_client(self):
        return anthropic.Anthropic(api_key=self.api_key)

    def send_message(self, model: str, messages: List[Dict], system_prompt: str, params: Dict) -> str:
        if not self.client:
            raise ValueError("Anthropic API key not configured")

        extracted_params = self.extract_params(params)

        response = self.client.messages.create(
            model=model,
            max_tokens=extracted_params["max_tokens"],
            temperature=extracted_params["temperature"],
            system=system_prompt,
            messages=messages
        )

        return response.content[0].text


class OpenAIProvider(BaseAIProvider):
    """OpenAI GPT provider"""

    def _initialize_client(self):
        return openai.OpenAI(api_key=self.api_key)

    def send_message(self, model: str, messages: List[Dict], system_prompt: str, params: Dict) -> str:
        if not self.client:
            raise ValueError("OpenAI API key not configured")

        formatted_messages = self.format_messages(messages, system_prompt)
        extracted_params = self.extract_params(params)

        response = self.client.chat.completions.create(
            model=model,
            messages=formatted_messages,
            **extracted_params
        )

        return response.choices[0].message.content


class GoogleProvider(BaseAIProvider):
    """Google Gemini provider"""

    def _initialize_client(self):
        # New SDK uses client instantiation with API key
        # The environment variable GEMINI_API_KEY is automatically picked up
        if self.api_key:
            import os
            os.environ['GEMINI_API_KEY'] = self.api_key
        return genai.Client()

    def send_message(self, model: str, messages: List[Dict], system_prompt: str, params: Dict) -> str:
        if not self.client:
            raise ValueError("Google API key not configured")

        extracted_params = self.extract_params(params)

        # Format conversation with system prompt
        contents = []

        # Add system prompt as context
        if system_prompt:
            contents.append(f"{system_prompt}\n")

        # Add conversation history
        for msg in messages:
            role = "User" if msg["role"] == "user" else "Assistant"
            contents.append(f"{role}: {msg['content']}")

        full_prompt = "\n".join(contents)

        # Use the new client API
        response = self.client.models.generate_content(
            model=model,
            contents=full_prompt,
            config={
                "temperature": extracted_params["temperature"],
                "max_output_tokens": extracted_params["max_tokens"],
            }
        )

        return response.text


class AIProviderRegistry:
    """Registry for AI providers with automatic initialization"""

    def __init__(self):
        self.providers = {
            "anthropic": AnthropicProvider(os.getenv("ANTHROPIC_API_KEY")),
            "openai": OpenAIProvider(os.getenv("OPENAI_API_KEY")),
            "google": GoogleProvider(os.getenv("GEMINI_API_KEY"))
        }

    def send_message(self, provider: str, model_id: str, messages: List[Dict],
                     system_prompt: str, parameters: Dict) -> str:
        """Route message to appropriate provider"""
        provider_name = provider.lower()

        if provider_name not in self.providers:
            raise ValueError(f"Unsupported provider: {provider}")

        provider_instance = self.providers[provider_name]
        return provider_instance.send_message(model_id, messages, system_prompt, parameters)

    def get_available_providers(self) -> List[str]:
        """Get list of configured providers"""
        return [name for name, provider in self.providers.items() if provider.api_key]


def create_user_context(user_id: str, attributes: Dict[str, Any] = None) -> Context:
    """Create a LaunchDarkly context for a user"""
    builder = Context.builder(user_id)
    if attributes:
        for key, value in attributes.items():
            builder.set(key, value)
    return builder.build()


def run_chatbot():
    """Main chatbot loop"""
    print("=" * 70)
    print("  Simple AI Chatbot with LaunchDarkly Targeting")
    print("=" * 70)
    print("\nSupporting: Anthropic Claude, OpenAI GPT, Google Gemini")
    print("Type 'exit' or 'quit' to end the conversation\n")

    # Initialize clients
    try:
        ld_ai_client = LaunchDarklyAIClient(
            sdk_key=os.getenv("LD_SDK_KEY", ""),
            agent_config_key=os.getenv("LAUNCHDARKLY_AGENT_CONFIG_KEY", "default-config")
        )
        ai_registry = AIProviderRegistry()

        available = ai_registry.get_available_providers()
        logger.info(f"✓ Clients initialized. Available providers: {', '.join(available)}")
    except Exception as e:
        logger.error(f"Failed to initialize clients: {e}")
        return

    # Create user context
    user_context = create_user_context(
        user_id="demo-user-001",
        attributes={"name": "Demo User", "environment": "development"}
    )

    # Get initial config to show provider/model
    config = ld_ai_client.get_ai_config(user_context)
    logger.info(f"✓ Using {config.provider.name} with model {config.model.name}")

    conversation_history = []

    # Main chat loop
    while True:
        try:
            user_input = input("You: ").strip()

            if user_input.lower() in ['exit', 'quit', 'q']:
                print("\nGoodbye! Thanks for chatting.")
                break

            if not user_input:
                continue

            # Fetch fresh config from LaunchDarkly for each message
            config = ld_ai_client.get_ai_config(user_context)

            # Extract configuration
            provider = config.provider.name
            model_id = config.model.name
            system_prompt = config.messages[0].content if config.messages else "You are a helpful assistant."


            # Get model parameters
            model_params = config.model.parameters if hasattr(config.model, 'parameters') and config.model.parameters else {}
            parameters = {
                "temperature": model_params.get("temperature", 0.7),
                "max_tokens": model_params.get("max_tokens", 500)
            }

            # Add user message to history
            conversation_history.append({"role": "user", "content": user_input})

            # Send to AI provider
            print("\nAssistant: ", end="", flush=True)

            response = ai_registry.send_message(
                provider=provider,
                model_id=model_id,
                messages=conversation_history,
                system_prompt=system_prompt,
                parameters=parameters
            )

            print(response)

            # Add assistant response to history
            conversation_history.append({"role": "assistant", "content": response})

        except KeyboardInterrupt:
            print("\n\nInterrupted. Goodbye!")
            break
        except Exception as e:
            logger.error(f"Error in chat loop: {e}")
            print(f"\nError: {e}")

            # Provide helpful guidance for common errors
            if "API key not valid" in str(e) and "googleapis.com" in str(e):
                print("\n💡 Tip: For Google Gemini, you need an API key from Google AI Studio:")
                print("   1. Go to https://aistudio.google.com/app/apikey")
                print("   2. Click 'Get API Key' and create a new key")
                print("   3. Add it to your .env file as GEMINI_API_KEY=your-key-here")
            elif "API key" in str(e).lower():
                print("\n💡 Tip: Check that your API key is correct and has the necessary permissions.")

def run_chatbot_with_persona(persona: str = "business"):
    """
    Run chatbot with a specific persona context

    Args:
        persona: The persona to use (business, creative, or default)
    """
    print("=" * 70)
    print(f"  AI Chatbot - Persona: {persona.upper()}")
    print("=" * 70)
    print("\nType 'exit' to quit, 'switch' to change persona\n")

    # Initialize clients
    try:
        ld_ai_client = LaunchDarklyAIClient(
            sdk_key=os.getenv("LD_SDK_KEY", ""),
            agent_config_key=os.getenv("LAUNCHDARKLY_AGENT_CONFIG_KEY", "default-config")
        )
        ai_registry = AIProviderRegistry()

        available = ai_registry.get_available_providers()
        logger.info(f"✓ Clients initialized. Available providers: {', '.join(available)}")
    except Exception as e:
        logger.error(f"Failed to initialize clients: {e}")
        return False

    # Create user context with persona attribute
    user_context = create_user_context(
        user_id=f"{persona}-user-001",
        attributes={
            "persona": persona,
            "name": f"{persona.title()} User"
        }
    )

    # Conversation loop
    conversation_history = []

    while True:
        try:
            user_input = input("You: ").strip()

            if user_input.lower() in ['exit', 'quit']:
                print("\n👋 Goodbye!\n")
                break

            if user_input.lower() == 'switch':
                print("\n🔄 Switching persona...\n")
                return True

            if not user_input:
                continue

            # Fetch fresh config from LaunchDarkly for each message
            config = ld_ai_client.get_ai_config(user_context)

            # Extract configuration
            provider = config.provider.name
            model_id = config.model.name
            system_prompt = config.messages[0].content if config.messages else "You are a helpful assistant."


            # Get model parameters
            model_params = config.model.parameters if hasattr(config.model, 'parameters') and config.model.parameters else {}
            parameters = {
                "temperature": model_params.get("temperature", 0.7),
                "max_tokens": model_params.get("max_tokens", 500)
            }

            # Add user message to history
            conversation_history.append({"role": "user", "content": user_input})

            # Send to AI provider
            print("\nAssistant: ", end="", flush=True)

            response = ai_registry.send_message(
                provider=provider,
                model_id=model_id,
                messages=conversation_history,
                system_prompt=system_prompt,
                parameters=parameters
            )

            print(response + "\n")

            # Add assistant response to history
            conversation_history.append({"role": "assistant", "content": response})

        except KeyboardInterrupt:
            print("\n\n👋 Goodbye!\n")
            break
        except Exception as e:
            logger.error(f"Error in chat loop: {e}")
            print(f"\n❌ Error: {e}\n")

    return False


def main_with_personas():
    """Main entry point with persona selection"""
    print("\n" + "=" * 70)
    print("  LaunchDarkly AgentControl config - Persona Demo")
    print("=" * 70)

    personas = {
        "1": "business",
        "2": "creative",
        "3": None  # Default
    }

    while True:
        print("\nSelect a persona:")
        print("  1. Business (professional and concise)")
        print("  2. Creative (imaginative and engaging)")
        print("  3. Default (friendly and helpful)")
        print("  q. Quit")

        choice = input("\nYour choice (1-3, q): ").strip()

        if choice.lower() == 'q':
            print("\n👋 Goodbye!\n")
            break

        if choice not in personas:
            print("❌ Invalid choice. Please select 1-3 or q.")
            continue

        persona = personas[choice]

        # Run with selected persona or default
        if persona:
            should_switch = run_chatbot_with_persona(persona)
            if not should_switch:
                break
            # If should_switch is True, loop continues to persona selection
        else:
            run_chatbot()  # Run default chatbot (no persona context)
            break  # Default mode exits after session ends

if __name__ == "__main__":
    import sys

    # Check for LaunchDarkly configuration (optional - will use fallback if not provided)
    sdk_key = os.getenv("LD_SDK_KEY")
    config_key = os.getenv("LAUNCHDARKLY_AGENT_CONFIG_KEY", "default-config")

    if not sdk_key or sdk_key == "your-launchdarkly-sdk-key":
        logger.info("No LaunchDarkly SDK key found - using fallback configuration")
        logger.info("To use LaunchDarkly features, add LD_SDK_KEY to your .env file")

    # Check for at least one AI provider key
    provider_keys = ["ANTHROPIC_API_KEY", "OPENAI_API_KEY", "GEMINI_API_KEY"]
    if not any(os.getenv(key) for key in provider_keys):
        logger.error("No AI provider API keys found. Please add at least one:")
        for key in provider_keys:
            logger.error(f"  - {key}")
        exit(1)

    # Check if persona mode is requested
    if len(sys.argv) > 1 and sys.argv[1] == "--personas":
        main_with_personas()
    else:
        # Default behavior - run original chatbot
        run_chatbot()

Step 2.4: Test the friendly variation

Run the chatbot:

python simple_chatbot_with_targeting.py

Try asking: “What’s the best way to learn a new programming language?” The response is warm and casual, possibly with an emoji.

✓ Configuration: LaunchDarkly AgentControl config
✓ Using: ANTHROPIC - claude-3-haiku-20240307
✓ Parameters: Temperature=0.8, Max Tokens=500

You: What's the best way to learn a new programming language? 

Assistant: Great question! Here are a few tips for learning a new programming language effectively:

🤓 Start with the basics - focus on learning the syntax, data types, and fundamental concepts first. Build a solid foundation before moving on to more advanced topics.

👩‍💻 Practice, practice, practice! The more you code, the more comfortable you'll become. Work through tutorials, build small projects, and challenge yourself.

💬 Engage with the community - join online forums, attend meetups, or find a programming buddy to learn with. Discussing concepts and getting feedback can really accelerate your progress.

Hope this helps! Let me know if you have any other questions. Happy coding! 💻

Step 2.5: Real-time configuration changes with no redeploy

LaunchDarkly AgentControl let you change AI behavior instantly without redeploying your application. This example changes the response language. Keep your chatbot running from Step 2.4 and follow these steps to update the system prompt to respond in Portuguese:

In LaunchDarkly, navigate to your AgentControl config.
Go to your simple-config.
Click the Variations tab.
Select the friendly variation.

Change the system prompt to:

You are a friendly AI assistant who ALWAYS responds in Portuguese (Brazilian),
regardless of the input language. Use a warm, conversational tone.
Keep responses concise (2-3 sentences). Even if the user writes in English,
always respond in Portuguese.

Click Save changes.

Updating instructions to respond in Portuguese

Now, test the change without restarting the chatbot. In your still-running chatbot, type a new message:

You: Hello, how are you today?

Assistant: Olá! Estou muito bem, obrigado por perguntar! 😊
Como posso ajudá-lo hoje?

Notice how the chatbot’s behavior changed instantly without:

Restarting the application
Redeploying code
Changing any configuration files
Any downtime

This demonstrates how AgentControl configs support real-time experimentation and iteration. You can update prompts, adjust behaviors, or switch languages based on user feedback or business needs.

Part 3: Advanced configuration and persona-based targeting

Now, you can explore advanced targeting capabilities by creating persona-based variations. This demonstrates how to deliver different AI experiences to different user segments. To learn more about targeting capabilities, read Config targeting.

Step 3.1: Create persona-based variations

Create three persona variations in LaunchDarkly:

Navigate to your simple-config AgentControl config.
Click the Variations tab.

Click Add Variation and add a business persona:

Variation Name: business
Model: claude-3-haiku-20240307

System Prompt:

You are a professional business consultant. Provide concise, data-driven insights.
Focus on ROI, efficiency, and strategic value. Use bullet points for clarity.
Avoid casual language and emojis.

Temperature: 0.4

Repeat the process for a creative persona variation:

Variation Name: creative
Model: chatgpt-4o-latest (OpenAI)

System Prompt:

You are a creative AI companion. Be imaginative, playful, and engaging.
Use storytelling, metaphors, and creative language to inspire.
Think outside the box and encourage creative exploration.

Temperature: 0.9

Creating persona-based variations in LaunchDarkly

Step 3.2: Configure persona-based targeting

Now, configure persona-based targeting:

Navigate to the Targeting tab for your simple-config.
Click Edit.
Click + Add Rule.
Select Build a custom rule.
Add a Rule Name of Persona-based targeting.
Configure the rule:
- Context Kind: User
- Attribute: persona
- Operator: is one of
- Values: business
- Serve: business variation
Repeat for the creative persona:
- If persona is one of creative, serve creative.
Set the Default rule to serve friendly.
Click Review and save.
Add a comment and confirm.

Setting up persona-based targeting rules

Step 3.3: Test different personas

Run the chatbot with persona support:

python simple_chatbot_with_targeting.py --personas

Test the business persona:

Ask: “How can AI improve our sales process?”
Expected: Professional, ROI-focused response with bullet points

📋 Active Variation: BUSINESS
🤖 Provider: ANTHROPIC | Model: claude-3-haiku-20240307

You: How can AI improve our sales process?

Assistant: AI can significantly optimize your sales process through several high-impact applications:

• **Lead Scoring & Prioritization**: ML models analyze historical data to identify high-conversion
    prospects, improving sales efficiency by 30-50%

• **Predictive Analytics**: Forecast sales pipeline with 85%+ accuracy, enabling better resource
    allocation and revenue planning

• **Automated Personalization**: Dynamic content generation for emails and proposals, increasing
    engagement rates by 40%

• **Conversation Intelligence**: Real-time coaching during calls and automated CRM data entry,
    saving 2-3 hours per rep daily

• **Churn Prevention**: Identify at-risk accounts 60-90 days before potential churn, enabling
    proactive retention strategies

ROI typically ranges from 3-5x within first year, with average sales productivity gains of 20-35%.

Type switch and select option 2 for the creative persona:

Ask: “Tell me about the future”
Expected: Imaginative, engaging response, especially if using OpenAI

📋 Active Variation: CREATIVE (imaginative responses)
🤖 Provider: OPENAI | Model: chatgpt-4o-latest
🌡️  Temperature: 0.9

You: Tell me about the future

Assistant: Ah, the future—a shimmering veil of possibility fluttering just beyond the now. Let me take you on a journey, not with cold facts or dry predictions, but with a story woven from stars and silicon, from dreams and data.

---

The year is 2142.

Beneath twin moons over a restless ocean, a ten-year-old girl named Nara stands barefoot on the shore of Viridia, a floating archipelago that drifts with the sea currents. Her sky-silk dress glows faintly in the dusk; a gift from her Dreamweaver AI, Brii, who knits clothing from light and emotion.

She watches as a silver pod rises from the horizon, its surface rippling like water. It’s a courier from the Mars-Titan Cultural Alliance, bringing news, music, and fruit seeds engineered in low gravity. Nara isn’t surprised. Interplanetary mail has become as common as breezes.

Above, the sky is alive.

Not with stars alone, but with twinkling habitats—orbital cities where trees grow upside down and whales swim through engineered clouds. The Great Ring of Gaia, Earth's outer atmosphere dwelling, now hosts ten million humans and creatures reimagined by conscious evolution. People there live symbiotically with sentient environments that sing lullabies at night.

Back on Earth, the scars of past centuries—climate change, war, and division—have not been erased, but healed into wisdom. Cities have become vertical forests. Old oceans have been rewilded. And the global economy—once fueled by scarcity—is now driven by creativity, empathy, and play.

Oh, and languages? They’ve blossomed. People speak in colors, in gestures, even in smells. Communication is no longer a limitation but an art form—a dance between minds and media.

Yet, even with so much progress, the essence of the future remains the same as always: a question, a beckoning. For every astounding breakthrough, every breathless moment of harmony, there are still mysteries wrapped in dark matter and dreams—questions about consciousness, love, the soul.

And in this future, just like now, there will always be someone—perhaps someone like you—looking up at the stars, wondering what comes next.

---

Would you like to explore a particular thread of this future? Space travel, AI, climate rebirth, or perhaps time itself? The future’s door is open; step through with me.

In this section, you learned:

How to add persona-based contexts to your existing code
How to target AI variations based on simple user attributes
How LaunchDarkly enables dynamic behavior without code changes

Part 4: Monitoring and verifying data

LaunchDarkly provides monitoring for AgentControl configs. Next, ensure your data flows correctly. To learn more about monitoring capabilities, read Monitor AgentControl configs.

Step 4.1: Understanding AI metrics

LaunchDarkly AI SDKs provide comprehensive metrics tracking to help you monitor and optimize your AI model performance. The SDK includes both individual track* methods and provider-specific convenience methods for recording metrics. Available metrics include:

Duration: Time taken for AI model generation, including network latency
Token Usage: Input, output, and total tokens consumed (critical for cost management)
Generation Success: Successful completion of AI generation
Generation Error: Failed generations with error tracking
Time to First Token: Latency until the first response token (important for streaming)
Output Satisfaction: User feedback (positive/negative ratings)

Tracking Methods

The AI SDKs provide two approaches to recording metrics:

Generic extractor-based method: track_metrics_of(extractor, func) runs the wrapped chat completion call, then applies a provider-specific extractor (such as get_ai_metrics_from_response from ldai_openai) to record duration, token usage, and success or error in one call.
Individual track methods: Granular methods like track_duration(), track_tokens(), track_success(), track_error(), and track_feedback() for manual metric recording.

Create a tracker for each generation by calling config.create_tracker() on the result of completion_config(). The tracker is specific to that config variation, so always call completion_config() and create a fresh tracker each time you generate content to ensure metrics are correctly associated with the right variation. For delayed feedback, such as user ratings that arrive after generation, persist the tracker’s resumption token and use it to recreate the tracker later:

token = tracker.resumption_token
# ... later, possibly in a different process:
result = ai_client.create_tracker(token, context)
if result.is_success():
    new_tracker = result.value
    new_tracker.track_feedback({"kind": FeedbackKind.Positive})

create_tracker returns an ldclient.Result. Check result.is_success() before reading result.value. To learn more about tracking AI metrics, read Tracking AI metrics.

Step 4.2: Add comprehensive tracking

Create a file called simple_chatbot_with_targeting_and_tracking.py:

Click to expand the complete simple_chatbot_with_targeting_and_tracking.py code

"""
Simple AI Chatbot with LaunchDarkly AgentControl config
Complete LaunchDarkly integration with targeting and metrics
Tracks token usage, response times, and success rates
"""

import os
import logging
import time
from typing import Dict, List, Any, Tuple, Optional
from abc import ABC, abstractmethod
import dotenv

# LaunchDarkly imports
import ldclient
from ldclient import Context
from ldclient.config import Config
from ldai import LDAIClient, AICompletionConfig, AICompletionConfigDefault, ModelConfig, LDMessage, ProviderConfig
from ldai_openai import get_ai_metrics_from_response

# AI Provider imports
import anthropic
import openai
import google.genai as genai

# Set up logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)

# Suppress HTTP request logs from libraries
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("httpcore").setLevel(logging.WARNING)
logging.getLogger("openai").setLevel(logging.WARNING)
logging.getLogger("anthropic").setLevel(logging.WARNING)

# Load environment variables
dotenv.load_dotenv()


class LaunchDarklyAIClient:
    """Manages LaunchDarkly AgentControl configuration"""

    def __init__(self, sdk_key: str, agent_config_key: str):
        """Initialize LaunchDarkly client"""
        self.sdk_key = sdk_key
        self.agent_config_key = agent_config_key
        self.ld_client = None
        self.ai_client = None

        # Only initialize if we have a valid SDK key
        if sdk_key and sdk_key != "your-launchdarkly-sdk-key" and not sdk_key.startswith("your-"):
            try:
                ldclient.set_config(Config(sdk_key))
                self.ld_client = ldclient.get()
                self.ai_client = LDAIClient(self.ld_client)
                # Check if client initialized successfully
                if not self.ld_client.is_initialized():
                    logger.info("LaunchDarkly client not initialized, will use fallback configuration")
                    self.ld_client = None
                    self.ai_client = None
            except Exception as e:
                logger.info(f"LaunchDarkly initialization skipped: {e}")
                self.ld_client = None
                self.ai_client = None
        else:
            logger.info("No valid LaunchDarkly SDK key provided, using fallback configuration")

    def get_ai_config(self, user_context: Context, variables: Dict[str, Any] = None) -> AICompletionConfig:
        """Get AgentControl configuration for a specific user context"""
        fallback_config = self._get_fallback_config()

        if not self.ai_client:
            return fallback_config

        config = self.ai_client.completion_config(
            self.agent_config_key,
            user_context,
            fallback_config,
            variables or {}
        )
        return config

    def _get_fallback_config(self) -> AICompletionConfigDefault:
        """Fallback configuration when LaunchDarkly is unavailable"""
        # Detect which provider is available
        provider_name = "anthropic"  # default
        model_name = "claude-3-haiku-20240307"

        if os.getenv("ANTHROPIC_API_KEY"):
            provider_name = "anthropic"
            model_name = "claude-3.5-haiku-20241022"
        elif os.getenv("OPENAI_API_KEY"):
            provider_name = "openai"
            model_name = "chatgpt-4o-latest"
        elif os.getenv("GEMINI_API_KEY"):
            provider_name = "google"
            model_name = "gemini-2.5-flash-lite"
        else:
            logger.warning("No AI provider API keys found for fallback configuration")

        return AICompletionConfigDefault(
            enabled=True,
            model=ModelConfig(
                name=model_name,
                parameters={"temperature": 0.7, "max_tokens": 500}
            ),
            messages=[LDMessage(
                role="system",
                content="You are a helpful AI assistant. Provide clear, concise, and friendly responses."
            )],
            provider=ProviderConfig(name=provider_name)
        )


class BaseAIProvider(ABC):
    """Base class for AI providers"""

    def __init__(self, api_key: Optional[str] = None):
        self.api_key = api_key
        self.client = self._initialize_client() if api_key else None

    @abstractmethod
    def _initialize_client(self):
        """Initialize the provider's client"""
        pass

    @abstractmethod
    def send_message(self, model: str, messages: List[Dict], system_prompt: str, params: Dict, tracker=None) -> str:
        """Send message to the AI provider"""
        pass

    def format_messages(self, messages: List[Dict], system_prompt: str) -> List[Dict]:
        """Default message formatting (can be overridden by providers)"""
        formatted = [{"role": "system", "content": system_prompt}] if system_prompt else []
        formatted.extend([{"role": msg["role"], "content": msg["content"]} for msg in messages])
        return formatted

    def extract_params(self, params: Dict) -> Dict:
        """Extract common parameters"""
        return {
            "temperature": params.get("temperature", 0.7),
            "max_tokens": params.get("max_tokens", 500)
        }


class AnthropicProvider(BaseAIProvider):
    """Anthropic Claude provider"""

    def _initialize_client(self):
        return anthropic.Anthropic(api_key=self.api_key)

    def send_message(self, model: str, messages: List[Dict], system_prompt: str, params: Dict, tracker=None) -> str:
        if not self.client:
            raise ValueError("Anthropic API key not configured")

        extracted_params = self.extract_params(params)

        if tracker:
            # Track API call duration and response metrics
            response = tracker.track_duration_of(
                lambda: self.client.messages.create(
                    model=model,
                    max_tokens=extracted_params["max_tokens"],
                    temperature=extracted_params["temperature"],
                    system=system_prompt,
                    messages=messages
                )
            )

            # Track token usage if available
            if hasattr(response, 'usage'):
                from ldai.tracker import TokenUsage
                token_usage = TokenUsage(
                    input=response.usage.input_tokens,
                    output=response.usage.output_tokens,
                    total=response.usage.input_tokens + response.usage.output_tokens
                )
                tracker.track_tokens(token_usage)

            tracker.track_success()
        else:
            response = self.client.messages.create(
                model=model,
                max_tokens=extracted_params["max_tokens"],
                temperature=extracted_params["temperature"],
                system=system_prompt,
                messages=messages
            )

        return response.content[0].text


class OpenAIProvider(BaseAIProvider):
    """OpenAI GPT provider"""

    def _initialize_client(self):
        return openai.OpenAI(api_key=self.api_key)

    def send_message(self, model: str, messages: List[Dict], system_prompt: str, params: Dict, tracker=None) -> str:
        if not self.client:
            raise ValueError("OpenAI API key not configured")

        formatted_messages = self.format_messages(messages, system_prompt)
        extracted_params = self.extract_params(params)

        if tracker:
            response = tracker.track_metrics_of(
                get_ai_metrics_from_response,
                lambda: self.client.chat.completions.create(
                    model=model,
                    messages=formatted_messages,
                    **extracted_params
                )
            )
        else:
            response = self.client.chat.completions.create(
                model=model,
                messages=formatted_messages,
                **extracted_params
            )

        return response.choices[0].message.content


class GoogleProvider(BaseAIProvider):
    """Google Gemini provider"""

    def _initialize_client(self):
        # New SDK uses client instantiation with API key
        # The environment variable GEMINI_API_KEY is automatically picked up
        if self.api_key:
            import os
            os.environ['GEMINI_API_KEY'] = self.api_key
        return genai.Client()

    def send_message(self, model: str, messages: List[Dict], system_prompt: str, params: Dict, tracker=None) -> str:
        if not self.client:
            raise ValueError("Google API key not configured")

        extracted_params = self.extract_params(params)

        # Format conversation with system prompt
        contents = []

        # Add system prompt as context
        if system_prompt:
            contents.append(f"{system_prompt}\n")

        # Add conversation history
        for msg in messages:
            role = "User" if msg["role"] == "user" else "Assistant"
            contents.append(f"{role}: {msg['content']}")

        full_prompt = "\n".join(contents)

        # Manual metrics tracking for Google Gemini
        start_time = time.time()

        # Use the new client API
        response = self.client.models.generate_content(
            model=model,
            contents=full_prompt,
            config={
                "temperature": extracted_params["temperature"],
                "max_output_tokens": extracted_params["max_tokens"],
            }
        )

        duration = time.time() - start_time

        if tracker:
            # Track duration and success
            tracker.track_duration(duration)
            tracker.track_success()

        return response.text


class AIProviderRegistry:
    """Registry for AI providers with automatic initialization"""

    def __init__(self):
        self.providers = {
            "anthropic": AnthropicProvider(os.getenv("ANTHROPIC_API_KEY")),
            "openai": OpenAIProvider(os.getenv("OPENAI_API_KEY")),
            "google": GoogleProvider(os.getenv("GEMINI_API_KEY"))
        }

    def send_message(self, provider: str, model_id: str, messages: List[Dict],
                     system_prompt: str, parameters: Dict, tracker=None) -> str:
        """Route message to appropriate provider"""
        provider_name = provider.lower()

        if provider_name not in self.providers:
            raise ValueError(f"Unsupported provider: {provider}")

        provider_instance = self.providers[provider_name]
        return provider_instance.send_message(model_id, messages, system_prompt, parameters, tracker)

    def get_available_providers(self) -> List[str]:
        """Get list of configured providers"""
        return [name for name, provider in self.providers.items() if provider.api_key]


def create_user_context(user_id: str, attributes: Dict[str, Any] = None) -> Context:
    """Create a LaunchDarkly context for a user"""
    builder = Context.builder(user_id)
    if attributes:
        for key, value in attributes.items():
            builder.set(key, value)
    return builder.build()


def run_chatbot():
    """Main chatbot loop with full tracking"""
    print("=" * 70)
    print("  Simple AI Chatbot with LaunchDarkly AgentControl config")
    print("=" * 70)
    print("\nSupporting: Anthropic Claude, OpenAI GPT, Google Gemini")
    print("Type 'exit' or 'quit' to end the conversation\n")

    # Initialize clients
    try:
        ld_ai_client = LaunchDarklyAIClient(
            sdk_key=os.getenv("LD_SDK_KEY", ""),
            agent_config_key=os.getenv("LAUNCHDARKLY_AGENT_CONFIG_KEY", "default-config")
        )
        ai_registry = AIProviderRegistry()

        available = ai_registry.get_available_providers()
        logger.info(f"✓ Clients initialized. Available providers: {', '.join(available)}")
    except Exception as e:
        logger.error(f"Failed to initialize clients: {e}")
        return

    # Create user context
    user_context = create_user_context(
        user_id="demo-user-001",
        attributes={"name": "Demo User", "environment": "development"}
    )

    # Get initial config to show provider/model
    config = ld_ai_client.get_ai_config(user_context)
    logger.info(f"✓ Using {config.provider.name} with model {config.model.name}")

    conversation_history = []

    # Main chat loop
    while True:
        try:
            user_input = input("You: ").strip()

            if user_input.lower() in ['exit', 'quit', 'q']:
                print("\nGoodbye! Thanks for chatting.")
                break

            if not user_input:
                continue

            # Fetch fresh config from LaunchDarkly for each message
            config = ld_ai_client.get_ai_config(user_context)

            # Extract configuration
            provider = config.provider.name
            model_id = config.model.name
            system_prompt = config.messages[0].content if config.messages else "You are a helpful assistant."
            tracker = config.create_tracker() if config.enabled else None

            # Get model parameters
            model_params = config.model.parameters if hasattr(config.model, 'parameters') and config.model.parameters else {}
            parameters = {
                "temperature": model_params.get("temperature", 0.7),
                "max_tokens": model_params.get("max_tokens", 500)
            }

            # Add user message to history
            conversation_history.append({"role": "user", "content": user_input})

            # Send to AI provider
            print("\nAssistant: ", end="", flush=True)

            response = ai_registry.send_message(
                provider=provider,
                model_id=model_id,
                messages=conversation_history,
                system_prompt=system_prompt,
                parameters=parameters,
                tracker=tracker
            )

            print(response)

            # Add assistant response to history
            conversation_history.append({"role": "assistant", "content": response})

        except KeyboardInterrupt:
            print("\n\nInterrupted. Goodbye!")
            break
        except Exception as e:
            logger.error(f"Error in chat loop: {e}")
            print(f"\nError: {e}")

            # Provide helpful guidance for common errors
            if "API key not valid" in str(e) and "googleapis.com" in str(e):
                print("\n💡 Tip: For Google Gemini, you need an API key from Google AI Studio:")
                print("   1. Go to https://aistudio.google.com/app/apikey")
                print("   2. Click 'Get API Key' and create a new key")
                print("   3. Add it to your .env file as GEMINI_API_KEY=your-key-here")
            elif "API key" in str(e).lower():
                print("\n💡 Tip: Check that your API key is correct and has the necessary permissions.")

def run_chatbot_with_persona(persona: str = "business"):
    """
    Run chatbot with a specific persona context

    Args:
        persona: The persona to use (business, creative, or default)
    """
    print("=" * 70)
    print(f"  AI Chatbot - Persona: {persona.upper()}")
    print("=" * 70)
    print("\nType 'exit' to quit, 'switch' to change persona\n")

    # Initialize clients
    try:
        ld_ai_client = LaunchDarklyAIClient(
            sdk_key=os.getenv("LD_SDK_KEY", ""),
            agent_config_key=os.getenv("LAUNCHDARKLY_AGENT_CONFIG_KEY", "default-config")
        )
        ai_registry = AIProviderRegistry()

        available = ai_registry.get_available_providers()
        logger.info(f"✓ Clients initialized. Available providers: {', '.join(available)}")
    except Exception as e:
        logger.error(f"Failed to initialize clients: {e}")
        return False

    # Create user context with persona attribute
    user_context = create_user_context(
        user_id=f"{persona}-user-001",
        attributes={
            "persona": persona,
            "name": f"{persona.title()} User"
        }
    )

    # Conversation loop
    conversation_history = []

    while True:
        try:
            user_input = input("You: ").strip()

            if user_input.lower() in ['exit', 'quit']:
                print("\n👋 Goodbye!\n")
                break

            if user_input.lower() == 'switch':
                print("\n🔄 Switching persona...\n")
                return True

            if not user_input:
                continue

            # Fetch fresh config from LaunchDarkly for each message
            config = ld_ai_client.get_ai_config(user_context)

            # Extract configuration
            provider = config.provider.name
            model_id = config.model.name
            system_prompt = config.messages[0].content if config.messages else "You are a helpful assistant."
            tracker = config.create_tracker() if config.enabled else None

            # Get model parameters
            model_params = config.model.parameters if hasattr(config.model, 'parameters') and config.model.parameters else {}
            parameters = {
                "temperature": model_params.get("temperature", 0.7),
                "max_tokens": model_params.get("max_tokens", 500)
            }

            # Add user message to history
            conversation_history.append({"role": "user", "content": user_input})

            # Send to AI provider
            print("\nAssistant: ", end="", flush=True)

            response = ai_registry.send_message(
                provider=provider,
                model_id=model_id,
                messages=conversation_history,
                system_prompt=system_prompt,
                parameters=parameters,
                tracker=tracker
            )

            print(response + "\n")

            # Add assistant response to history
            conversation_history.append({"role": "assistant", "content": response})

        except KeyboardInterrupt:
            print("\n\n👋 Goodbye!\n")
            break
        except Exception as e:
            logger.error(f"Error in chat loop: {e}")
            print(f"\n❌ Error: {e}\n")

    return False


def main_with_personas():
    """Main entry point with persona selection"""
    print("\n" + "=" * 70)
    print("  LaunchDarkly AgentControl config - Persona Demo")
    print("=" * 70)

    personas = {
        "1": "business",
        "2": "creative",
        "3": None  # Default
    }

    while True:
        print("\nSelect a persona:")
        print("  1. Business (professional and concise)")
        print("  2. Creative (imaginative and engaging)")
        print("  3. Default (friendly and helpful)")
        print("  q. Quit")

        choice = input("\nYour choice (1-3, q): ").strip()

        if choice.lower() == 'q':
            print("\n👋 Goodbye!\n")
            break

        if choice not in personas:
            print("❌ Invalid choice. Please select 1-3 or q.")
            continue

        persona = personas[choice]

        # Run with selected persona or default
        if persona:
            should_switch = run_chatbot_with_persona(persona)
            if not should_switch:
                break
            # If should_switch is True, loop continues to persona selection
        else:
            run_chatbot()  # Run default chatbot (no persona context)
            break  # Default mode exits after session ends

if __name__ == "__main__":
    import sys

    # Check for LaunchDarkly configuration (optional - will use fallback if not provided)
    sdk_key = os.getenv("LD_SDK_KEY")
    config_key = os.getenv("LAUNCHDARKLY_AGENT_CONFIG_KEY", "default-config")

    if not sdk_key or sdk_key == "your-launchdarkly-sdk-key":
        logger.info("No LaunchDarkly SDK key found - using fallback configuration")
        logger.info("To use LaunchDarkly features, add LD_SDK_KEY to your .env file")

    # Check for at least one AI provider key
    provider_keys = ["ANTHROPIC_API_KEY", "OPENAI_API_KEY", "GEMINI_API_KEY"]
    if not any(os.getenv(key) for key in provider_keys):
        logger.error("No AI provider API keys found. Please add at least one:")
        for key in provider_keys:
            logger.error(f"  - {key}")
        exit(1)

    # Check if persona mode is requested
    if len(sys.argv) > 1 and sys.argv[1] == "--personas":
        main_with_personas()
    else:
        # Default behavior - run original chatbot
        run_chatbot()

Step 4.3: Testing complete monitoring flow

To test the monitoring flow:

python simple_chatbot_with_targeting_and_tracking.py

Step 4.4: Verify data in LaunchDarkly

After running the monitored chatbot:

Navigate to AgentControl configs.
Select simple-chatbot-config.
View the Monitoring tab.

The monitoring dashboard provides real-time insights into your AgentControl config’s performance:

AgentControl config monitoring dashboard overview

In the dashboard, you see several key sections:

Usage Overview: Displays the total number of requests served by your AgentControl config, broken down by variation. This helps you understand which configurations are used most frequently.
Performance Metrics: Shows response times and success rates for each interaction. A healthy AgentControl config should maintain high success rates (typically 95%+) and consistent response times.
Cost Analysis: Tracks token usage across different models and providers, helping you optimize spending. Token tracking is essential for cost management and performance optimization. You can see both input and output token counts, which directly correlate to your AI provider costs.

Detailed token usage and cost tracking metrics

The token metrics include:

Input Tokens: The number of tokens sent to the model, including prompt and context. Longer conversations accumulate more input tokens as history grows.
Output Tokens: The number of tokens generated by the model in responses. This varies based on your max_tokens parameter and the model’s verbosity.
Total Token Usage: Combined input and output tokens, which determines your provider billing. Monitor this to predict costs and identify optimization opportunities.
Tokens by Variation: Compare token usage across different variations to identify which configurations are most efficient for your use case.

To learn more about monitoring and metrics, read Monitor AgentControl configs.

Before you ship

You built a working chatbot. When building your own application, check the following before releasing it to end users.

Your config is actually being used

config = ai_client.completion_config(key, context, fallback)
tracker = config.create_tracker()

# Log this somewhere visible
print(f"Using model: {config.model.name}")
print(f"Provider: {config.provider.name}")

If you always see the fallback model, your LaunchDarkly connection is not working.

Errors do not crash the application

try:
    response = provider.generate(config, user_message)
except Exception as e:
    logger.error(f"AI generation failed: {e}")
    response = "Sorry, I'm having trouble right now. Try again?"

AI providers can become unavailable. Your application should continue to function.

You have a rollout plan

Do not roll out changes to all users at once:

Test with your internal team at 5%
Roll out to beta users at 25%
Monitor error rates and token usage
Gradually increase to 100%

LaunchDarkly makes this easy with percentage rollouts on the Targeting tab.

Online evals are considered

You will not know if your AI is giving good answers unless you measure it. Consider adding online evaluations once you are live. Read when to add online evals for guidance.

When AgentControl makes sense

AgentControl configs are not the right fit for every project. Here are some common use cases:

You’re experimenting with prompts

If you are updating prompts frequently, hardcoding them can slow iteration. AgentControl lets you test different prompts without redeploying. Example: You run a customer support chatbot. You want to test whether a formal tone or casual tone works better. With AgentControl configs, you create two variations and switch between them from the dashboard.

You need different AI behavior for different users

Free users get faster, lower-cost responses. Paid users get slower, higher-quality responses. Example: A SaaS application with tiered pricing. The free tier uses gpt-4o-mini with temperature 0.3. The premium tier uses claude-3-5-sonnet with temperature 0.7. You target based on the tier attribute in the user context.

You want to switch providers without code changes

Your primary provider is unavailable. You need to switch to a backup immediately. Example: Anthropic has an outage. You log in to LaunchDarkly, change the default variation from Anthropic to OpenAI, and save. All requests now use OpenAI, with no redeployment needed.

You’re running cost optimization experiments

You think a lower-cost model might perform well enough for most queries. You want to test it with real traffic. Example: You create one variation using claude-3-haiku and another using claude-3-5-sonnet. You roll out the lower-cost model to 20% of users and compare quality metrics.

When AgentControl might be overkill

One-off batch jobs: If you’re processing 10,000 documents once, hardcoding the configuration may be sufficient.
Single model, no experimentation: If you’re using one model and do not plan to change it, AgentControl may add unnecessary complexity.

Completion mode versus agent mode

If you’re using LangGraph or CrewAI, use agent mode instead of the completion mode shown in this tutorial. This tutorial uses completion mode with messages in array format. If you’re building:

Simple chatbots, content generation, or single-turn responses: Use completion mode
Complex multi-step workflows with LangGraph, CrewAI, or custom agents: Use agent mode

The choice depends on your architecture. If you’re calling client.chat.completions.create() or a similar method, completion mode is probably the right choice.

Conclusion

In this guide, you learned how to:

Build an AI chatbot with support for multiple providers
Create and manage AI variations in LaunchDarkly
Use contexts for targeted AI behavior
Monitor AI performance and usage

LaunchDarkly AgentControl let you manage AI behavior across multiple providers without code changes, so you can iterate more quickly and deploy more safely.

​Prerequisites

​Required accounts

​Development environment requirements

​API and SDK keys

​Before you start

​Do not cache configs across users

​Provide a fallback config

​Check if the config is enabled

​Do not include personally identifiable information (PII) in contexts

​Limit conversation history

​Track token usage

​Example 1: Your first chatbot

​Step 1.1: Project setup

​Step 1.2: Install dependencies

​Step 1.3: Environment configuration

​Step 1.4: Create the basic chatbot

​Step 1.5: Run your basic chatbot

​Part 2: Creating your first AgentControl config with two variations

​Step 2.1: Create an AgentControl config in LaunchDarkly

​Step 2.2: Add your AgentControl config key to your .env file

​Step 2.3: Edit targeting

​Step 2.4: Test the friendly variation

​Step 2.5: Real-time configuration changes with no redeploy

​Part 3: Advanced configuration and persona-based targeting

​Step 3.1: Create persona-based variations

​Step 3.2: Configure persona-based targeting

​Step 3.3: Test different personas

​Part 4: Monitoring and verifying data

​Step 4.1: Understanding AI metrics

​Tracking Methods

​Step 4.2: Add comprehensive tracking

​Step 4.3: Testing complete monitoring flow

​Step 4.4: Verify data in LaunchDarkly

​Before you ship

​Your config is actually being used

​Errors do not crash the application

​You have a rollout plan

​Online evals are considered

​When AgentControl makes sense

​You’re experimenting with prompts

​You need different AI behavior for different users

​You want to switch providers without code changes

​You’re running cost optimization experiments

​When AgentControl might be overkill

​Completion mode versus agent mode

​Conclusion

Prerequisites

Required accounts

Development environment requirements

API and SDK keys

Before you start

Do not cache configs across users

Provide a fallback config

Check if the config is enabled

Do not include personally identifiable information (PII) in contexts

Limit conversation history

Track token usage

Example 1: Your first chatbot

Step 1.1: Project setup

Step 1.2: Install dependencies

Step 1.3: Environment configuration

Step 1.4: Create the basic chatbot

Step 1.5: Run your basic chatbot

Part 2: Creating your first AgentControl config with two variations

Step 2.1: Create an AgentControl config in LaunchDarkly

Step 2.2: Add your AgentControl config key to your .env file

Step 2.3: Edit targeting

Step 2.4: Test the friendly variation

Step 2.5: Real-time configuration changes with no redeploy

Part 3: Advanced configuration and persona-based targeting

Step 3.1: Create persona-based variations

Step 3.2: Configure persona-based targeting

Step 3.3: Test different personas

Part 4: Monitoring and verifying data

Step 4.1: Understanding AI metrics

Tracking Methods

Step 4.2: Add comprehensive tracking

Step 4.3: Testing complete monitoring flow

Step 4.4: Verify data in LaunchDarkly

Before you ship

Your config is actually being used

Errors do not crash the application

You have a rollout plan

Online evals are considered

When AgentControl makes sense

You’re experimenting with prompts

You need different AI behavior for different users

You want to switch providers without code changes

You’re running cost optimization experiments

When AgentControl might be overkill

Completion mode versus agent mode

Conclusion