AgentControlManage AgentControl

Configure token limits

This topic explains how to configure token limits for offline evaluations in AgentControl.

As a way to manage AI costs, Admins can set a uniform daily token limit for each member’s offline evaluations. Token usage counts towards the daily limit regardless of which large language model (LLM) provider API key the member uses. Each member’s token allotment resets daily at 00:00 UTC. Token limits default to disabled.

Prerequisites

To configure token limits, you must have the following prerequisites:

  • You must have an Admin or Owner role in your LaunchDarkly organization
  • An Admin or Owner role in your LaunchDarkly organization
  • Access to AgentControl

Viewing token usage

Only Admins and Owners can access the Evaluation tokens page. To view token usage:

  1. Log in to LaunchDarkly as an Admin or Owner.
  2. Click Settings. Under “Plans and usage,” click Evaluation tokens.

Your organization’s average token usage for the current day appears under “Today’s usage,” along with a breakdown showing each member’s usage.

All members, regardless of role, can track their own personal token usage from the Daily token usage counter in the Playground sidebar. To reveal the sidebar, select the Show sidebar button.

Setting a daily token limit

Only Admins and Owners can set a daily token limit. To set a daily token limit for your organization:

  1. Log in to LaunchDarkly as an Admin or Owner.
  2. Click Settings. Under “Plans and usage,” click Evaluation tokens.
  3. Under “Daily token limit per member,” choose one of the following options:
    • Drag the slider to the token amount you want to set, or
    • Enter a token amount in the tokens field.
  4. Click Save.

Admins can allot a maximum of 10,000,000 tokens to each member, or set the limit to “No limit.” This limit applies to every member individually. You cannot set different limits for different members.

Reaching the token limit

Token limits help your organization manage AI costs predictably. When a member exhausts their daily token limit, LaunchDarkly blocks all subsequent offline evaluation runs. The member also sees the following banner text:

Daily token limit banner text

You have reached your daily token limit. Wait until it resets in {AMOUNT} hours, or ask your LaunchDarkly administrator to raise your limit.

Restoring tokens

Token usage resets automatically at 00:00 UTC each day. Admins can also restore tokens before the daily reset in the following ways:

  • Increase the daily token limit
  • Disable the token limit

Token usage during concurrent offline evaluations

Because token usage is calculated asynchronously, members who are close to their daily token limit may exceed it slightly when they run an offline evaluation. For example, even if a member’s token limit is 10,000 tokens, the member’s ending usage might show 11,500 tokens. The member cannot start another offline evaluation after they exceed their token limit.

When working with multi-row datasets, the evaluation stops mid-run if the token usage is exceeded after a certain row. LaunchDarkly saves all completed rows.

Next steps