[Virtual event] Automate Your Software Factory - Nov 5 — Save my seat
Stay up to date with our latest product changes.
By supplying my contact information, I authorize LaunchDarkly to contact me with personalized marketing communications about our products and services. See our Privacy Policy for more details, or Opt-Out at any time.
Run evaluations from code
You can now run AgentControl offline evaluations from a script or a CI job, against the same code your application runs in production. A single `run()` call in the Python or Node.js AI SDK sends each row of a dataset through your own handler. That handler calls your model provider and your real tools, the same way your app does. The SDK records every tool call, so a judge can check that your agent called the right tools, in the right order, with the right arguments.
You can score each output with checks you write in code, LaunchDarkly judges, or both. Each run returns pass or fail, so a regression can fail your build, and the results appear on the Evaluations page in LaunchDarkly. Evaluations from code is included in AgentControl at no extra cost, and model calls use your own provider credentials.
To get started, click Evaluations, then Run evaluations, then From code. To learn more, read https://launchdarkly.com/docs/home/agentcontrol/evaluations-from-code.
Agent Skills is generally available
Agent Skills is out of beta. You can manage your agents' instruction files in the AgentControl Library, version them, and attach them to config variations without shipping code. To learn more, read https://launchdarkly.com/docs/home/agentcontrol/agent-skills.
AI SDK 1.0 for Python and Node.js
The Python and Node.js AI SDKs have reached 1.0. Evaluations from code and Agent Skills need them, so upgrade to `launchdarkly-ai-server` 1.0.0 or later with Python 3.12 or later, or `@launchdarkly/ai-node` 1.0.0 or later with Node.js 22 or later. To learn more, read https://launchdarkly.com/docs/sdk/ai.
For help or feedback, contact your LaunchDarkly account team or email product@launchdarkly.com.
Share link copied!
Agent skills are now available in beta. A skill is a Markdown file that tells an agent how to do a specific job. You can now write, version, and roll out those files from AgentControl instead of keeping them in your codebase and redeploying every time one changes.
Create a skill in the Library under Agents, attach it to a config variation, and the SDK delivers it to your running application. Because the skill is attached to a config, it picks up the targeting, rollouts, approvals, version history, and audit trail you already use everywhere else in AgentControl. You can see every config that references a skill before you change it, and publishing a new version never changes a running agent until you choose to.
The SDK can write skills to disk where runtimes such as the Claude Agent SDK expect to find them, hand the content to your own code, or pass the pinned references through to a model provider that hosts skills itself.
The beta works with the Python and Node.js AI SDKs.
To get started, read https://launchdarkly.com/docs/home/agentcontrol/agent-skills and https://launchdarkly.com/docs/sdk/features/agent-skills.
Share link copied!
You can now run a warehouse native experiment where audience assignment comes from a table in your own warehouse rather than a LaunchDarkly flag evaluation. If your team randomizes in its own service, or assignment happens in a third-party tool such as an email platform, those tests can now be analyzed in LaunchDarkly with the same statistics, exposures, and sliced results as any other experiment, rather than rebuilt in a notebook or dbt.
It works backwards too: assignment data already sitting in your warehouse can be analyzed retroactively, so you can get a read on a past experiment without waiting out a fresh one. Define an assignment source once and reuse it across experiments in the same project and environment.
See Assignment data sources and Manage assignment data sources in our documentation to get started.
Share link copied!
A dataset holds the inputs for an offline evaluation, where each row is one case that AgentControl runs your config against and scores. You can now edit those rows in place instead of uploading a replacement file every time one of them needs to change.
Open a dataset from the AgentControl Library page and you get a Rows tab. Add a row by hand or append a file, edit a row's input, expected output, variables and metadata, search by input or expected output, drag rows into a different order, and delete the ones you no longer want. You can also create an empty dataset and add rows to it, rather than needing a finished CSV or JSONL file first. Changes stage until you click Save, so a whole editing session commits at once.
Each save creates a new dataset version that records who made the change, when, and the resulting row count, and you can open any earlier version or restore it. Evaluation runs pin to the version that existed when the run started, so editing a dataset never changes the results of a run you have already completed.
To get started, read Datasets.
To get help or share feedback, contact your LaunchDarkly account team or email product@launchdarkly.com.
Share link copied!
You can now work out how much traffic a frequentist experiment needs, and how long it will take to get there, inside LaunchDarkly. Start from the experiment list for a quick gut-check before you commit to a design, or from the builder to attach the estimate to that iteration, so the experiment carries its own sample size target and anyone reviewing it can see when it's ready for a decision. You can view, edit, or remove that target from the results page while the experiment runs.
Settling on a sample size up front is what holds your false positive rate to the significance level you chose, and until now it meant a spreadsheet or a separate trip to our website. Estimates assume steady traffic, so an experiment may hit its sample size earlier or later. Available for fixed-horizon frequentist experiments.
Read Sample size calculations (https://launchdarkly.com/docs/home/experimentation/sample-size-calculator) and Sample size calculations for frequentist experiments (https://launchdarkly.com/docs/guides/experimentation/sample-size-calc) to get started.
Share link copied!
