For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
Sign inTry it free
DocsGuidesSDKsIntegrationsAPI docsTutorialsFlagship blog
DocsGuidesSDKsIntegrationsAPI docsTutorialsFlagship blog
  • REST API overview
    • Access Tokens
    • Account Members
    • Account Usage Beta
    • AI Configs
    • Announcements
    • Applications Beta
    • Approvals
    • Approvals Beta
    • Audit Log
    • Code References
    • Contexts
    • Context Settings
    • Custom Roles
    • Data Export Destinations
    • Environments
    • Experiments
    • Feature Flags
    • Feature Flags Beta
    • Flag Import Configurations Beta
    • Flag Links Beta
    • Flag Triggers
    • Follow Flags
    • Holdouts Beta
      • POSTCreate holdout
      • GETGet all holdouts
      • GETGet holdout
      • GETGet Holdout by Id
      • PATCHPatch holdout
    • Insights Charts Beta
    • Insights Deployments Beta
    • Insights Flag Events Beta
    • Insights Pull Requests Beta
    • Insights Repositories Beta
    • Insights Scores Beta
    • Integration Audit Log Subscriptions
    • Integration Delivery Configurations Beta
    • Integrations Beta
    • Layers
    • Metrics
    • Metrics Beta
    • OAuth2 Clients
    • Persistent Store Integrations Beta
    • Projects
    • Relay Proxy Configurations
    • Release Pipelines Beta
    • Releases Beta
    • Scheduled Changes
    • Segments
    • Tags
    • Teams
    • Teams Beta
    • Users
    • Users Beta
    • User Settings
    • Views Beta
    • Webhooks
    • Workflows
    • Workflow Templates
    • Other
Sign inTry it free
LogoLogo
REST API overviewHoldouts Beta

Create holdout

POST
/api/v2/projects/:projectKey/environments/:environmentKey/holdouts
POST
/api/v2/projects/:projectKey/environments/:environmentKey/holdouts
$curl -X POST https://app.launchdarkly.com/api/v2/projects/projectKey/environments/environmentKey/holdouts \
>     -H "Authorization: <apiKey>" \
>     -H "Content-Type: application/json" \
>     -d '{}'
1{
2  "_id": "string",
3  "status": "created",
4  "holdoutAmount": "string",
5  "createdAt": 1,
6  "updatedAt": 1,
7  "baseExperiment": {
8    "key": "experiment-key-123abc",
9    "name": "Example experiment",
10    "_maintainerId": "12ab3c45de678910fgh12345",
11    "_creationDate": 1654104600000,
12    "environmentKey": "string",
13    "_links": {
14      "parent": {
15        "href": "/api/v2/projects/my-project/environments/my-environment",
16        "type": "application/json"
17      },
18      "self": {
19        "href": "/api/v2/projects/my-project/environments/my-environment/experiments/my-experiment",
20        "type": "application/json"
21      }
22    },
23    "_id": "12ab3c45de678910fgh12345",
24    "description": "An example experiment, used in testing",
25    "methodology": "bayesian",
26    "dataSource": "snowflake",
27    "archivedDate": 1654104600000,
28    "tags": [
29      "experiment",
30      "feature"
31    ],
32    "holdoutId": "f3b74309-d581-44e1-8a2b-bb2933b4fe40",
33    "currentIteration": {
34      "hypothesis": "The new button placement will increase conversion",
35      "status": "running",
36      "createdAt": 1654104600000,
37      "_id": "12ab3c45de678910fgh12345",
38      "startedAt": 1655314200000,
39      "endedAt": 1656610200000,
40      "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58",
41      "winningReason": "We ran this iteration for two weeks and the winning variation was clear",
42      "canReshuffleTraffic": true,
43      "flags": {},
44      "reallocationFrequencyMillis": 3600000,
45      "version": 0,
46      "primaryMetric": {
47        "key": "metric-key-123abc",
48        "_versionId": "string",
49        "name": "My metric",
50        "kind": "custom",
51        "_links": {
52          "self": {
53            "href": "/api/v2/metrics/my-project/my-metric",
54            "type": "application/json"
55          }
56        },
57        "isGroup": true,
58        "isNumeric": true,
59        "eventKey": "event-key-123abc",
60        "metrics": [
61          {
62            "key": "metric-key-123abc",
63            "name": "Example metric",
64            "kind": "custom",
65            "_links": {
66              "self": {
67                "href": "/api/v2/metrics/my-project/my-metric",
68                "type": "application/json"
69              }
70            },
71            "_versionId": "version-id-123abc",
72            "isNumeric": true,
73            "unitAggregationType": "sum",
74            "eventKey": "event-key-123abc",
75            "nameInGroup": "Step 1",
76            "analysisUnits": [
77              "user"
78            ],
79            "randomizationUnits": [
80              "user"
81            ]
82          }
83        ]
84      },
85      "primarySingleMetric": {
86        "key": "metric-key-123abc",
87        "name": "Example metric",
88        "kind": "custom",
89        "_links": {
90          "self": {
91            "href": "/api/v2/metrics/my-project/my-metric",
92            "type": "application/json"
93          }
94        },
95        "_versionId": "version-id-123abc",
96        "isNumeric": true,
97        "unitAggregationType": "sum",
98        "eventKey": "event-key-123abc"
99      },
100      "primaryFunnel": {
101        "key": "metric-group-key-123abc",
102        "name": "My metric group",
103        "kind": "funnel",
104        "_links": {
105          "parent": {
106            "href": "/api/v2/projects/my-project",
107            "type": "application/json"
108          },
109          "self": {
110            "href": "/api/v2/projects/my-project/metric-groups/my-metric-group",
111            "type": "application/json"
112          }
113        },
114        "metrics": [
115          {
116            "key": "metric-key-123abc",
117            "name": "Example metric",
118            "kind": "custom",
119            "_links": {
120              "self": {
121                "href": "/api/v2/metrics/my-project/my-metric",
122                "type": "application/json"
123              }
124            },
125            "_versionId": "version-id-123abc",
126            "isNumeric": true,
127            "unitAggregationType": "sum",
128            "eventKey": "event-key-123abc",
129            "nameInGroup": "Step 1",
130            "analysisUnits": [
131              "user"
132            ],
133            "randomizationUnits": [
134              "user"
135            ]
136          }
137        ]
138      },
139      "randomizationUnit": "user",
140      "attributes": [
141        "string"
142      ],
143      "treatments": [
144        {
145          "name": "Treatment 1",
146          "allocationPercent": "10",
147          "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58",
148          "baseline": true,
149          "parameters": [
150            {
151              "variationId": "string",
152              "flagKey": "string"
153            }
154          ]
155        }
156      ],
157      "metrics": [
158        {
159          "key": "metric-key-123abc",
160          "_versionId": "string",
161          "name": "My metric",
162          "kind": "custom",
163          "_links": {
164            "self": {
165              "href": "/api/v2/metrics/my-project/my-metric",
166              "type": "application/json"
167            }
168          },
169          "isGroup": true,
170          "isNumeric": true,
171          "eventKey": "event-key-123abc",
172          "metrics": [
173            {
174              "key": "metric-key-123abc",
175              "name": "Example metric",
176              "kind": "custom",
177              "_links": {
178                "self": {
179                  "href": "/api/v2/metrics/my-project/my-metric",
180                  "type": "application/json"
181                }
182              },
183              "_versionId": "version-id-123abc",
184              "isNumeric": true,
185              "unitAggregationType": "sum",
186              "eventKey": "event-key-123abc",
187              "nameInGroup": "Step 1",
188              "analysisUnits": [
189                "user"
190              ],
191              "randomizationUnits": [
192                "user"
193              ]
194            }
195          ]
196        }
197      ],
198      "layerSnapshot": {
199        "key": "checkout-flow",
200        "name": "Checkout Flow",
201        "reservationPercent": 10,
202        "otherReservationPercent": 70
203      },
204      "covariateInfo": {
205        "id": "74a49a2b-4834-4246-917e-5d85231d8c2a",
206        "fileName": "covariate.csv",
207        "createdAt": 1654104600000
208      },
209      "secondaryMetrics": [
210        {
211          "key": "metric-key-123abc",
212          "name": "Example metric",
213          "kind": "custom",
214          "_links": {
215            "self": {
216              "href": "/api/v2/metrics/my-project/my-metric",
217              "type": "application/json"
218            }
219          },
220          "_versionId": "version-id-123abc",
221          "isNumeric": true,
222          "unitAggregationType": "sum",
223          "eventKey": "event-key-123abc"
224        }
225      ]
226    },
227    "type": "experiment",
228    "draftIteration": {
229      "hypothesis": "The new button placement will increase conversion",
230      "status": "running",
231      "createdAt": 1654104600000,
232      "_id": "12ab3c45de678910fgh12345",
233      "startedAt": 1655314200000,
234      "endedAt": 1656610200000,
235      "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58",
236      "winningReason": "We ran this iteration for two weeks and the winning variation was clear",
237      "canReshuffleTraffic": true,
238      "flags": {},
239      "reallocationFrequencyMillis": 3600000,
240      "version": 0,
241      "primaryMetric": {
242        "key": "metric-key-123abc",
243        "_versionId": "string",
244        "name": "My metric",
245        "kind": "custom",
246        "_links": {
247          "self": {
248            "href": "/api/v2/metrics/my-project/my-metric",
249            "type": "application/json"
250          }
251        },
252        "isGroup": true,
253        "isNumeric": true,
254        "eventKey": "event-key-123abc",
255        "metrics": [
256          {
257            "key": "metric-key-123abc",
258            "name": "Example metric",
259            "kind": "custom",
260            "_links": {
261              "self": {
262                "href": "/api/v2/metrics/my-project/my-metric",
263                "type": "application/json"
264              }
265            },
266            "_versionId": "version-id-123abc",
267            "isNumeric": true,
268            "unitAggregationType": "sum",
269            "eventKey": "event-key-123abc",
270            "nameInGroup": "Step 1",
271            "analysisUnits": [
272              "user"
273            ],
274            "randomizationUnits": [
275              "user"
276            ]
277          }
278        ]
279      },
280      "primarySingleMetric": {
281        "key": "metric-key-123abc",
282        "name": "Example metric",
283        "kind": "custom",
284        "_links": {
285          "self": {
286            "href": "/api/v2/metrics/my-project/my-metric",
287            "type": "application/json"
288          }
289        },
290        "_versionId": "version-id-123abc",
291        "isNumeric": true,
292        "unitAggregationType": "sum",
293        "eventKey": "event-key-123abc"
294      },
295      "primaryFunnel": {
296        "key": "metric-group-key-123abc",
297        "name": "My metric group",
298        "kind": "funnel",
299        "_links": {
300          "parent": {
301            "href": "/api/v2/projects/my-project",
302            "type": "application/json"
303          },
304          "self": {
305            "href": "/api/v2/projects/my-project/metric-groups/my-metric-group",
306            "type": "application/json"
307          }
308        },
309        "metrics": [
310          {
311            "key": "metric-key-123abc",
312            "name": "Example metric",
313            "kind": "custom",
314            "_links": {
315              "self": {
316                "href": "/api/v2/metrics/my-project/my-metric",
317                "type": "application/json"
318              }
319            },
320            "_versionId": "version-id-123abc",
321            "isNumeric": true,
322            "unitAggregationType": "sum",
323            "eventKey": "event-key-123abc",
324            "nameInGroup": "Step 1",
325            "analysisUnits": [
326              "user"
327            ],
328            "randomizationUnits": [
329              "user"
330            ]
331          }
332        ]
333      },
334      "randomizationUnit": "user",
335      "attributes": [
336        "string"
337      ],
338      "treatments": [
339        {
340          "name": "Treatment 1",
341          "allocationPercent": "10",
342          "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58",
343          "baseline": true,
344          "parameters": [
345            {
346              "variationId": "string",
347              "flagKey": "string"
348            }
349          ]
350        }
351      ],
352      "metrics": [
353        {
354          "key": "metric-key-123abc",
355          "_versionId": "string",
356          "name": "My metric",
357          "kind": "custom",
358          "_links": {
359            "self": {
360              "href": "/api/v2/metrics/my-project/my-metric",
361              "type": "application/json"
362            }
363          },
364          "isGroup": true,
365          "isNumeric": true,
366          "eventKey": "event-key-123abc",
367          "metrics": [
368            {
369              "key": "metric-key-123abc",
370              "name": "Example metric",
371              "kind": "custom",
372              "_links": {
373                "self": {
374                  "href": "/api/v2/metrics/my-project/my-metric",
375                  "type": "application/json"
376                }
377              },
378              "_versionId": "version-id-123abc",
379              "isNumeric": true,
380              "unitAggregationType": "sum",
381              "eventKey": "event-key-123abc",
382              "nameInGroup": "Step 1",
383              "analysisUnits": [
384                "user"
385              ],
386              "randomizationUnits": [
387                "user"
388              ]
389            }
390          ]
391        }
392      ],
393      "layerSnapshot": {
394        "key": "checkout-flow",
395        "name": "Checkout Flow",
396        "reservationPercent": 10,
397        "otherReservationPercent": 70
398      },
399      "covariateInfo": {
400        "id": "74a49a2b-4834-4246-917e-5d85231d8c2a",
401        "fileName": "covariate.csv",
402        "createdAt": 1654104600000
403      },
404      "secondaryMetrics": [
405        {
406          "key": "metric-key-123abc",
407          "name": "Example metric",
408          "kind": "custom",
409          "_links": {
410            "self": {
411              "href": "/api/v2/metrics/my-project/my-metric",
412              "type": "application/json"
413            }
414          },
415          "_versionId": "version-id-123abc",
416          "isNumeric": true,
417          "unitAggregationType": "sum",
418          "eventKey": "event-key-123abc"
419        }
420      ]
421    },
422    "previousIterations": [
423      {
424        "hypothesis": "The new button placement will increase conversion",
425        "status": "running",
426        "createdAt": 1654104600000,
427        "_id": "12ab3c45de678910fgh12345",
428        "startedAt": 1655314200000,
429        "endedAt": 1656610200000,
430        "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58",
431        "winningReason": "We ran this iteration for two weeks and the winning variation was clear",
432        "canReshuffleTraffic": true,
433        "flags": {},
434        "reallocationFrequencyMillis": 3600000,
435        "version": 0,
436        "primaryMetric": {
437          "key": "metric-key-123abc",
438          "_versionId": "string",
439          "name": "My metric",
440          "kind": "custom",
441          "_links": {
442            "self": {
443              "href": "/api/v2/metrics/my-project/my-metric",
444              "type": "application/json"
445            }
446          },
447          "isGroup": true,
448          "isNumeric": true,
449          "eventKey": "event-key-123abc",
450          "metrics": [
451            {
452              "key": "metric-key-123abc",
453              "name": "Example metric",
454              "kind": "custom",
455              "_links": {
456                "self": {
457                  "href": "/api/v2/metrics/my-project/my-metric",
458                  "type": "application/json"
459                }
460              },
461              "_versionId": "version-id-123abc",
462              "isNumeric": true,
463              "unitAggregationType": "sum",
464              "eventKey": "event-key-123abc",
465              "nameInGroup": "Step 1",
466              "analysisUnits": [
467                "user"
468              ],
469              "randomizationUnits": [
470                "user"
471              ]
472            }
473          ]
474        },
475        "primarySingleMetric": {
476          "key": "metric-key-123abc",
477          "name": "Example metric",
478          "kind": "custom",
479          "_links": {
480            "self": {
481              "href": "/api/v2/metrics/my-project/my-metric",
482              "type": "application/json"
483            }
484          },
485          "_versionId": "version-id-123abc",
486          "isNumeric": true,
487          "unitAggregationType": "sum",
488          "eventKey": "event-key-123abc"
489        },
490        "primaryFunnel": {
491          "key": "metric-group-key-123abc",
492          "name": "My metric group",
493          "kind": "funnel",
494          "_links": {
495            "parent": {
496              "href": "/api/v2/projects/my-project",
497              "type": "application/json"
498            },
499            "self": {
500              "href": "/api/v2/projects/my-project/metric-groups/my-metric-group",
501              "type": "application/json"
502            }
503          },
504          "metrics": [
505            {
506              "key": "metric-key-123abc",
507              "name": "Example metric",
508              "kind": "custom",
509              "_links": {
510                "self": {
511                  "href": "/api/v2/metrics/my-project/my-metric",
512                  "type": "application/json"
513                }
514              },
515              "_versionId": "version-id-123abc",
516              "isNumeric": true,
517              "unitAggregationType": "sum",
518              "eventKey": "event-key-123abc",
519              "nameInGroup": "Step 1",
520              "analysisUnits": [
521                "user"
522              ],
523              "randomizationUnits": [
524                "user"
525              ]
526            }
527          ]
528        },
529        "randomizationUnit": "user",
530        "attributes": [
531          "string"
532        ],
533        "treatments": [
534          {
535            "name": "Treatment 1",
536            "allocationPercent": "10",
537            "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58",
538            "baseline": true,
539            "parameters": [
540              {
541                "variationId": "string",
542                "flagKey": "string"
543              }
544            ]
545          }
546        ],
547        "metrics": [
548          {
549            "key": "metric-key-123abc",
550            "_versionId": "string",
551            "name": "My metric",
552            "kind": "custom",
553            "_links": {
554              "self": {
555                "href": "/api/v2/metrics/my-project/my-metric",
556                "type": "application/json"
557              }
558            },
559            "isGroup": true,
560            "isNumeric": true,
561            "eventKey": "event-key-123abc",
562            "metrics": [
563              {
564                "key": "metric-key-123abc",
565                "name": "Example metric",
566                "kind": "custom",
567                "_links": {
568                  "self": {
569                    "href": "/api/v2/metrics/my-project/my-metric",
570                    "type": "application/json"
571                  }
572                },
573                "_versionId": "version-id-123abc",
574                "isNumeric": true,
575                "unitAggregationType": "sum",
576                "eventKey": "event-key-123abc",
577                "nameInGroup": "Step 1",
578                "analysisUnits": [
579                  "user"
580                ],
581                "randomizationUnits": [
582                  "user"
583                ]
584              }
585            ]
586          }
587        ],
588        "layerSnapshot": {
589          "key": "checkout-flow",
590          "name": "Checkout Flow",
591          "reservationPercent": 10,
592          "otherReservationPercent": 70
593        },
594        "covariateInfo": {
595          "id": "74a49a2b-4834-4246-917e-5d85231d8c2a",
596          "fileName": "covariate.csv",
597          "createdAt": 1654104600000
598        },
599        "secondaryMetrics": [
600          {
601            "key": "metric-key-123abc",
602            "name": "Example metric",
603            "kind": "custom",
604            "_links": {
605              "self": {
606                "href": "/api/v2/metrics/my-project/my-metric",
607                "type": "application/json"
608              }
609            },
610            "_versionId": "version-id-123abc",
611            "isNumeric": true,
612            "unitAggregationType": "sum",
613            "eventKey": "event-key-123abc"
614          }
615        ]
616      }
617    ],
618    "analysisConfig": {
619      "bayesianThreshold": "string",
620      "significanceThreshold": "string",
621      "testDirection": "string",
622      "multipleComparisonCorrectionMethod": "bonferroni",
623      "multipleComparisonCorrectionScope": "variations",
624      "sequentialTestingEnabled": true
625    },
626    "mutableFieldsByStatus": {
627      "not_started": {},
628      "running": {},
629      "stopped": {}
630    }
631  },
632  "description": "string",
633  "experiments": [
634    {
635      "key": "string",
636      "name": "string",
637      "environment": "string"
638    }
639  ]
640}
Create a new holdout in the specified project.
Was this page helpful?
Previous

Get all holdouts

Next
Built with

Authentication

Authorizationstring
API Key authentication via header

Path parameters

projectKeystringRequiredformat: "string"
The project key
environmentKeystringRequiredformat: "string"
The environment key

Request

This endpoint expects an object.
namestringOptional

A human-friendly name for the holdout

keystringOptional
A key that identifies the holdout
descriptionstringOptional
Description of the holdout
randomizationunitstringOptional
The chosen randomization unit for the holdout base experiment
attributeslist of stringsOptional
The attributes that the holdout iteration's results can be sliced by
holdoutamountstringOptional
Audience allocation for the holdout
primarymetrickeystringOptional
The key of the primary metric for this holdout
metricslist of objectsOptional
Details on the metrics for this experiment
prerequisiteflagkeystringOptional
The key of the flag that the holdout is dependent on
maintainerIdstringOptional
Maintainer id

Response

Holdout response
_idstring
statusenum
Allowed values:
holdoutAmountstring
The percentage of traffic allocated to this holdout.
createdAtlong
updatedAtlong
baseExperimentobject
descriptionstring
experimentslist of objects

Errors

400
Bad Request Error