> ## Documentation Index
> Fetch the complete documentation index at: https://launchdarkly.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Amazon S3 Data Export

<View title="Developer" />

<View title="Federal docs">
  > [!CAUTION]
  >
  > **This integration is not available in federal environments**
  >
  > To learn more, read [LaunchDarkly in federal environments](/docs/home/infrastructure/federal).
</View>

<View title="EU docs" />

> [!NOTE]
>
> **Data Export is an add-on feature**
>
> Data Export is available as an add-on feature to select plans. To learn more, [read about our pricing](https://launchdarkly.com/pricing/). To add Data Export to your plan, [contact Sales](https://launchdarkly.com/contact-sales/).

This topic explains how to create and test an Amazon S3 destination for Data Export. Amazon S3 is an object storage service that stores your data as files in a bucket. By exporting your LaunchDarkly flag evaluation and experiment data to an S3 bucket, you can archive the data or load it into your own data pipeline and analytics tools.

## Prerequisites

To configure the S3 Data Export integration, you must have the following prerequisites:

* Data Export enabled for the project and environment you want to export from.
* An Amazon S3 bucket, and permissions in AWS to create IAM roles and policies. To learn more, read [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html) in the AWS documentation.
* Access to the AWS Management Console or AWS Command Line Interface (CLI).
* A [LaunchDarkly role](/docs/home/account/roles) that allows you to [add and edit integrations](/docs/home/account/roles/role-actions.md#integration-actions) and [add destinations](/docs/home/account/roles/role-actions.md#destination-actions).

## How S3 Data Export authenticates

S3 Data Export uses AWS Identity and Access Management (IAM) role-based access. You create an IAM role in your AWS account and grant LaunchDarkly permission to assume that role, so LaunchDarkly can write export files to your bucket without storing long-lived access keys.

LaunchDarkly generates two policies for you during setup:

* An IAM permissions policy that grants write access to your bucket.
* An IAM trust policy that allows the LaunchDarkly data transfer service to assume your role.

You copy both policies from LaunchDarkly into AWS, create the role, and paste the role's ARN back into LaunchDarkly.

## Create an S3 bucket

First, create the bucket that LaunchDarkly writes exported files to. We recommend using a dedicated bucket for LaunchDarkly Data Export to keep the export data separate from other workloads and to scope permissions tightly.

To create a bucket:

1. Navigate to the [Amazon S3 console](https://console.aws.amazon.com/s3/) and click **Create bucket**.
2. Enter a **Bucket name** and select an **AWS Region**. Save the bucket and region information to enter into LaunchDarkly in the next section.
3. Adjust the remaining settings as needed. You can keep "Block all public access" enabled.
4. Click **Create bucket**.

## Start the integration in LaunchDarkly

Next, begin configuring the destination in LaunchDarkly:

1. In LaunchDarkly, navigate to the [**Integrations**](https://app.launchdarkly.com/settings/integrations) page and find "S3 Data Export."
2. Click **Add new**. The "Create S3 Data Export configuration" page appears.
3. Give the integration a human-readable **Name**.
4. Select a **Project and environment** to export data from.
5. Enter the **Bucket name** of the bucket you created in the previous section.
6. Select the **Bucket region** that matches the region of your bucket you selected in the previous section.
7. (Optional) Enter a **Folder name**. This is a prefix within the bucket that LaunchDarkly writes exported files under. If you leave it blank, LaunchDarkly uses `ld_export`.
8. Select an **Exported file format**. Parquet is the default and recommended format. To learn more, read [Supported file formats](#supported-file-formats).
9. Click **Generate policies**. LaunchDarkly displays an **IAM permissions policy** and an **IAM trust policy**. Copy each policy to your clipboard using its copy button. You use these policies in the following two sections.

When you click **Generate policies**, LaunchDarkly saves a pending destination named `S3 Data Export - ENVIRONMENT-KEY`, where `ENVIRONMENT-KEY` is the key of the environment you selected. You can keep this page open while you work in AWS, or close it and finish setup later.

To finish setup later:

1. Find the pending destination on the **Integrations** page under "S3 Data Export" and open it. The pending destination does not keep the details you entered.
2. Re-enter the bucket name, bucket region, folder name, and file format, and change the **Name** if you want a custom one. The file format resets to Parquet. Select CSV, JSON, or JSONL again if you chose one of those.
3. Click **Generate policies** again.
4. Continue with [Test and save the configuration](#test-and-save-the-configuration).

Each environment can have only one S3 Data Export destination. If an environment already has a completed S3 destination, you must delete it before you can create a new one.

## Create the IAM policy in AWS

Use the IAM permissions policy you copied to create a policy in AWS:

1. In the [AWS IAM console](https://console.aws.amazon.com/iam/), navigate to **Policies** and click **Create policy**.
2. Select the **JSON** editor.
3. Paste the IAM permissions policy you copied from LaunchDarkly.
4. Click **Next**, name the policy, and click **Create policy**.

You attach this policy to the role in the next section.

The generated IAM permissions policy looks like this, with `BUCKET_NAME` replaced by your bucket name:

```json title="IAM permissions policy" lines wrap theme={null}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::BUCKET_NAME/*"
    }
  ]
}
```

LaunchDarkly needs `s3:PutObject` to write exported files. It needs `s3:DeleteObject` only for the connection test that runs when you first save the destination. You can remove `s3:DeleteObject` from the policy after setup is complete. If you remove `s3:DeleteObject` and later create a new destination with the same IAM role, add it back before you test the connection.

## Create the IAM role in AWS

Next, create the IAM role that LaunchDarkly assumes:

1. In the AWS IAM console, navigate to **Roles** and click **Create role**.
2. Select **Custom trust policy**.
3. Paste the IAM trust policy you copied from LaunchDarkly, then click **Next**.
4. Select the permissions policy you created in the previous section, then click **Next**.
5. Enter a **Role name**, then click **Create role**.
6. Open the role you created and copy its **ARN**.

## Test and save the configuration

Finally, return to LaunchDarkly to finish setup:

1. Paste the role's ARN into the **IAM role ARN** field.
2. After reading the Integration Terms and Conditions, check the **I have read and agree to the Integration Terms and Conditions** checkbox.
3. Click **Test connection and save**.

If the connection test succeeds, LaunchDarkly creates the destination and begins the first data transfer. The initial transfer can take 30 to 60 minutes to complete.

Your S3 Data Export destination is now configured.

## Editing an existing destination

After setup is complete, you can change only the destination's **Name** and **Maintainer**. You can change the **Name** using the edit button next to the destination name, and the **Maintainer** from the side panel.

The project and environment, bucket name, bucket region, folder name, exported file format, and IAM role ARN are permanent and you cannot change them. If you need to update these settings, delete the destination and create a new one.

## Health checks

If your S3 Data Export integration develops connection problems or other errors, the integration displays a health check warning. To learn more, read [Warehouse health checks](/docs/home/warehouse-native/warehouse-health-checks.md).

## Supported file formats

S3 Data Export can write exported data in the following file formats:

* Parquet: this is the default and recommended format for its compression and query performance
* CSV
* JSON
* JSON Lines (JSONL)

## How data is organized in your bucket

LaunchDarkly writes exported data as files in [Apache Hive](https://hive.apache.org/)-style partitions, using the following path within your bucket:

```text title="Bucket path" lines wrap theme={null}
<folder>/<model_name>/dt=<transfer_date>/<file_part>_<transfer_timestamp>.<extension>
```

In this path, `<folder>` is the folder name you configured, and `<model_name>` is the name of the exported data model. Each transfer can write more than one file per model.

To signal when a transfer completes, LaunchDarkly writes a manifest file for each model under a `_manifests` folder at the root of your configured folder. Downstream pipelines can watch for these manifest files to determine when a transfer's data is ready to read.

> [!NOTE]
>
> **Exported data can contain duplicate records**
>
> Object storage is append-only, and LaunchDarkly re-scans a lookback window on each transfer to make sure no data is missed. As a result, the same record can appear in more than one transfer. Deduplicate on the primary key in your downstream pipeline, keeping the most recent record.

## Use a bucket with KMS encryption

If your bucket is encrypted with an AWS Key Management Service (KMS) customer-managed key (CMK), the IAM permissions policy also needs permission to use that key. Add the following statement to the `Statement` array of the IAM permissions policy before you create the policy in AWS, and replace `REGION`, `ACCOUNT_ID`, and `KEY_ID` with your values:

```json title="Encryption statement" lines wrap theme={null}
{
  "Effect": "Allow",
  "Action": ["kms:GenerateDataKey", "kms:Decrypt"],
  "Resource": "arn:aws:kms:REGION:ACCOUNT_ID:key/KEY_ID"
}
```

Buckets encrypted with server-side encryption with customer-provided keys (SSE-C) are not supported.