AWS FireLens

Overview

This topic explains how to send container logs from Amazon ECS to LaunchDarkly’s observability dashboards using AWS FireLens. FireLens is AWS’s log router for Amazon ECS that lets you route container logs to various destinations using Fluent Bit.

There are two ways to send logs using FireLens:

• Direct forwarding: Use the Fluent Forward protocol to send logs directly to LaunchDarkly’s OpenTelemetry collector.
• Kinesis Firehose: Ship structured JSON logs through AWS Kinesis Data Streams, then forward them to LaunchDarkly using Kinesis Data Firehose. This approach preserves JSON fields in your logs as structured attributes in LaunchDarkly.

After you set up the configuration, these logs appear in the Logs page of the LaunchDarkly user interface.

Prerequisites

Before you configure FireLens, you must:

• Have LaunchDarkly observability enabled for your project
• Have an Amazon ECS cluster running
• Know your LaunchDarkly project key

To learn how to find and copy your LaunchDarkly project key, read Project keys.

Option 1: Direct forwarding with Fluent Forward

This approach uses a Fluent Bit sidecar container with the awsfirelens log driver to send logs directly to LaunchDarkly’s OpenTelemetry collector.

Here is an example ECS task definition:

1
{
2
  "family": "my-app",
3
  "containerDefinitions": [
4
    {
5
      "essential": true,
6
      "image": "906394416424.dkr.ecr.us-east-1.amazonaws.com/aws-for-fluent-bit:stable",
7
      "name": "log_router",
8
      "firelensConfiguration": {
9
        "type": "fluentbit"
10
      }
11
    },
12
    {
13
      "essential": true,
14
      "image": "my-app:latest",
15
      "name": "app",
16
      "logConfiguration": {
17
        "logDriver": "awsfirelens",
18
        "options": {
19
          "Name": "Forward",
20
          "Host": "otel.observability.app.launchdarkly.com",
21
          "Port": "24224",
22
          "Tag": "launchdarkly.project_id=YOUR_PROJECT_KEY"
23
        }
24
      }
25
    }
26
  ]
27
}

Replace YOUR_PROJECT_KEY with your LaunchDarkly project key.

The key configuration values are:

Option 2: Structured JSON logging using Kinesis

This approach ships structured JSON logs through AWS Kinesis Data Streams. It preserves JSON fields in your logs as structured attributes in LaunchDarkly.

Configure the ECS task definition

The task definition uses a Fluent Bit sidecar with the parse-json.conf configuration file to preserve structured JSON fields through the pipeline.

Here is an example ECS task definition:

1
{
2
  "family": "my-app",
3
  "containerDefinitions": [
4
    {
5
      "essential": true,
6
      "image": "public.ecr.aws/aws-observability/aws-for-fluent-bit:stable",
7
      "name": "log_router",
8
      "firelensConfiguration": {
9
        "type": "fluentbit",
10
        "options": {
11
          "config-file-type": "file",
12
          "config-file-value": "/fluent-bit/configs/parse-json.conf",
13
          "enable-ecs-log-metadata": "true"
14
        }
15
      }
16
    },
17
    {
18
      "essential": true,
19
      "image": "my-app:latest",
20
      "name": "app",
21
      "logConfiguration": {
22
        "logDriver": "awsfirelens",
23
        "options": {
24
          "region": "us-east-2",
25
          "stream": "YOUR_KINESIS_DATA_STREAM",
26
          "Name": "kinesis_streams"
27
        }
28
      }
29
    }
30
  ]
31
}

Replace YOUR_KINESIS_DATA_STREAM with the name of your Kinesis Data Stream and update the region to match your AWS region.

Configure Kinesis Data Firehose

Create a Kinesis Data Firehose delivery stream that reads from your Kinesis Data Stream and forwards to LaunchDarkly:

• Source: Your Kinesis Data Stream
• Destination: HTTP endpoint
• Endpoint URL: https://pub.observability.app.launchdarkly.com/v1/logs/firehose

Configure IAM permissions

Update your ECS task execution role to allow writing to your Kinesis Data Stream:

1
{
2
  "Version": "2012-10-17",
3
  "Statement": [
4
    {
5
      "Effect": "Allow",
6
      "Action": "kinesis:PutRecords",
7
      "Resource": "arn:aws:kinesis:us-east-2:YOUR_ACCOUNT_ID:stream/YOUR_KINESIS_DATA_STREAM"
8
    }
9
  ]
10
}

Replace YOUR_ACCOUNT_ID and YOUR_KINESIS_DATA_STREAM with your AWS account ID and Kinesis stream name.

Verify that logs are being received

After you configure FireLens, your container logs begin forwarding to LaunchDarkly.

To verify that your logs are being received:

1. Open the Telemetry section and navigate to the Logs list.
2. Look for logs from your ECS services.

It may take a few minutes for logs to appear after you first configure the log router.

Filtering logs

You can configure ingestion filters to control which logs are stored in LaunchDarkly. This is useful for excluding noisy logs or limiting ingestion to stay within your observability quotas.

To learn more, read Filters.