AWS ECS Fargate – Lacework

Overview

Lacework’s workload security provides visibility into all processes and applications within an organization’s cloud environments such as workload/runtime analysis, and automated anomaly and threat detection.

After you install the Lacework agent, Lacework scans hosts and streams select metadata to the Lacework data warehouse to build a baseline of normal behavior, which is updated hourly. From this, Lacework can provide detailed in-context alerts for anomalous behavior by comparing each hour to the previous one. Anomaly detection uses machine learning to determine, for example, if a machine sends data to an unknown IP, or if a user logs in from an IP that has not been seen before.

Support Information

AWS Fargate platform versions 1.3 or 1.4
OS distributions, see public documentation for supported OS distributions
Lacework agent 3.2 or later
Available from https://github.com/lacework/lacework-agent-releases

Access Tokens

In an environment with Fargate and non-Fargate Lacework deployments, Lacework recommends using different access tokens for Fargate and non-Fargate deployments. This can make deployments easier to manage.

Fargate and ECR Setup

AWS Services Used

ECS (Fargate)->Clusters
ECS (Fargate)->Task Definitions
ECR->Repositories
EC2->Security Groups
EC2->Network Interfaces
VPC->VPCs
VPC->Subnets
VPC->Route Tables
VPC->Internet Gateways
VPC->Security Groups
VPC->Network ACLs
Policy trust configuration

Gather Required Information

distro:ver - Replace with Linux distro and version to use. Should be a supported OS
Lacework Agent Token - Copy the agent token from Lacework
YourLWUrl - Replace with your Lacework server URL
yourAWSacctID - Your AWS ID
region - Set the correct region where the Fargate environment is located (i.e. us-west-1)
logGroupName - Your own name
imageName - Name of the Lacework agent image
configName - Desired name for config
taskDefinition - Your task definition name
ECR Repository URI
useYour/tag - Name the docker image tag
/path/to/Dockerfile - Path to your Dockerfile

Required Build File Templates

Dockerfile
Run.sh
The example starts the Lacework agent as a process. In actual production, the activation should be done within your service execution.

Environment

All region configuration should be the same as the Fargate environment.
Create a repository in ECS or locate the URI of an existing repository to be used. This will be required in Step 6 when preparing the Docker image for the Lacework agent.

Prepare Docker Image for Lacework Agent Replica Service

This will keep you authenticated to the ECR.
Set the correct region and AWS ID in the URL in the following.
- region - Your region
- yourAWSacctID - Your AWS account ID
```
aws ecr get-login-password --region region | docker login --username AWS --password-stdin yourAWSacctID.dkr.ecr.region.amazonaws.com   
```

Prepare the multi-stage Dockerfile (the file name).

Lacework Agent Token - Copy the agent access token from Lacework, Settings > Agents.
Best practice is to use a vault and variabilize the token in the file. Do not pass it as clear text.
To use a variable for the agent token, the line should resemble the following:
```
echo "{\"tokens\": {\"accesstoken\":\"${LW_ACCESS_TOKEN}\"}}" > /var/lib/lacework/config/config.json     
```

Below are examples for Alpine Linux and Ubuntu. In general, when modifying for a particular OS, verify that the commands in the Dockerfile reflect the commands available for the respective OS being utilized.
Alpine Linux

FROM lacework/datacollector:latest-sidecar as laceworkagent
FROM Alpine:3.12
COPY --from=laceworkagent /var/lib/lacework-backup /var/lib/lacework-backup
RUN apk update && \
apk add bind-tools bash bash-doc bash-completion curl gzip jq
# Copy in application workload and run script
COPY run.sh /usr/bin/run.sh
# Create LW config directory and write LW Agent token to config file
RUN chmod +x /usr/bin/run.sh && \
mkdir -p /var/lib/lacework/config && \
echo '{"tokens": {"accesstoken":"Lacework Agent Token"}}' > /var/lib/lacework/config/config.json
CMD ["sh", "-c", "/usr/bin/run.sh"]

Ubuntu

FROM lacework/datacollector:latest-sidecar as laceworkagent
FROM ubuntu:18.04
COPY --from=laceworkagent /var/lib/lacework-backup /var/lib/lacework-backup
RUN apt-get update && \
apt-get install dnsutils curl gzip jq -y && \
apt-get clean
# Copy in application workload and run script
COPY run.sh /usr/bin/run.sh
# Create LW config directory and write LW Agent token to config file
RUN chmod +x /usr/bin/run.sh && \
mkdir -p /var/lib/lacework/config && \
echo '{"tokens": {"accesstoken":"Lacework Agent Token"}}' > /var/lib/lacework/config/config.json
CMD /usr/bin/run.sh

Prepare the run.sh file, or insert into your service startup task.
This is where the Lacework agent datacollector will be located. This should not be changed.
```
#!/bin/bash
/var/lib/lacework-backup/lacework-sidecar.sh
sleep 10s
head /var/log/lacework/datacollector.log
tail -f /dev/null   
```
Build the image.
- useYour/tag - Name the docker image tag
- /path/to/Dockerfile - Path to your Dockerfile
```
docker build --force-rm=true -t useYour/tag /path/to/Dockerfile   
```
When the build is complete, save the following output.
- Successfully built yourImageId
Update the tag on the docker image created in step 4.
- repository URI - Existing one or created in prerequisites: ECR Repository URI.
  NOTE: This URI must match the Repository URI where the image is being pushed to.
- yourImageId - Your image ID saved in step 5
- yourAWSacctID - Your AWS account ID
- clustername - Your cluster name
  Important: This does not need to exist to complete this step. However, you will not be able to proceed to step 7, until this points to a cluster. See Prepare Fargate before proceeding, if one doesn’t exist. Continue here when done.
```
docker tag yourImageId repository URI   
```
Push the just built docker image to the ECR.
- The repo needs to exist before completing this step. See the Prerequisites setup section for more information.
  - repository URI - Existing one or created in prerequisites: ECR Repository URI.
    NOTE: This URI must match the Repository URI where the image is being pushed to.
```
docker push repository URI   
```
    Tip: Compare the sha256 hash of the image in the ECR and the local image pushed in this step. They should be identical. See Verify Image Accuracy.

Prepare Fargate

If this is a new Fargate environment, use this command to create a new cluster, otherwise skip to step 2.
1. cluster-name - Replace with a custom name
```
aws ecs create-cluster --cluster-name cluster-name      
```
2. Save the cluster arn

Prepare fargate-task.json using this template. This will be used in the next step to generate the task.
Replace the following values:

yourAWSacctID - Your AWS ID
logGroupName - Your own name
region - Set the correct region where the Fargate environment is located (i.e. us-west-1)
"awslogs-stream-prefix": "ecs" - Optional: Replace with a custom prefix
imageName - This will be the name of the Lacework agent image
containerName - Desired name for the Lacework agent container

familyName - This will be the task family name Template fargate-task.json

{
"executionRoleArn": "arn:aws:iam::yourAWSacctID:role/ecsInstanceRole",
"taskRoleArn": "arn:aws:iam::yourAWSacctID:role/ecsTaskExecutionRole", 
"containerDefinitions": [{
   "logConfiguration": {
       "logDriver": "awslogs",
       "options": {
           "awslogs-group": "/ecs/logGroupName",
           "awslogs-region": "region",
           "awslogs-stream-prefix": "ecs"
       }
   },
   "cpu": 0,
"image": "yourAWSacctID.dkr.ecr.us-east-1.amazonaws.com/imageName:latest",
   "essential": true,
   "name": "containerName"
}],
"memory": "1GB",
"family": "familyName",
"requiresCompatibilities": [
   "FARGATE"
],
"networkMode": "awsvpc",
"cpu": "256"
}

Register fargate task definition.
NOTE: You will not be able to continue until the Lacework Agent Replica is running.
```
aws ecs register-task-definition --cli-input-json file://path/to/fargate-task.json   
```
Tip: Verify the task definition was created in Amazon ECS.
Create a service using the new task definition.
- clusterName - Cluster name created in step 1 of Prepare Fargate
- serviceName - Name for the Agent Replica service
- taskDefinition:1 - Select the definition set in steps 2 and 3 in Prepare Fargate
  NOTE: If you’ve made revisions to your task definition, make sure to set the correct revision number by replacing :1 with the revision to use. Otherwise leave the default of 1
- subnet-y,subnet-x - Replace with the subnets this to be serviced by the Lacework Agent Replica. Separate multiple entries with a comma. See step 6 of Create a VPC
- sg-x - Replace with the security group for this service. See Creating a Security Group
```
aws ecs create-service --cluster clusterName --service-name serviceName --task-definition taskDefinition:1 --desired-count 1 --launch-type "FARGATE" --network-configuration "awsvpcConfiguration={subnets=[subnet-y,subnet-x],securityGroups=[sg-x],assignPublicIp=ENABLED}"   
```
  Tip: Verify Cluster Service and Task Created
Go to Amazon ECS.
1. Click Clusters.
2. Click the cluster name created in step 1 of Prepare Fargate.
3. Click the Tasks tab.
4. The task created in step 4 should be RUNNING.
  
  Tip: Verify the Agent Appears in Lacework.

Prerequisite Setup

Amazon ECR

An ECR repository is required to store and run the Lacework agent image. Follow these steps to create a repo if one does not exist. If using an existing repo, skip to the end to locate the needed information.

In Amazon ECR, Repositories > Create repository.
Enter the Repository name and click Create repository.

Save: ECR Repository URI

yourAWSacctID.dkr.ecr.us-west-1.amazonaws.com/reponame

Existing Repository URI

Go to Amazon ECS

Click Repositories.
Click the icon left of the URI to copy it to the clipboard.

Create a VPC

Go to AWS VPC Dashboard.
Click VPCs > Create VPC.
On the setup screen, set the desired values.
Click Create VPC.
Go to AWS VPC Dashboard.
Click Subnets > Create subnet.
Set the desired values.
VPC must be the same one created in Create a VPC.
Click Create.
Save the Subnet ID for step 4 of Prepare Fargate.
Click to select the VPC.
Click Actions, Edit DNS hostnames, click the checkbox next to Enable, click Save changes.

Tip: See Verify Subnet Attached to VPC

Get Existing VPC Info

Go to AWS VPC Dashboard.
Click VPCs.
Get the Name and VPC ID values for the VPC being used for this setup.

Creating a Security Group

Go to AWS EC2 > Security Groups.
Click Create security group.
1. Security group name: Set the name.
2. Description: Set the description.
3. VPC: Select the VPC created in Create a VPC.
4. Click Create security group.
Save the Security group ID for step 4 of Prepare Fargate.

Troubleshooting

List Task Definitions

aws ecs list-task-definitions

List Services

aws ecs list-services --cluster clusterName

List Images on Local System

docker images

Describe Running Services

aws ecs describe-services --cluster clusterName --services serviceName

List Images in AWS ECR

docker images yourAWSacctID.dkr.ecr.us-east-1.amazonaws.com/repoName

List Service in AWS ECS

aws ecs list-services --cluster clusterName

Lacework Agent Does Not Appear in the Lacework UI

To test the Lacework Datacollector service: 1. Copy the imageID of the local container used for the setup 2. docker run -it imageID 3. Open another shell session 4. docker run -it imageID bash 5. ps -aux * datacollector should be running. * Run: service datacollector status * If error: datacollector: unrecognized service continue to step 6 * No error: continue to step 8 6. Delete all the local images created, force remove if locked, go to Prepare Docker…, review the Dockerfile and run.sh created, fix any typos and build the image again 7. Repeat steps 1 - 5. When the datacollector service is running, continue to step 8 * You can check the log with tail -f /var/log/lacework/datacollector.log 8. Repeat the docker tag and push steps to push the working image to your repo 9. Go to Create New Revision... to create a new task definition revision 10. Run the same command in step 4 from Prepare Fargate * taskDefinition:1 - Set this to the correct revision by replacing :1 with the revision to use

Get a bash Shell in Your Container

docker run -it imageID bash

Force Remove Docker Images

sudo docker rmi -f imageID

Verify Image Accuracy

Amazon ECR

Click Repositories
Click the repository name to show images
Click the icon left of Copy digest to copy the value to the clipboard
Paste the value into your favorite text editor

Local System

docker images --digests | grep repo name from URI
repository URI - Existing one or created in prerequisites: ECR Repository URI.
Copy/paste the sha256 value into your favorite text editor...Hopefully the same one from step 4, or this won’t work.

Review the following if these do not match
- ECR Repository
- Prepare Docker Image for Lacework Agent Replica Service, Step 6 and 7

Verify Task Definition Created

Go to Amazon ECS.
Click Task Definitions.
The task definition set for family in Step 2 of Prepare Fargate and registered in Register Fargate Task Definition should exist as a Task Definition.
Click the task name to access the configuration to compare values.
- Task Definition Name: Should be the same as the task definition
- Task Role: The AWS managed role assigned
- Task execution role: The AWS managed role assigned
- Container Name: The same name set in fargate-task.json or if changed, check the revision name being used
- Image: The tag you set for the agent image before pushing to ECR
- awslogs-group: The name you set for the log group in fargate-task.json.

Verify Subnet Attached to VPC

Go to AWS VPC Dashboard.
Click VPCs and select your VPC.
Click the link for the Main network ACL.
The Subnet ID should match the subnet ID created in step 6 of Create a VPC.

Verify Cluster Service and Task Created

Go to Amazon ECS.
Click Clusters and the Service tab.
Click the cluster name created in step 1 of Prepare Fargate.
The service created in steps 3 and 4 in Register fargate task definition should appear here.
Click the Tasks tab.
You should see the task that was started.
The Status should be RUNNING.
1. If it’s not, click the task link and check for error messages. Click the refresh icon if the status is still in a pending state.
2. If it fails, make sure the attached role has the correct trust configuration, check for an error message and see step 6 of Task Fails to Start for common reasons and solutions.

Task Fails to Start

Cluster Service Failing to start Tasks - Role Trust Configuration

Go to Amazon ECS.
Click Clusters.
Click the cluster name created in step 1 of Prepare Fargate.
Click the service created in steps 3 and 4 in Register fargate task definition.
Click Events to check Event messages.
- Common reason for this to fail is if the IAM Role Policy is not set correctly
Click Task Definitions.
Click the task and edit the definition created in steps 2 and 3 in Prepare Fargate.
There are 2 roles attached. Repeat steps 9 - 12 for both roles.
- Task role: ecsInstanceRole
- Task execution role: ecsTaskExecutionRole
Click ecsInstanceRole and make sure the following AWS managed policy is attached.
- AmazonEC2ContainerServiceforEC2Role
- If it is not, click Attach policies and attach it.
- Click Trust relationships and make sure the following provider is there.
- ecs-tasks.amazonaws.com
- If not, click Edit trust relationship
- Add the following under the Service section in the JSON, "ecs-tasks.amazonaws.com",
- Click Update Trust Policy.

Cluster Service Failing to start Tasks - VPC Communication, SG, Route Tables

Go to Amazon ECS.
Click Clusters.
Click the cluster name created in step 1 of Prepare Fargate.
Click the Tasks tab.
Click the task id link for the Task.
- Common status error: STOPPED (CannotPullContainerError: Error response from daem)
Common reasons:
- The VPC does not have an Internet Gateway attached
  1. Go to AWS VPC Dashboard, Internet Gateways, click Create Internet gateway.
  2. Enter a name and click Create internet gateway.
  3. Click Actions > Attach to VPC.
  4. Available VPCs: Select the VPC used for Fargate and click Attach internet gateway.
  5. The State should show Attached.
- The task does not have Auto-assign public IP set as Enabled
- Security Group doesn’t have the correct inbound ports open
  1. Go to EC2 Dashboard, Security Groups, select the Security Group.
  2. Click Inbound rules > Edit inbound rules.
  3. Type: HTTP
  4. Description: Optional
  5. Source: Custom
  6. CIDR: 0.0.0.0/0
  7. Click Save rules.
- Internet Gateway is not attached to the Subnet
  1. Go to AWS VPC Dashboard, Subnets, click the subnet.
  2. Click Route Table.
  3. There should be 2 routes
  4. Local CIDR:local
  5. 0.0.0.0/0:gateway name (This needs to be added)
  6. Click the link for the Route Table.
  7. Click Routes, Edit routes, Add route.
  8. Click Destination and select 0.0.0.0/0.
  9. Click the drop-down under Target and select Internet Gateway.
  10. Select the gateway that is attached to the VPC and click Save routes.
  11. Check the Route Table in step iv above. It should have both entries.
- Cannot Pull Container because the repository is not found
  - The status reason will provide the repository name that is not found. Note the name.
  - The repository name must match what was set for imageName in steps 2 and 3 in Prepare Fargate
  - If it doesn’t match, create a matching repository and repeat the docker tag and push steps to push the agent image to the correct repository.
  - Or repeat the docker tag and push steps to point to a repository that exists.
  - Create New Revision of Task Definition to point the task to the updated repository.
  - Go to Amazon ECS.
  - Click Task Definitions.
  - Click the task definition set for family in Step 2 of Prepare Fargate and registered in Register Fargate Task Definition and click Create new revision.
  - Scroll down to Container Definitions and delete the container with the unwanted image by clicking the X icon on the right.
  - Click Add Container.
    1. Container name: Enter name to use.
    2. Image: Enter the Image URI.
    3. Click Add.
  - Click Create.

Create a Fargate Task for the Lacework Agent Replica

The task should be created as part of steps 3 and 4 in Register fargate task definition. This should only be required for troubleshooting. Try to Create New Revision of Task Definition, before creating a new task.

Location: Amazon ECS

Click Clusters.
Click the cluster name created in step 1 of Prepare Fargate.
Click Tasks > Run new Task.
1. Launch type: FARGATE.
2. Task Definition: Select family set in Step 2 of Prepare Fargate.
3. Cluster VPC: Select VPC. See Create a VPC or Get Existing VPC.
4. Subnets: Select subnet(s) created in step 6 of Create a VPC or get existing subnet(s) info from Verify Subnet Attached to VPC.
5. Security groups: Select the security group or see Creating a Security Group.
6. Auto-assign public IP: ENABLED
7. Click Run Task.

Network Interface Does Not Have a Private DNS (IPv4)

Go to Amazon ECS.
Click Clusters.
Click the cluster name created in step 1 of Prepare Fargate.
Click the Tasks tab.
Click the link ID under Task for the running agent replica task.
Locate the Network section and click the link for the ENI Id.
Check Private DNS (IPv4) under the Details section. It should not be empty.
Go to AWS VPC Dashboard, Your VPCs, click to select the VPC.
Click Actions, Edit DNS hostnames, click the checkbox next to Enable, click Save changes.

Verify the Agent Appears in Lacework

Go to Amazon ECS.
Click Clusters.
Click the cluster name created in step 1 of Prepare Fargate.
Click the Tasks tab.
Click the link ID under Task for the running agent replica task.
Locate the Private IP, open your Lacework UI, and go to Monitor > Agents.
Open the pre-canned time selector and click Today.
1. Repeat this to refresh the Agent Monitor every few minutes.
2. After about 5 to 15 minutes, the agent will show up.
You can compare some details with AWS to validate this is the same agent.
- IP Address in Lacework and the Private IP noted in step 6 will be the same
- Hostname: This is the same as the Private DNS (IPv4) of the network interface
- lw_cluster: This will have the same name as the cluster you used for the setup

Fargate Information in the Lacework Console

Fargate Task Information

To view Fargate task information, navigate to Monitor > Agents then the Agent Monitor table. Note that the displayed hostname is not an actual host but rather an attribute (taskarn_containerID). Expand the hostname to display its tags, which contain Fargate task information, additional metadata collected by the agent.

This tag provides the Fargate engine version: VmInstanceType:AWS_ECSVxFARGATE

Where x can be 3 or 4 (1.3 or 1.4 engine respectively).

The following tags are prepended by net.lacework.aws.fargate.

Tag	Description
cluster	The cluster where the task was started
family	The task definition family
pullstartedat	The time the container pull started
pullstoppedat	The time the container pull stopped. The diff from pullstartedat represents how long the pull took to start.
revision	The task revision
taskarn	The full task ARN

You can also view the above tag information at Host > Applications in the List of Active Containers table. Make the Machine Tags column visible; it is hidden by default.

Fargate Container Information

You can find container information at Host > Applications in the List of Active Container table. The container ID is available, but AWS currently does not expose underlying infrastructure. To view tags, make the Machine Tags column visible; it is hidden by default.

The containers listed in Container Image Information are application containers. If you used the sidecar deployment method, the sidecar container itself is not displayed in this table because it is not running and has no runtime or cost associated with it. The Lacework application, however, runs as a process inside your application container, so it is visible in the Applications Information table (search for datacollector) because there is no running container for the Lacework agent.

Additional Notes

This section provides additional clarification about Lacework behavior with Fargate deployments.

FIM

By default Lacework does not perform file integrity checks within containers. This means that for a Fargate-only environment, there is no information under the Host > Files (FIM) menu. In an environment containing Fargate and non-Fargate deployments, the expected information displays. The checks are not performed by default because they are expensive for CPU and memory.

Event Triage

Lacework has the following methods for accessing Lacework events:

Alert channels such as Email, Jira, Slack
Lacework Console: Monitor > Events

Alert Channels

The following figure illustrates an email alert channel with 2 medium alerts.

Click View Details → to load the Event Details page in your default browser.

Lacework Console: Monitor > Events

Click Monitor > Events and set the time frame for your inspection.

A summary of the currently selected event is available in the Event Summary in the upper right quadrant of the page.
This page presents the following major launch points:

Event Summary—You can review high level event information. In the above figure, the description contains a destination host, a destination port, an application, and a source host. Each of these links provides a resource-specific view. For example, if you click the destination host, you can view all activity for that destination host seen by the Lacework Platform.
Timeline > Selected Event > Details—This opens the Event Details page. The Event Details page is the same page that opens when you click the View Details link from an alert channel.

Triage Example

The following exercise focuses on the Event Details, which is illustrated in the following figure.

On this page you can view the 5W structure for Lacework events.

Key Data Points to Triage a Fargate Event

WHAT

Expand the details for WHAT.

Header Row

Key	Value Description	Notes
APPLICATION	What is running	None
MACHINE	The Fargate ARN:task for the job	Machine-centric view
CONTAINER	The container the task is running	None

Application Row

Key	Value Description	Notes
APPLICATION	What is running	None
EARLIEST KNOWN TIME	First time an application was seen	Useful for lining up incident responses with timelines in non-Lacework services.

Container

Key	Value Description	Notes
FIRST SEEN TIME	First time a container was seen	Useful for lining up incident responses with timelines in non-Lacework services. Useful for spot checks of your expected container lifecycle versus what is running.

WHERE Expand the details for WHERE. There is a WHERE section if the event is network related.

Key	Value Description	Notes
HOSTNAME	Pivot to inspect the occurrence of the host activity within your environment	None
PORT LIST	Ports activity was seen over	Context. Do you expect this service to connect over these ports? Are there any ports that are interesting, such as non standard ports for servers.
IN/OUT BYTES	Data transmitted/received	Inferring risk based on payload size.

Investigation High level situational context that relates to the application deployment and your knowledge of its expected behavior.

Was the application involved in the event from Packaged software? In this case, the event is for nginx and tomcat. If you install tomcat / nginx using the package manager, this is marked as yes. If you install tomcat / nginx using the package manager and this is marked no, investigate if there was a change to how you build your container. If there is no change to your build process, investigate how a non packaged version was installed.

Has the Hash(SHA256) of the application been involved in an event change? Was the application involved altered / replaced. If so, check your build and upgrade process. It is typical that containers will not see application updates. A change here warrants continued investigations to determine the cause of change.

Has the application involved in the event been running as Root? Indicator of the scope of privilege the application has. If running as root is not part of your build processes, check for an alteration to your build processes and privilege escalation.

Has the application transferred more than the median amount of data compared to the last day?
Consider the amount of data transferred versus what is expected by the application.

Related Events Timeline The timeline is useful for building context for events around the Event Details you are currently investigating.

Summary

Using the above data, you should be able to create the following triage process.

Determine the scope of the event:

Containers
Applications
Hosts
arn:task

Drill down into the event relationships to get context across your infrastructure secured by the Lacework platform.

Determine what is expected as a part of your build processes and what is unexpected.

From this path, continue to narrow down if there was a change to build processes or a net new build process or infrastructure event/service. For these scenarios, focus on updating application, build, and network configurations.

For scenarios where there is a risk of compromise, focus on the timeline that led to the potential compromise, investigate the entities involved, and create a structured story that enables rapid IR and processes updates.

Agent Upgrades

For Docker versions 17.05 and later with multi-stage builds, the latest agent release is used anytime you revision to a newer version of the application container. The runtime upgrade for the agent is available as well.

For Docker versions pre-17.05, when the agent is running, it always checks for the current version and downloads the latest. The version and git hash are available on the public github agent release page and also encoded in the S3 path.

Sidecar-based Deployment

NOTE: The recommended deployment method is documented in Fargate and ECR Setup.

The sidecar deployment method allows you to monitor AWS Fargate containers without changing underlying container images. This deployment method leverages AWS Fargate’s VolumesFrom capabilities to map a predefined Lacework container image to be run alongside your production workloads.

Predeployment

Before attempting to publish a container to ECR, ensure you have the following items and information:

Configure the ECS task execution role to include AmazonECSTaskExecutionRolePolicy.

{  
 "Version": "2012-10-17",  
 "Statement": [  
   {  
     "Effect": "Allow",  
     "Action": [  
       "ecr:GetAuthorizationToken",  
       "ecr:BatchCheckLayerAvailability",  
       "ecr:GetDownloadUrlForLayer",  
       "ecr:BatchGetImage",  
       "logs:CreateLogStream",  
       "logs:PutLogEvents"  
     ],  
     "Resource": "*"  
   }  
 ]  
}

If it is not already configured, you may need to include some of the ECS access policies as well, such as AmazonECS_FullAccess.

Agent Upgrades

When running, the agent always checks for the current version and downloads the latest. The sidecar container always points to the latest in hub.docker.com, for example, lacework/datacollector:latest-sidecar.

Task Definition

A task definition describes the containerized workloads that you want to run.

Create a new task definition.
Add task details.
Set the task execution role.
Set memory and CPU requirements for the application.

Sidecar Container Definition

Click Add container to add the agent sidecar container.
In the Environment section, deselect the Essential checkbox. This is not an essential container.
You do not need to change any fields in the remaining sections.
Click Add to finish adding this sidecar container.
Container Definitions now lists the new container.

Application Container Definition

Click Add container to add the application container.
Ensure the application container has a different name than the Lacework agent sidecar (datacollector) container.
1. In the Environment section, ensure Essential is selected.
2. Before continuing, you must identify whether the container uses ENTRYPOINT and/or CMD. Ideally, you could examine the Dockerfile used to build the container to identify ENTRYPOINT and/or CMD usage in building the container. Alternatively, you must use tools to inspect the Docker container image to identify ENTRYPOINT and/or CMD usage.
  For example, using the Docker Hub interface, you can inspect the container image layers to identify the usage.
  The following figure illustrates an Ubuntu container using the CMD directive.
  
  The following figure illustrates an Nginx container using the ENTRYPOINT and CMD directive.
After identifying whether the container uses ENTRYPOINT and/or CMD, place an appropriate definition in the ENTRYPOINT or CMD container definition section of ENVIRONMENT. Depending on the application container, in order to sidecar the Lacework agent in the application, place the following definition.
```
sh, -c, /var/lib/lacework-backup/lacework-sidecar.sh && APPLICATION_INIT   
```
NOTE: By definition, ENTRYPOINT + CMD = default container command with arguments. It is possible that the override of ENTRYPOINT + CMD may not provide the desired outcome. If the application container uses ENTRYPOINT, it is recommended to use ENTRYPOINT override with the Lacework sidecar because it takes precedence over CMD override.
The following figure illustrates an example that overrides the application container’s CMD section while using /usr/bin/runapp.sh as the startup wrapper for the application container.
In the Startup Dependency Ordering section, ensure the application container has the dependency on the Lacework agent sidecar container so that the application container only starts after the sidecar container successfully starts.
In the Storage and Logging section, ensure the application container imports the volume that is exported by the Lacework agent sidecar container. This makes the volume that contains the Lacework agent executable available in the application container.

For Volumes from Source container, add the sidecar container name. Also select the Read only checkbox.
Click Add to finish this application container. Container Definitions now lists both the sidecar and application containers.
Click Create to create the task.

JSON Task Definition

The sidecar container is available in hub.docker.com as lacework/datacollector:latest-sidecar.

    {  
        "requiresCompatibilities": [  
            "FARGATE"  
        ],  
        "inferenceAccelerators": [],  
        "containerDefinitions": [  
            {  
                "name": "datacollector-sidecar",  
                "image": "YourAWSId.dkr.ecr.us-west-2.amazonaws.com/datacollector:latest-sidecar",  
                "resourceRequirements": null,  
                "essential": false,  
                "portMappings": [],  
                "environment": null,  
                "secrets": null,  
                "mountPoints": null,  
                "volumesFrom": null,  
                "hostname": null,  
                "user": null,  
                "workingDirectory": null,  
                "extraHosts": null,  
                "logConfiguration": {  
                    "logDriver": "awslogs",  
                    "options": {  
                        "awslogs-group": "/ecs/TestSideCarTaskDef",  
                        "awslogs-region": "us-west-2",  
                        "awslogs-stream-prefix": "ecs"  
                    }  
                },  
                "ulimits": null,  
                "dockerLabels": null,  
                "dependsOn": null,  
                "repositoryCredentials": {  
                    "credentialsParameter": ""  
                }  
            },  
            {  
                "name": "app-container",  
                "image": "YourAWSId.dkr.ecr.us-west-2.amazonaws.com/datacollector-sidecar-app:latest",  
                "resourceRequirements": null,  
                "essential": true,  
                "portMappings": [],  
                "command": [  
                    "sh",  
                    "-c",  
                    "/var/lib/lacework-backup/lacework-sidecar.sh && /usr/bin/runapp.sh"  
                ],  
                "environment": [  
                    {  
                        "name": "LaceworkAccessToken",  
                        "value": "YourAccessToken"  
                    }  
                ],  
                "secrets": null,  
                "mountPoints": null,  
                "volumesFrom": [  
                    {  
                        "sourceContainer": "datacollector-sidecar",  
                        "readOnly": true  
                    }  
                ],  
                "hostname": null,  
                "user": null,  
                "workingDirectory": null,  
                "extraHosts": null,  
                "logConfiguration": {  
                    "logDriver": "awslogs",  
                    "options": {  
                        "awslogs-group": "/ecs/TestSideCarTaskDef",  
                        "awslogs-region": "us-west-2",  
                        "awslogs-stream-prefix": "ecs"  
                    }  
                },  
                "ulimits": null,  
                "dockerLabels": {  
                    "Monitoring": "Lacework"  
                },  
                "dependsOn": [  
                    {  
                        "containerName": "datacollector-sidecar",  
                        "condition": "SUCCESS"  
                    }  
                ],  
                "repositoryCredentials": {  
                    "credentialsParameter": ""  
                }  
            }  
        ],  
        "volumes": [],  
        "networkMode": "awsvpc",  
        "memory": "1024",  
        "cpu": "512",  
        "executionRoleArn": "arn:aws:iam::YourAWSId:role/ecsTaskExecutionRoleTest",  
        "family": "TestSideCarTaskDef",  
        "taskRoleArn": "arn:aws:iam::YourAWSId:role/ecsTaskExecutionRoleTest",  
        "tags": [  
            {  
                "key": "AppContainerSecurity",  
                "value": "Lacework"  
            }  
        ]  
    }

Full JSON Task Definition with IAM Role

    {  
      "ipcMode": null,  
      "executionRoleArn": "arn:aws:iam::YourAWSId:role/ecsTaskExecutionRoleTest",  
      "containerDefinitions": [  
        {  
          "dnsSearchDomains": null,  
          "environmentFiles": null,  
          "logConfiguration": {  
            "logDriver": "awslogs",  
            "secretOptions": null,  
            "options": {  
              "awslogs-group": "/ecs/TestSideCarTaskDef",  
              "awslogs-region": "us-west-2",  
              "awslogs-stream-prefix": "ecs"  
            }  
          },  
          "entryPoint": null,  
          "portMappings": [],  
          "command": null,  
          "linuxParameters": null,  
          "cpu": 0,  
          "environment": [],  
          "resourceRequirements": null,  
          "ulimits": null,  
          "dnsServers": null,  
          "mountPoints": [],  
          "workingDirectory": null,  
          "secrets": null,  
          "dockerSecurityOptions": null,  
          "memory": null,  
          "memoryReservation": null,  
          "volumesFrom": [],  
          "stopTimeout": null,  
          "image": "YourAWSId.dkr.ecr.us-west-2.amazonaws.com/datacollector:latest-sidecar",  
          "startTimeout": null,  
          "firelensConfiguration": null,  
          "dependsOn": null,  
          "disableNetworking": null,  
          "interactive": null,  
          "healthCheck": null,  
          "essential": false,  
          "links": null,  
          "hostname": null,  
          "extraHosts": null,  
          "pseudoTerminal": null,  
          "user": null,  
          "readonlyRootFilesystem": null,  
          "dockerLabels": null,  
          "systemControls": null,  
          "privileged": null,  
          "name": "datacollector-sidecar"  
        },  
        {  
          "dnsSearchDomains": null,  
          "environmentFiles": null,  
          "logConfiguration": {  
            "logDriver": "awslogs",  
            "secretOptions": null,  
            "options": {  
              "awslogs-group": "/ecs/TestSideCarTaskDef",  
              "awslogs-region": "us-west-2",  
              "awslogs-stream-prefix": "ecs"  
            }  
          },  
          "entryPoint": null,  
          "portMappings": [],  
          "command": [  
            "sh",  
            "-c",  
            "/var/lib/lacework-backup/lacework-sidecar.sh && /usr/bin/runapp.sh"  
          ],  
          "linuxParameters": null,  
          "cpu": 0,  
          "environment": [  
            {  
              "name": "LaceworkAccessToken",  
              "value": "YourAccessToken"  
            }  
          ],  
          "resourceRequirements": null,  
          "ulimits": null,  
          "dnsServers": null,  
          "mountPoints": [],  
          "workingDirectory": null,  
          "secrets": null,  
          "dockerSecurityOptions": null,  
          "memory": null,  
          "memoryReservation": null,  
          "volumesFrom": [  
            {  
              "sourceContainer": "datacollector-sidecar",  
              "readOnly": true  
            }  
          ],  
          "stopTimeout": null,  
          "image": "YourAWSId.dkr.ecr.us-west-2.amazonaws.com/datacollector-sidecar-app:latest",  
          "startTimeout": null,  
          "firelensConfiguration": null,  
          "dependsOn": [  
            {  
              "containerName": "datacollector-sidecar",  
              "condition": "SUCCESS"  
            }  
          ],  
          "disableNetworking": null,  
          "interactive": null,  
          "healthCheck": null,  
          "essential": true,  
          "links": null,  
          "hostname": null,  
          "extraHosts": null,  
          "pseudoTerminal": null,  
          "user": null,  
          "readonlyRootFilesystem": null,  
          "dockerLabels": null,  
          "systemControls": null,  
          "privileged": null,  
          "name": "app-container"  
        }  
      ],  
      "placementConstraints": [],  
      "memory": "1024",  
      "taskRoleArn": "arn:aws:iam::YourAWSId:role/ecsTaskExecutionRoleTest",  
      "compatibilities": [  
        "EC2",  
        "FARGATE"  
      ],  
      "taskDefinitionArn": "arn:aws:ecs:us-west-2:YourAWSId:task-definition/TestSideCarTaskDef:1",  
      "family": "TestSideCarTaskDef",  
      "requiresAttributes": [  
        {  
          "targetId": null,  
          "targetType": null,  
          "value": null,  
          "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"  
        },  
        {  
          "targetId": null,  
          "targetType": null,  
          "value": null,  
          "name": "ecs.capability.execution-role-awslogs"  
        },  
        {  
          "targetId": null,  
          "targetType": null,  
          "value": null,  
          "name": "com.amazonaws.ecs.capability.ecr-auth"  
        },  
        {  
          "targetId": null,  
          "targetType": null,  
          "value": null,  
          "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"  
        },  
        {  
          "targetId": null,  
          "targetType": null,  
          "value": null,  
          "name": "com.amazonaws.ecs.capability.task-iam-role"  
        },  
        {  
          "targetId": null,  
          "targetType": null,  
          "value": null,  
          "name": "ecs.capability.container-ordering"  
        },  
        {  
          "targetId": null,  
          "targetType": null,  
          "value": null,  
          "name": "ecs.capability.execution-role-ecr-pull"  
        },  
        {  
          "targetId": null,  
          "targetType": null,  
          "value": null,  
          "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18"  
        },  
        {  
          "targetId": null,  
          "targetType": null,  
          "value": null,  
          "name": "ecs.capability.task-eni"  
        }  
      ],  
      "pidMode": null,  
      "requiresCompatibilities": [  
        "FARGATE"  
      ],  
      "networkMode": "awsvpc",  
      "cpu": "512",  
      "revision": 1,  
      "status": "ACTIVE",  
      "inferenceAccelerators": null,  
      "proxyConfiguration": null,  
      "volumes": []  
    }

Run Task

Select Fargate as the Launch type.
Select your Platform version.

Articles in this section

Overview

Support Information

Access Tokens

Fargate and ECR Setup

AWS Services Used

Gather Required Information

Prerequisite Setup

Troubleshooting

Fargate Information in the Lacework Console

Fargate Task Information

Fargate Container Information

Additional Notes

FIM

Event Triage

Alert Channels

Lacework Console: Monitor > Events

Triage Example

Key Data Points to Triage a Fargate Event

Summary

Agent Upgrades

Sidecar-based Deployment

Predeployment

Agent Upgrades

Task Definition

Sidecar Container Definition

Application Container Definition

JSON Task Definition

Full JSON Task Definition with IAM Role

Run Task