Skip to main content

AWS - Integrate Agentless Workload Scanning with CloudFormation

This integration method uses AWS CloudFormation.

Overview

This article provides the prerequisites and troubleshooting steps for an Agentless Workload Scanning integration.

Once you have read through the access and resource requirements, complete the integration steps depending on your chosen integration level:

Access and Resource Requirements

A new VPC and Internet Gateway will be created in each scanning region. This applies to both integration types. In the AWS Organization integration, only one account is set up to perform scanning (a scanning account or security account). This is the only account where a new VPC and Internet Gateway is created.

The target AWS account must have Service Quotas allowing at least one more of these resources to be created in each region selected. One way to verify is to use AWS Trusted Advisor, then the Service limits link on the left and search by keyword "VPC" then expand both VPC and VPC Internet Gateways search results. Make sure at least one more of each can be created in each scanning region.

CloudFormation StackSets are used in both the Single Account and Organization deployment methods. You must have two non-default AWS roles before StackSets can be used. The details are described on AWS's StackSet pre-requirements guide. This requirement can be verified by using the AWS IAM interface to check for the roles AWSCloudFormationStackSetAdministrationRole and AWSCloudFormationStackSetExecutionRole. If either of these roles do not exist then the Lacework CloudFormation template can create them for you by answering Yes to Deploy one-time AWS CloudFormation Self-Managed Permissions Roles?.

note

If Lacework deploys these roles then they will also be deleted if the CloudFormation stack is deleted.

The Amazon Elastic Compute Service (ECS) is used in both the Single Account and Organization deployment methods. In the AWS Organization integration, only one account is set up to perform scanning (a scanning account or security account). This is the only account where ECS is used. You must have the non-default AWS role AWSServiceRoleForECS created. If this role does not exist then the Lacework CloudFormation template can create it for you by answering Yes to Deploy one-time AWS ECS Service Linked Role?.

tip

Lacework automatically tries to detect whether the StackSet and ECS Service roles already exist and populate the CloudFormation template accordingly.

Alternatively, you can use our "Pre Flight" script prior to deployment to check requirements and recommend the correct selections for the CloudFormation template inputs.

Single Account: Access Requirements

  • Access to run CloudFormation Stacks. A StackSet will be run in each region selected.
  • Access to create ECS clusters, and access to create a VPC, subsets, and Internet Gateway for the ECS cluster.
  • IAM to create an ECS task execution role, task role, and an EventBridge role for starting ECS tasks.
  • IAM to create a cross-account role that has permissions to read from a newly created S3 bucket and start ECS tasks.
  • Access to create CloudWatch Log Groups and Streams.
  • Access to create a new S3 bucket.
  • Access to create a new secret in AWS Secrets Manager.

Organization: Access Requirements

Two separate CloudFormation StackSets will be run. One in a scanning account and another on the billing or main/top-level AWS account. There are access requirements specific to both of these.

The access requirements for the scanning account are the same as the requirements for the Single Account integration. See Single Account: Access requirements.

The access requirements for the top-level AWS account are:

  • Access to the organization APIs.
  • IAM to create a role to provide the scanning account the ability to list accounts in the organization.
  • Access to run a CloudFormation StackSet on each Organization Unit (OU) or account selected in the template.
  • Access to create an IAM role on each of the accounts mentioned above. This role will have access to create snapshots and optionally decrypt the content.
note

CloudFormation trusted access with AWS Organizations is required to create service-managed (or SERVICE_MANAGED) StackSets.

Troubleshooting

Installation

There are some known limitations:

  • The Lacework Query Language (LQL) query you specify for an integration is not validated. If an improper query is specified, the scanning will fail with the status "fail closed".
  • It is possible to create multiple agentless scanning integrations in the same region. If overlapping integrations are created, they are not optimized. This can result in hosts being snapshotted and scanned more than once.

CloudFormation

Errors in StackSets

When StackSets encounter an error then the root cause is found on the StackSets page.

For example, the following error is encountered as a CloudFormation event:

Resource handler returned message: "Stack set operation [STACK_ID] was unexpectedly stopped or failed"
(RequestToken: TOKEN_ID, HandlerErrorCode: InternalFailure)

This error can be debugged with the following steps:

  1. In the AWS Console open the CloudFormation page. Make sure you have selected the AWS region where the Agentless Scanning template was installed.
  2. On the left-hand side menu click StackSets.
  3. Click on the link for the StackSet matching the name of the CloudFormation Stack created.
  4. Click on the Stack Instances tab.
  5. Review each Instance and check that the Detailed Status is "Success", if there is an error then the Status Reason will provided a detailed error message.

This will display the root cause of the "Internal Failure" referenced above.

Service Quota reached for VPC or VPC Internet Gateway

The following error may indicate that Service Quotas have been reached for a region:

Embedded stack arn:aws:cloudformation:REGION:ACCOUNT_ID:stack/PREFIX-AgentlessScanRegionalStack-SUFFIX was not successfully created:
The following resource(s) failed to create: [AgentlessScanVPC, AgentlessScanInternetGateway].

The most likely root cause is that the maximum number of VPCs (20) or Internet Gateways (5) has been reached for a region. The solution is to request the soft quota limit to be increased through the Service Quotas tool.

note

Be aware that these requests may not be auto-approved and could take up to 24 hours.

Before starting the CloudFormation, the Trusted Advisor tool can be used to inspect Service limits for all regions.

Missing Service Linked Role (SLR) for Elastic Compute Service (ECS)

If this is the first time Elastic Compute Service (ECS) is used on your AWS account, the Service Linked Role (SLR) may not exist. The role name is AWSServiceRoleForECS and the AWS IAM console can be used to check existence. The CloudFormation template has an option to create this role (a one time operation per AWS account) called Deploy one-time AWS ECS Service Linked Role?. If this option is not selected (if "No" is input) and if no SLR exists for ECS, the CloudFormation will fail with the following error.

Resource handler returned message: "Invalid request provided: CreateCluster Invalid Request: Unable to assume the service linked role.
Please verify that the ECS service linked role exists.

Deleting Stacks fails to delete the S3 bucket

It is not possible to delete a non-empty S3 bucket. If scanning has run before a Stack is deleted, the S3 bucket might fail to delete because it contains scanning results. Note that there is a Lifecycle Policy to delete the content after a period.

Delete a Stack by performing the following steps:

  1. Disable the integration in the Lacework console.
  2. Note the value of the Bucket ARN in the integration details.
  3. Use the AWS console to find and view the associated S3 bucket content, select all, and choose Delete.
  4. Use the CloudFormation console to delete the Stack.

Deleting Stacks fails to delete Regional StackSet

This error may occur if scanning is active. This means an ECS Task is running and CloudFormation will refuse to delete the ECS resources until the task stops. The recommended fix is to disable or delete the Lacework integration and wait for 2 hours. This will assure all Tasks have stopped.

Alternatively, locate the region where the Task is running.

  1. Use the AWS console to open the Elastic Container Service page.
  2. Find the name of the cluster associated with this integration. These clusters typically have a "lacework-agentless-" prefix.
  3. Click the cluster name and then click the Tasks tab.
  4. Check the Stop all button.
  5. Use the CloudFormation console to delete the Stack.

Uncommon CloudFormation errors

Here are uncommon situations that lead to errors, and example error messages.

A trailing comma exists in the Regions input field
Properties validation failed for resource AgentlessScanRegionalStackSet with message:
#: #: only 1 subschema matches out of 2 #/StackInstancesGroup/0/Regions/2: failed validation constraint for keyword [pattern]

In this situation the Regions input field contained a trailing comma or other invalid character.

A duplicate region exists in the Regions input field
Properties validation failed for resource AgentlessScanRegionalStackSet with message:
#: #: only 1 subschema matches out of 2 #/StackInstancesGroup/0/Regions: array items are not unique

In this situation the Regions input field contained a duplicate region name.

An invalid region was supplied in the Regions input field
Resource handler returned message: "Region $name is not supported (Service: CloudFormation, Status Code: 400, Request ID: $id)"
(RequestToken: $token, HandlerErrorCode: InvalidRequest)

In this situation the Regions input field contained an unknown or invalid region name.

Incorrect AWS Account ID or Organizational Unit used with Org-scanning
Properties validation failed for resource AgentlessSnapshotRoleServiceManagedStackSet with message:
#: #: only 1 subschema matches out of 2 #/StackInstancesGroup/0/DeploymentTargets/OrganizationalUnitIds/0: failed validation for keyword [pattern].

In this situation the Monitored Account Deployment selection (either SELF_MANAGED or SERVICE_MANAGED) did not match the input for Monitored Account IDs.

CloudFormation trusted access is not enabled for AWS Organizations

If the Step 2 CloudFormation for AWS Organizations fails with the following error:

You must enable organizations access to operate a service managed stack set

Then trusted access for AWS Organizations has not been enabled. The CloudFormation StackSets page will show a prompt and button to enable trusted access. The CloudFormation Stack must be run again after trusted access is enabled.

The ECS Service Linked Role did not exist during deployment
ResourceLogicalId:AgentlessScanECSCluster, ResourceType:AWS::ECS::Cluster, ResourceStatusReason:Resource handler returned message:
"Invalid request provided: CreateCluster Invalid Request: Unable to assume the service linked role. Please verify that the ECS service linked role exists.
(Service: AmazonECS; Status Code: 400; Error Code: InvalidParameterException; Request ID: $id; Proxy: null)"
(RequestToken: $token, HandlerErrorCode: InvalidRequest).

In this situation the input question Deploy one-time AWS ECS Service Linked Role? was answered No and it should have been Yes. This failure occurs once since AWS will automatically create the role the first time an ECS API is used. However, this single error will still cause the StackSet to fail.

The solution is to update the StackSet.

Updating CloudFormation StackSets

If a StackSet failed and you remedied the issue out of band, then existing StackSets can be updated.

  1. In the AWS Console on the CloudFormation page, select StackSets on the left-hand side menu.
  2. Select the radio StackSet Name for the Agentless scanning CloudFormation Stack.
  3. Click Actions in the top-right and select Edit StackSet Details.
  4. For "Choose a template", use the default values and click Next.
  5. For "Specify StackSet details", use the default values and click Next.
  6. For "Configure StackSet options", use the default values and click Next.
  7. For "Set deployment options", input the Account numbers used in this StackSet. This is usually a single AWS account number and the same one being used to update the StackSet. Then select Add all regions, and click Next.
  8. Scroll to the bottom and click Submit.