Skip to main content

Pub/Sub-Based Google Cloud Audit Log Integration - Manual Configuration

This topic describes how to manually create a Lacework Google Cloud Pub/Sub-based audit log integration to gather, process, report, and alert on Google Cloud Cloud Audit Log data.

info

If you want to manually migrate an existing Storage-based audit log integration to a Pub/Sub-based audit log integration, see Migrate from Storage-based to Pub/Sub-Based Google Cloud Audit Log Integration - Manual Configuration.

note

You must create a separate Google Cloud integration for reporting and alerting on Google Cloud configuration compliance. For more information, see Google Cloud Configuration Integration - Manual Configuration. You can, however, use the same credentials for both integrations.

When a Google Cloud Audit Log integration is created, Lacework uses a Google Cloud Pub/Sub topic to gather the logs of administrative operations from Google Cloud audit logs. Google Cloud captures detailed audit log data and stores that data in audit logs. Lacework aggregates and organizes this audit log data into useful maps and dashboards that illustrate the following. Lacework also automatically generates alerts whenever an audit log event represents a security risk.

  • conceptual relationships
  • causes and effects
  • interactions between Google Cloud entities

Prerequisites​

  • Decide if you want to monitor the audit logs at the organization, folder, or project level. If you specify the option to include children when creating the sink at the organization or folder level, all the projects in the organization or the folder export the audit logs to the bucket.
  • If you are already exporting your audit logs to a Pub/Sub topic, you can reuse that Pub/Sub topic but the project with the Pub/Sub topic must be granted role access as described in the procedure below.
  • Google Cloud Console - Administrator access to Google Cloud Console to create projects, service accounts, grant access, and provision resources.
  • gcloud CLI - To configure resources in Google Cloud.
  • Lacework Console - Org admin or Account admin access to the Lacework Console is required to create API keys and configure integrations with Lacework.
  • jq utility
  • Ensure that you are deploying the integration to a supported Google Cloud region.

Create a Google Cloud Service Account and Grant Access​

Follow the procedure in Create a Google Cloud Service Account and Grant Access. You can skip this step if you have already created a service account and granted role access when creating a new Google Cloud configuration integration. You can also reuse the same service account.

Enable the Required Google Cloud APIs​

Enable APIs for the projects you want to integrate using the procedure in Enable the Required Google Cloud APIs.

Configure Google Cloud for Audit Log Integration​

This procedure configures the Google Cloud Audit Log integration to ingest audit log data.

  1. Open a terminal window and initialize the gcloud CLI with the following command: 

    gcloud init

    Leave the terminal open.

  2. Log in to the Google Cloud Console.

  3. Decide if you want to export audit logs at the organization, folder, or project level. In addition, select an entity to export audit logs from. An entity is either an organization, folder, or project. The entity must be granted access to the service account and must have the role access as described in Create a Google Cloud Service Account and Grant Access. A project can have access because it was directly granted access or because its parent organization was granted access. Billing must also be enabled for any projects that you want to export audit logs from. Note that if you select the option to include children when creating the sink at the organization or folder level, all projects in the organization or the folder export the audit logs to the Pub/Sub topic.

  4. Obtain the ID of the project where the Pub/Sub topic and sink will be created.

    • For an organization level integration, Lacework recommends that you create a new project. If you decide to export audit logs at the project level, this is the project that will export its audit logs.
    • For a project level integration, obtain the ID of the project being integrated.

  5. In the terminal, set the following variable. Replace my-first-project-654321 with the ID of the project where the Pub/Sub topic and sink will be created.

    projectName=my-first-project-654321

  6. Set the current project using the following command. 

    gcloud config set project "$projectName"

    If the Updated property [core/project] status is not displayed, make sure the quotes are straight double-quotes. Use straight double-quotes for all the variables used in CLI commands in this procedure.

  7. Create a Pub/Sub topic in the project. Replace mytopicname with your topic name.

    topicName=mytopicname
    gcloud pubsub topics create "$topicName"

  8. Set the name of sink. Replace mysink with your sink name.

    sinkName=mysink

  9. Create the sink. Decide if you want to export the audit logs at the organization, folder, or project level and create the sink. If creating the sink at the organization level, add \ --include-children --organization=myorganizationid to the end of the following command. If creating the sink at the folder level, add \ --include-children --folder=myfolderid to the end of the following command. If you specify the \ --include-children option when creating the sink at the organization or folder level, all the projects in the organization or the folder export audit logs to the Pub/Sub topic.

    gcloud logging sinks create "$sinkName" "pubsub.googleapis.com/projects/$projectName/topics/$topicName" --log-filter '(protoPayload.@type=type.googleapis.com/google.cloud.audit.AuditLog) AND NOT (protoPayload.serviceName="k8s.io") AND NOT (protoPayload.serviceName="login.googleapis.com") AND NOT (protoPayload.methodName="storage.objects")'

  10. Copy the service account returned by the previous command. Copy the text between the single quotes. In the following example, copy serviceAccount:123567@gcp-sa-logging.iam.gserviceaccount.com. 

    Please remember to grant `serviceAccount:123567@gcp-sa-logging.iam.gserviceaccount.com` the Pub/Sub Publisher role on the topic.

    The next two steps grant the roles/pubsub.publisher role on the Pub/Sub topic.

  11. Enter sinkServiceAccount=, paste the service account name returned by the gcloud logging sinks create command, then press Enter.

    sinkServiceAccount=serviceAccount:123567@gcp-sa-logging.iam.gserviceaccount.com

  12. Grant roles/pubsub.publisher permissions to the service account returned by the gcloud logging sinks create command.

    gcloud pubsub topics add-iam-policy-binding "$topicName" --member="$sinkServiceAccount" --role=roles/pubsub.publisher

  13. Create the Pub/Sub subscription (queue) for the Pub/Sub topic. Replace mytopicsub with your Pub/Sub subscription name.

    topicSubscriptionName=mytopicsub
    gcloud pubsub subscriptions create "$topicSubscriptionName" --topic "$topicName" --ack-deadline=300

  14. In the terminal, set the following variable. Replace my-service-account with the email ID of service account you created in your project. This is not the service account returned by the gcloud logging sinks create command above.

    The service account email ID is in the format: my-service-account@my-project-name.iam.gserviceaccount.com. Note that the string serviceAccount: must preface the service account email ID.

    serviceAccount=serviceAccount:my-service-account

  15. Grant the service account you created in your project the subscriber role on the Pub/Sub subscription. If prompted to install a command group, enter y.

    gcloud pubsub subscriptions add-iam-policy-binding "projects/$projectName/subscriptions/$topicSubscriptionName" --member="$serviceAccount" --role=roles/pubsub.subscriber

  16. Enter the following command to get the Pub/Sub subscription path that is used when creating the integration in the Lacework Console.

    echo "projects/$projectName/subscriptions/$topicSubscriptionName"

  17. Copy the resulting Pub/Sub subscription path returned by the echo command for use in the next procedure that creates the integration.

note

If you are setting up new Google audit logging (instead of leveraging existing Google audit logging), Lacework recommends that you set a retention policy with a minimum of 7 days.

Create the Google Cloud Audit Log Integration on the Lacework Console​

  1. Log in to the Lacework Console.
  2. Go to Settings > Integrations > Cloud accounts.
  3. Click + Add New.
  4. Click Google Cloud Platform and select Manual configuration.
  5. Click Next.
  6. Select Audit Log (PubSub) as the integration type.
  7. For Name, enter a unique name for the audit log integration.
  8. Follow the steps to either upload GCP credentials or enter the credentials manually.

When creating the Google Cloud integration, you can either upload Google Cloud credentials or enter the credentials manually. Finish creating the integration in the Lacework Console by following the steps described in one of the following sections.

Upload GCP Credentials​

To upload Google Cloud credentials, follow these steps:

  1. For Upload GCP Credential, click Choose File, navigate to the JSON key file downloaded when you created the Google Cloud service account, then click Open.
    This populates the credential fields.

  2. For Integration Level, select Organization or Project. Select Organization if integrating at the organization level. Select Project if integrating at the project level.

  3. Do one of the following:

    • If integrating at the project level, enter the project ID in the Project ID field.

    • If integrating at the organization level, enter the Google Cloud organization ID in the Organization ID field.

      To obtain the organization ID, do the following:

      1. Log in to the Google Cloud console.

      2. Click the project selection drop-down list in the top menu bar.

      3. From the Select from drop-down list, select the organization that contains the projects that you want the integration to monitor.

        gcp_select_org.png

      4. Go to IAM & admin > Settings and copy the number in the Organization ID field.

  4. For Subscription Name, paste the Pub/Sub subscription path that you copied in an earlier step. The subscription path is in the following format:

    projects/$projectName/subscriptions/$topicSubscriptionName

  5. Enter the Pub/Sub topic ID in the Topic ID field. For example, if you created a Pub/Sub topic named mytopicname, enter mytopicname.

  6. Click Save to finish the Google Cloud integration and save your onboarding progress. The integration appears in the list of cloud accounts under Cloud accounts.

    The integration appears in the list of cloud accounts under Cloud accounts. A Pub/Sub-based integration has the provider as GCP and type as Audit Log (PubSub). The integration status Success indicates that Lacework is successfully injesting audit log data from Google Cloud. Audit Log Integration Status

Enter GCP Credentials Manually​

To manually enter Google Cloud credentials, follow these steps:

  1. Locate the service account key JSON file you downloaded when you created the Google Cloud service account.

  2. Open the file in a text editor.

  3. Copy the value of the client_id property from the JSON file and paste it into the Client ID field of the Lacework Console.

  4. Copy the value of the private_key_id property from the JSON file and paste it into the Private Key ID field of the Lacework Console.

  5. Copy the value of the client_email property from the JSON file and paste it into the Client Email field of the Lacework Console.

  6. Exit the text editor. You cannot just copy the private key from the editor because of an issue copying the new line characters. You must copy a raw version of the key using the jq utility as described below.

  7. Open a terminal window and go the directory where you downloaded the JSON file containing the service account key.

  8. Verify that the jq (command-line JSON processor) utility is available from your command-line shell. 

    jq

  9. Enter the following command, where YourFileName.json is the name of the service account key JSON file.

    cat YourFileName.json  | jq -r '.private_key'

  10. Copy all the text displayed in the output including the BEGIN and END lines.

    -----BEGIN PRIVATE KEY-----
    YourKeyInfo
    -----END PRIVATE KEY-----

  11. Paste the text into the Private Key field of the Lacework Console.

  12. For Integration Level, select Organization or Project. Select Organization if integrating at the organization level. Select Project if integrating at the project level.

  13. Do one of the following:

    • If integrating at the project level, enter the project ID in the Project ID field.

    • If integrating at the organization level, enter the organization ID in the Organization ID field.

      To obtain the organization ID, do the following:

      1. Log in to the Google Cloud console.

      2. Click the project selection drop-down list in the top menu bar.

      3. From the Select from drop-down list, select the organization that contains the projects that you want the integration to monitor.

        gcp_select_org.png

      4. Go to IAM & admin > Settings and copy the number in the Organization ID field.

  14. For Subscription Name, paste the Pub/Sub subscription path that you copied in an earlier step. The subscription path is in the following format:

    projects/$projectName/subscriptions/$topicSubscriptionName

  15. Enter the Pub/Sub topic ID in the Topic ID field.

  16. Click Save to finish the Google Cloud integration and save your onboarding progress. The integration appears in the list of cloud accounts under Cloud accounts.

    The integration appears in the list of cloud accounts under Cloud accounts. A Pub/Sub-based integration has the provider as GCP and type as Audit Log (PubSub). The integration status Success indicates that Lacework is successfully injesting audit log data from Google Cloud. Audit Log Integration Status

Last updated on