Articles in this section

GCP Audit Trail Log Integration - Manually using the GCP Console

This topic describes how to manually configure a Lacework GCP Audit Trail integration. Lacework recommends creating the GCP Audit Trail Log integration using GCP Compliance and Audit Log Integration - Run app.py Python Script.

To create a Lacework GCP Audit Trail integration to gather, process, report, and alert on GCP Audit Log data, follow the steps in the procedure below. Note that you must create a separate GCP integration for reporting and alerting on GCP compliance. For more information, see Lacework for GCP. You can, however, use the same credentials for both integrations.

When a GCP Audit Trail integration is created, Lacework gathers the logs of administrative operations from GCP audit logs. GCP captures detailed audit log data and stores that data in audit logs. Lacework aggregates and organizes this audit log data into useful maps and dashboards that illustrate the following:

  • conceptual relationships
  • causes and effects
  • interactions between GCP entities

Lacework uses StackDriver to get the GCP audit logs.

Lacework also automatically generates alerts whenever an audit log event represents a security risk.

Requirements

  • Decide if you want to monitor the audit logs at the organization, folder, or project level. If you specify the option to include children when creating the sink at the organization or folder level, all the projects in the organization or the folder export the audit logs to the bucket.
  • If you are already exporting your audit logs to a storage bucket, you can reuse that storage bucket but the project with the storage bucket must be granted role access as described in the procedure below.
  • The procedure below requires that the jq utility is installed. The jq utility is a flexible command-line JSON processor. For more information, see https://stedolan.github.io/jq/

Create a GCP Service Account and Grant Access

Follow the procedure provided in Create a GCP Service Account and Grant Access. You can skip this procedure if you have already created a service account and granted role access when creating a new GCP Compliance integration. You can also reuse the same service account.

Enable the Required GCP APIs

When you manually create a GCP Audit Trail Integration, you must enable APIs for the GCP projects you want to integrate with. Follow the procedure provided in Enable the Required GCP APIs.

Configure GCP for Audit Log Integration

Configure the GCP Audit Log integration to ingest audit log data:

  1. Verify that the jq (command-line JSON processor) utility is available from your command-line shell. Open a command-line window and enter the following command. If the jq utility is found, skip to step 3. 
    $ jq
  2. If the jq utility is not installed or not listed in your PATH, install it (https://stedolan.github.io/jq/ and verify that the path to the utility is listed in your PATH environment variable. The jq utility is required for some of the steps in the following procedure.
  3. If you do not have the GCP CLI installed on your system, install it. For more information, see CLOUD SDK. As part of the install process, make sure you initialize gcloud with the following command: gcloud init. Leave the CLI window open.
  4. Log in to the GCP Console.
  5. Decide if you want to export the audit logs at the organization, folder, or project level. In addition, pick an entity to export audit logs from. An entity is either an organization, folder, or project. This entity must be granted access to the service account and must have the role access as described in Create a GCP Service Account and Grant Access. A project can have access because it was directly granted access or because its parent organization was granted access. Billing must also be enabled for any projects that you want to export audit logs from. Note that if you select the option to include children when creating the sink at the organization or folder level, all the projects in the organization or the folder export the audit logs to the bucket.
  6. Locate and get the ID of a GCP project where the bucket and sink will be created as described in steps 7-13. In addition, if you decide to export at the project level, this is the project that will export its audit logs.
  7. Click the down arrow in the top menu bar. The Select from dialog appears.
  8. From the Select from the drop-down, select an organization that contains the GCP project to monitor audit logs.
  9. Select the ALL tab.
  10. Locate the GCP project.
  11. From the ID column, copy the Project ID.
  12. In the CLI window, enter the following text: 
    $ projectName=
  13. Paste the ID of the project that will monitor audit logs and enter return. Replace my-first-project-654321 with your project ID. 
    $ projectName=my-first-project-654321

  14. Set the current project by entering the following command. If Updated property [core/project] is not returned, make sure the quotes are straight double-quotes. Use straight quotes for all the CLI commands in this procedure.

    $ gcloud config set project "$projectName"

  15. Create a storage bucket in the project. Replace mybucketname with your bucket name.

    $ bucketName=mybucketname
$ gsutil mb "gs://$bucketName"

  16. Set the name of sink. Replace mysink with your sink name.

    $ sinkName=mysink

  17. Create the sink. Decide if you want to export the audit logs at the organization, folder, or project level and create the sink. If creating the sink at the organization level, add --include-children --organization=myorganizationid to the end of the following command. If creating the sink at the folder level, add --include-children --folder=myfolderid to the end of the following command. If you specify the --include-children option when creating the sink at the organization or folder level, all the projects in the organization or the folder export audit logs to the bucket.

    $ gcloud logging sinks create "$sinkName" "storage.googleapis.com/$bucketName" --log-filter "protoPayload.@type=type.googleapis.com/google.cloud.audit.AuditLog AND NOT protoPayload.methodName:’storage.objects.’"
  18. Copy the service account returned by the previous command. Copy the text between the single quotes. In the following example, the text to copy is italicized. The next two steps grants the objectViewer role on the bucket. In a later step, you grant Storage Object Creator role on the bucket. 
    Please remember to grant `serviceAccount:123567@gcp-sa-logging.iam.gserviceaccount.com` the Storage Object Creator role on the bucket.

  19. Enter sinkServiceAccount=, paste the text and enter return.

    $ sinkServiceAccount=serviceAccount:123567@gcp-sa-logging.iam.gserviceaccount.com

  20. Grant bucket read permissions to the service account returned by the preceding sink create command.

    $ gsutil iam ch "$sinkServiceAccount:objectViewer" "gs://$bucketName"

  21. Enter one of the following commands to configure the storage bucket to allow the project to write its logs to it. The first command is project-level command, the second command is the folder-level command, and the third command is the organization-level command

    $ sinkwriterIdentity=$(gcloud logging sinks describe "$sinkName" --format json | jq -r '.writerIdentity')
$ sinkwriterIdentity=$(gcloud logging sinks describe "$sinkName" --folder=myfolderid --format json | jq -r '.writerIdentity')
$ sinkwriterIdentity=$(gcloud logging sinks describe "$sinkName"  --organization=myorganizationid --format json | jq -r '.writerIdentity')

  22. Configure storage bucket to allow the project to write its logs to it.

    $ gsutil iam ch "$sinkwriterIdentity:objectCreator" "gs://$bucketName"

  23. Create the pubsub topic under the project for Audit log file notifications. Replace mytopic with your topic name.

    $ topicName=mytopic
$ gcloud config set project "$projectName"
$ gcloud pubsub topics create "$topicName"

  24. If this is a new project, you must grant the GCP-managed service account sufficient privileges to publish to the log integration pubsub topic. Get the name of the GCP-managed service account that is created when a new project is created. This service account is in the format: service-YourProjectNumber@gs-project-accounts.iam.gserviceaccount.com. Replace YourProjectNumber with your value. For instructions on getting the project number, see https://cloud.google.com/resource-manager/docs/creating-managing-projects. You can also get this service account by entering the project id in the projectId parameter in the Try this API panel of https://cloud.google.com/storage/docs/json_api/v1/projects/serviceAccount/get. NOTE: This service account may not be visible in the IAM & Admin > IAM panel of the GCP Console.

    $ gcloud alpha pubsub topics add-iam-policy-binding "projects/$projectName/topics/$topicName" --member=serviceAccount:service-YourProjectNumber@gs-project-accounts.iam.gserviceaccount.com --role=roles/pubsub.publisher
  25. Create the pubsub subscription (queue) for the audit log file notification topic created in the previous step. Replace mytopicsub with your topic subscription name. 
    $ topicSubscriptionName=mytopicsub
$ gcloud pubsub subscriptions create "$topicSubscriptionName" --topic "$topicName" --ack-deadline=300

  26. Configure the storage bucket to send notifications to pubsub. This also configures the topic policy to allow publishing from the bucket.

    $ gsutil notification create -e OBJECT_FINALIZE -f json -t "projects/$projectName/topics/$topicName" "gs://$bucketName"
  27. Grant bucket read permissions to the service account that you granted role access in Create a GCP Service Account and Grant Access. This is not the service account returned by the sink create command above. Replace the bold text with your service account. Note that the string serviceAccount: must preface the service account name. 
    $ serviceAccount=serviceAccount:myserviceaccount@gcp-sa-logging.iam.gserviceaccount.com
$ gsutil iam ch "$serviceAccount:objectViewer" "gs://$bucketName"

  28. Grant the integration service account the subscriber role on the subscription. If prompted to install a command group, answer y.

    $ gcloud alpha pubsub subscriptions add-iam-policy-binding "projects/$projectName/subscriptions/$topicSubscriptionName" --member="$serviceAccount" --role=roles/pubsub.subscriber

  29. Enter the following command to get the subscription path that is used when creating the integration in the Lacework console.

    $ echo "projects/$projectName/subscriptions/$topicSubscriptionName"
  30. Copy the resulting subscription path returned by the echo command for use in the next procedure that creates the integration.

Create the GCP Audit Log Integration on the Lacework Console

When creating the GCP Audit Log integration, you can either upload GCP credentials or enter all information manually. Finish creating the integration in the Lacework Console by following the steps described in one of the following sections.

Upload GCP Credentials

  1. Log in to the Lacework Console.
  2. Navigate to Settings > Cloud Accounts.
  3. Click + Create New.
  4. Select GCP and Audit Trail and click Next.
  5. Enter a unique name for the integration and click Next.
  6. In the Upload GCP Credential field, click Choose File and navigate to the JSON key file downloaded when you created the GCP Service Account.
    This populates the credential fields.
  7. From the Integration Level drop-down, select Organization or Project. Select Organization if integrating at the organization level. Select Project if integrating at the project level.
  8. Copy the appropriate ID value for your integration type:
    1. If integrating at the project level, copy the value of the project_id property from the JSON file into the Org/Project ID field of the Lacework Console.
    2. If integrating at the organization level, log in to the GCP console. Click the down arrow in the top menu bar. From the Select from the drop-down, select an organization that contains the GCP project(s) that you want the integration to monitor. Select IAM & admin > Settings and copy the number from the Organization ID field and paste the value into the Org/Project ID field of the Lacework Console.
      image-1.png
  9. Paste the subscription path that was copied in an earlier step into the Subscription Name field. The subscription path is in the following format: 
    projects/$projectName/subscriptions/$topicSubscriptionName
  10. Click Save. A new integration displays in Cloud Accounts.
  11. When the integration is complete and successful, the status changes to Integration Successful.

Enter Information Manually

  1. Verify that the jq (command-line JSON processor) utility is available from your command-line shell. Leave this command-line window open. 
    $ jq
  2. If the jq utility is found, skip to the next step. If the jq utility is not installed or not listed in your PATH, install it (https://stedolan.github.io/jq/ and verify that the path to the utility is listed in your PATH environment variable. The jq utility is required for some steps in the following procedure.
  3. Find the JSON file downloaded when you created the GCP Service Account.
  4. Open the file in an editor and leave it open.
  5. Log in to the Lacework Console.
  6. Navigate to Settings > Cloud Accounts.
  7. Click + Create New.
  8. Select GCP and Audit Trail and click Next.
  9. Enter a unique name for the integration and click Next.
  10. Copy the value of the client_id property from the JSON file and paste the value into the Client ID field of the Lacework Console.
  11. Copy the value of the client_email property from the JSON file and paste the value into the Client Email field of the Lacework Console.
  12. Exit the editor.
  13. You cannot just copy the private key from the editor because of an issue copying the new line characters. You must copy a raw version of the key using the “jq” utility as described in the next steps.

  14. To view the private key raw text, enter the following command, where YourFileName.json is the name of the file downloaded when you created the GCP Service Account.

    $ cat YourFileName.json  | jq -r '.private_key'
  15. Copy all text displayed in the output including the BEGIN and END lines. 
    -----BEGIN PRIVATE KEY-----
YourKeyInfo
-----END PRIVATE KEY-----
  16. Paste the text into the Private Key field of the Lacework Console.
  17. From the Integration Level drop-down, select the Organization or Project. Select Organization if integrating at the organization level. Select Project if integrating at the project level.
  18. Copy the appropriate value for your integration type as described in the next two steps.
    1. If integrating at the project level, copy the value of the project_id property from the JSON file into the Org/Project ID field of the Lacework Console.
    2. If integrating at the organization level, log in to the GCP console. Click the down arrow in the top menu bar. From the Select from the drop-down, select an organization that contains the GCP project(s) that you want the integration to monitor.  Select IAM & admin > Settings and copy the number from the Organization ID field and paste the value into the Org/Project ID field of the Lacework Console.
  19. Paste the subscription path that was copied in an earlier step into the Subscription Name field. The subscription path is in the following format: 
    projects/$projectName/subscriptions/$topicSubscriptionName
  20. Click Save. A new integration displays in Cloud Accounts.
  21. When the integration is complete and successful, the status changes to Integration Successful.