Log Streaming to Google Cloud Storage

Updated 2 years ago by admin

The Log Streaming service allows enriched logs from the USS platform to be streamed to external services such as SIEM solutions, analytics platforms and SoC services. To request access to the Log Streaming service, please contact your service provider.

This article requires a Google Cloud Platform account with permission to create Cloud Storage buckets and Service Account keys
Please note that a Google Cloud Storage bucket will be needed for each product you want to stream data from
  1. Log in to Google Cloud Platform and navigate to Cloud Storage
  2. Navigate to Cloud Storage and then click Create
  3. Enter a name for your bucket, e.g. clouduss-web-logs and make a note of this, then press Continue
  4. Select a region appropriate for your requirements and click Continue
  5. Select a storage class appropriate for your requirements and click Continue
  6. Ensure that Prevent Public Access is ticked and select an Access Control method appropriate for your requirements and then click Continue
  7. Select a data protection policy appropriate for your requirements and then click Create
  8. Navigate to IAM & Admin section and then click Service Accounts
  9. Click Create Service Account
  10. Enter a Service Account name and click Done
  11. Click the copy icon next to the newly created Service Account email and keep it somewhere safe for Step 16 below. It will be in the format <service key name>@<project ID>.iam.gserviceaccount.com
  12. Select the newly created Service Account and then the Keys tab
  13. Click Add Key and then Create New Key
  14. Select JSON and then click Create. A JSON file will be downloaded by the browser
  15. Important: send the JSON file and the bucket name from Step 3 to your Service Provider
  16. Navigate back to Cloud Storage and select your bucket and then click Permissions and then Grant Access
  17. Enter the email address copied from Step 11 above into the New Principal field
  18. Assign the role Storage Object Creator to the principal and click Save. The principal requires the ability to create objects in the bucket
  19. Wait for confirmation from your service provider that the log stream has been configured
  20. View the contents of your bucket and you should see objects being created by the Log Streaming service
Note that it could take up to an hour for objects to appear in the bucket for the first time. In most cases it is within 10-15 minutes.
  1. The log stream is now set up and you can consume the data in any service that supports Google Cloud Storage integration, such as Google Chronicle, Google BigQuery or any other third party application.

How did we do?