Open menu

Enable CloudWatch Logs Encryption for AWS Glue

Cloud Conformity allows you to automate the auditing process of this resolution page. Register for a 14 day evaluation and check your compliance level for free!

Start a Free Trial Product features
Security

Risk level: Medium (should be achieved)

Ensure that encryption at rest is enabled for your Amazon Glue security configurations in order to meet regulatory requirements and prevent unauthorized users from getting access to the logging data published to AWS CloudWatch Logs. A security configuration is a set of encryption properties that are used by Amazon Glue service to configure encryption for crawlers, jobs and development endpoints.

To meet security and compliance requirements, it is strongly recommended to implement encryption at rest when publishing AWS Glue logs to Amazon CloudWatch.

Audit

To determine if your AWS Glue security configurations have CloudWatch Logs encryption mode enabled, perform the following actions:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to Glue service dashboard at https://console.aws.amazon.com/glue/.

03 In the navigation panel, under Security, choose Security configurations.

04 Select the security configuration that you want to examine, then click on the resource name to access its configuration details page.

05 On the security configuration details page, check the CloudWatch logs encryption mode attribute value. If the configuration attribute value is DISABLED, the selected security configuration is not compliant, therefore Amazon Glue logs are not encrypted once these are published to AWS CloudWatch Logs.

06 Repeat step no. 4 and 5 to verify other Amazon Glue security configurations created in the current region.

07 Change the AWS region from the console navigation bar and repeat the audit process for other regions.

Using AWS CLI

01 Run get-security-configurations command (OSX/Linux/UNIX) to retrieve the names of all AWS Glue security configurations created within the selected AWS region – in this case the US East (N. Virginia) region:

aws glue get-security-configurations
	--region us-east-1
	--output table
	--query 'SecurityConfigurations[*].Name'

02 The command output should return a table with security configuration names:

-------------------------------
|  GetSecurityConfigurations  |
+-----------------------------+
|  cc-security-configuration  |
|  cc-glue-new-config-file    |
+-----------------------------+

03 Execute get-security-configuration command (OSX/Linux/UNIX) using the name of the AWS Glue security configuration that you want to examine as identifier and custom query filters to expose the AWS CloudWatch Logs encryption mode status:

aws glue get-security-configuration
	--name cc-security-configuration
	--region us-east-1
	--query 'SecurityConfiguration.EncryptionConfiguration.CloudWatchEncryption.CloudWatchEncryptionMode'

04 The command output should return the request information (i.e. encryption mode status):

"DISABLED"

If get-security-configuration command output returns "DISABLED", as shown in the example above, the selected security configuration is not compliant, therefore Amazon Glue logs are not encrypted after these are published to AWS CloudWatch Logs.

05 Repeat step no. 3 and 4 to check other Amazon Glue security configurations available in the selected region.

06 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 – 5 to perform the audit process for other regions.

Remediation / Resolution

To enable encryption at rest for Amazon Glue logging data published to AWS CloudWatch Logs, you need to re-create the necessary security configurations with the CloudWatch Logs encryption mode enabled. To create and configure a new AWS Glue security configuration, perform the following actions:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to Glue service dashboard at https://console.aws.amazon.com/glue/.

03 In the left navigation panel, under Security, choose Security configurations.

04 Click Add security configuration to initiate the setup process.

05 On Add security configuration page, perform the following:

  1. Enter a unique name for your new configuration within Security configuration name box.
  2. Select CloudWatch logs encryption checkbox to enable at-rest encryption when writing logs to AWS CloudWatch, then choose the ARN of the AWS KMS key that you want to use for encryption from AWS KMS key dropdown list.
  3. Make sure that S3 encryption and Job bookmark encryption are enabled within the configuration file, then click Finish to create the new AWS Glue security configuration.

06 Reconfigure (update) your existing Amazon Glue crawlers, jobs and development endpoints to make use of the new security configuration created at the previous step.

07 Change the AWS region from the navigation bar and repeat the entire process for other regions.

Using AWS CLI

Note: Enabling data-at-rest encryption with KMS Customer Master Keys (CMKs) for AWS Glue Data Catalog connection passwords using the AWS API via Command Line Interface (CLI) is not currently supported.

01 Define your new AWS Glue security configuration parameters and save the document to a JSON file named sec-config-logs-encrypted.json. Replace "KmsKeyArn" parameter value with the Amazon Resource Name (ARN) of your own AWS KMS key:

{
  "CloudWatchEncryption": {
    "CloudWatchEncryptionMode": "SSE-KMS",
    "KmsKeyArn": "arn:aws:kms:us-east-1:1234567890:key/abcdabcd-1234-abcd-1234-abcdabcdabcd"
  },
  "S3Encryption": [
    {
      "S3EncryptionMode": "DISABLED"
    }
  ],
  "JobBookmarksEncryption": {
    "JobBookmarksEncryptionMode": "DISABLED"
  }
}

02 Run create-security-configuration command (OSX/Linux/UNIX) using the sec-config-logs-encrypted.json file defined at the previous step as command parameter to create a new Amazon Glue security configuration that has AWS CloudWatch Logs encryption mode enabled:

aws glue create-security-configuration
	--region us-east-1
	--name cc-new-sec-configuration
	--encryption-configuration file://sec-config-logs-encrypted.json

03 The command output should return the command request metadata:

{
    "CreatedTimestamp": 1548331153.265,
    "Name": "cc-new-sec-configuration"
}

04 Now you can update your existing Amazon Glue crawlers, jobs and development endpoints configuration to make use of the new security configuration created at the previous steps.

05 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 – 4 to perform the remediation/resolution process for other regions.

References

Publication date Jan 17, 2019