Open menu
-->

Enable Firehose Delivery Stream Encryption with KMS Customer Master Keys

Cloud Conformity allows you to automate the auditing process of this resolution page. Register for a 14 day evaluation and check your compliance level for free!

Start a Free Trial Product features
Security

Risk level: High (not acceptable risk)

Ensure that your AWS Kinesis Firehose delivery streams are encrypted with Amazon KMS Customer Master Keys (CMKs) instead of AWS managed-keys (i.e. default keys used by Kinesis Firehose service when there are no customer keys defined) in order to have full control over the encryption and decryption process and meet regulatory requirements. AWS Kinesis Firehose is a fully managed service designed for real-time streaming data delivery to destinations such as Amazon S3, Amazon Redshift, Amazon ElasticSearch Service and Splunk.

This rule resolution is part of the Cloud Conformity Base Auditing Package

When you utilize your own AWS KMS Customer Master Keys (CMKs) to protect your Kinesis Firehose delivery streams, you have complete control over who can use these keys to access your streaming data. Amazon KMS service allows you to easily create, rotate, disable and audit the CMK encryption keys for your Kinesis Firehose delivery streams.

Audit

To determine the encryption status for your Firehose delivery streams, perform the following actions:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Kinesis dashboard at https://console.aws.amazon.com/kinesis/.

03 In the navigation panel, under Amazon Kinesis, choose Data Firehose.

04 Choose the Firehose delivery stream that you want to examine, then click on its name (link) to access the stream configuration details.

05 Select the Details tab from the top panel and check the Encryption attribute value, available in the Amazon S3 destination section. If the attribute value is set to Disabled, see this conformity rule to enable delivery stream encryption. If the Encryption attribute value is set to an ARN that ends in "alias/aws/s3" (i.e. arn:aws:kms:us-east-1:<aws-account-id>:alias/aws/s3), the selected Amazon Kinesis Firehose delivery stream is encrypted using the default master key (AWS-managed key) instead of an AWS KMS Customer Master Key (CMK).

06 Repeat step no. 4 and 5 for each Firehose delivery stream available in the current AWS region.

07 Change the AWS region from the navigation bar to repeat the audit process for other regions.

Using AWS CLI

01 Run list-streams command (OSX/Linux/UNIX) to list the names of all Firehose delivery streams available in the selected AWS region, i.e. US East (N. Virginia):

aws firehose list-delivery-streams
	--region us-east-1
	--query 'DeliveryStreamNames'

02 The command output should return the requested delivery stream names:

[
    "cc-project5-delivery-stream",
    "cc-iot-system-delivery-stream"
]

03 Run describe-delivery-stream command (OSX/Linux/UNIX) using the delivery stream name returned at the previous step as identifier and custom query filters to return the Server-Side Encryption (SSE) configuration for the delivery stream data destination:

aws firehose describe-delivery-stream
	--region us-east-1
	--delivery-stream-name cc-project5-delivery-stream
	--query 'DeliveryStreamDescription.Destinations[*].ExtendedS3DestinationDescription.{EncryptionConfiguration: EncryptionConfiguration}'

04 The command output should return the requested configuration details. In case the describe-delivery-stream command output returns just "NoEncryptionConfig": "NoEncryption" key-value pair for the EncryptionConfiguration attribute, see this conformity rule to enable delivery stream encryption:

[
  {
     "EncryptionConfiguration": {
        "KMSEncryptionConfig": {
           "AWSKMSKeyARN": "arn:aws:kms:us-east-1:123456789012:alias/aws/s3"
        }
     }
  }
]

05 Run describe-key command (OSX/Linux/UNIX) using the AWS KMS key ARN returned at the previous step as identifier and custom query filters to return the manager name for the encryption key used (the KMS key is either a customer managed or AWS managed):

aws kms describe-key
	--region us-east-1
	--key-id arn:aws:kms:us-east-1:123456789012:alias/aws/s3 --query 'KeyMetadata.KeyManager'

06 The command output should return the requested key details:

"AWS"

If the configuration value returned by the describe-key command output is "AWS", the encryption key manager is Amazon Web Services and not the customer, therefore the selected Amazon Kinesis Firehose delivery stream is encrypted using the default master key (AWS-managed key) instead of an AWS KMS Customer Master Key (CMK).

07 Repeat steps no. 3 – 6 for each Firehose delivery stream available in the current AWS region.

08 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 7 to perform the entire process for other regions.

Remediation / Resolution

To encrypt existing AWS Kinesis Firehose delivery streams with your own KMS Customer Master Key (CMKs), perform the following:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to IAM dashboard at https://console.aws.amazon.com/iam/.

03 In the left navigation panel click Encryption Keys.

04 Select the appropriate AWS region from the Filter menu (must match the region where your Firehose delivery streams are).

05 Click Create Key button from the dashboard top menu.

06 In the Alias (required) and Description fields, enter a unique name (alias) and a description for the new CMK, then click the Next Step button.

07 Under Key Administrators section, select which IAM users and/or roles can administer the new CMK, then click Next Step.

08 Under This Account section, select which IAM users and/or roles can use the new CMK to encrypt/decrypt the domain data with the AWS KMS API.

09 (Optional) Under External Accounts section, click Add an External Account and enter an external account ID in order to add another AWS account that can use this CMK to encrypt/decrypt the Firehose streaming data. The owners of the external AWS accounts must also provide access to this CMK by creating appropriate policies for their IAM users.

10 Click Next Step to continue.

11 Under Preview Key Policy section, review the key policy generated by AWS then click Finish to create your new CMK. Once the key is created, the KMS dashboard will display a confirmation message: “Your master key was created successfully. Alias: <cmk-alias>”.

12 Once the KMS key has been created, navigate to Amazon Kinesis dashboard at https://console.aws.amazon.com/kinesis/.

13 In the navigation panel, under Amazon Kinesis, choose Data Firehose.

14 Choose the delivery stream that you want to reconfigure (see Audit section part I to identify the right Firehose resource), then click on its name (link) to access the resource configuration.

15 Select the Details tab from the top panel and click the Edit button from the top-right menu to switch to edit mode.

16 Within Amazon S3 destination section, from KMS master key dropdown list, select the ID of the AWS KMS Customer Master Key (CMK) created earlier in the remediation section.

17 Click Save to apply the configuration changes. If successful, the AWS console should display the following confirmation message: "Successfully updated delivery stream".

18 Repeat steps no. 14 – 17 to configure Server-Side Encryption (SSE) with KMS CMKs for other Firehose delivery streams available within the selected AWS region.

19 Change the AWS region from the navigation bar and repeat the entire process for other regions.

Using AWS CLI

01 Before you create your KMS Customer Master Key (CMK), you must define a policy that enables the selected IAM users and/or roles to administer the new CMK and to encrypt/decrypt Firehose streaming data using the KMS API. Create a new policy document called firehose-kms-cmk-policy.json and paste the following content (replace the highlighted details, i.e. the ARNs for the IAM users and/or roles, with your own details):

{
  "Version": "2012-10-17",
  "Id": "kinesis-firehose-cmk-policy",
  "Statement": [
    {
      "Sid": "Enable IAM User Permissions",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::123456789012:root"
      },
      "Action": "kms:*",
      "Resource": "*"
    },
    {
      "Sid": "Grant access to CMK manager",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::123456789012:role/cc-kinesis-manager"
      },
      "Action": [
        "kms:Create*",
        "kms:Describe*",
        "kms:Enable*",
        "kms:List*",
        "kms:Put*",
        "kms:Update*",
        "kms:Revoke*",
        "kms:Disable*",
        "kms:Get*",
        "kms:Delete*",
        "kms:ScheduleKeyDeletion",
        "kms:CancelKeyDeletion"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Allow the use of the CMK",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::123456789012:user/cc-kinesis-admin"
      },
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:DescribeKey"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Allow attachment of persistent resources",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::123456789012:user/cc-kinesis-admin"
      },
      "Action": [
        "kms:CreateGrant",
        "kms:ListGrants",
        "kms:RevokeGrant"
      ],
      "Resource": "*",
      "Condition": {
        "Bool": {
          "kms:GrantIsForAWSResource": "true"
        }
      }
    }
  ]
}

02 Run create-key command (OSX/Linux/UNIX) using the file name of the policy document created at the previous step (i.e. firehose-kms-cmk-policy.json) as required command parameter to create the new KMS CMK:

aws kms create-key
	--region us-east-1
	--description 'KMS CMK for Kinesis Firehose streaming data'
	--policy file://firehose-kms-cmk-policy.json

03 The command output should return the new KMS CMK metadata. Copy the CMK unique ID (KeyID parameter value – highlighted) as this ID will be required later when you need to specify the key required for Firehose streaming data encryption:

{
    "KeyMetadata": {
        "Origin": "AWS_KMS",
        "KeyId": "abcdabcd-aaaa-bbbb-cccc-abcdabcdabcd",
        "Description": "KMS CMK for Kinesis Firehose streaming data",
        "Enabled": true,
        "KeyUsage": "ENCRYPT_DECRYPT",
        "KeyState": "Enabled",
        "CreationDate": 1517234889.280,
        "Arn": "arn:aws:kms:us-east-1:123456789012:key/abcdabcd-aaaa-bbbb-cccc-abcdabcdabcd",
        "AWSAccountId": "123456789012"
    }
}

04 Run create-alias command (OSX/Linux/UNIX) using the key ARN returned at the previous step to attach an alias to the new CMK. The alias must start with the prefix "alias/" (the command does not produce an output):

aws kms create-alias
	--region us-east-1
	--alias-name alias/firehose-data-cmk
	--target-key-id arn:aws:kms:us-east-1:123456789012:key/abcdabcd-aaaa-bbbb-cccc-abcdabcdabcd

05 Run describe-delivery-stream command (OSX/Linux/UNIX) using the name of the delivery stream that you want to reconfigure as identifier (see Audit section part II to identify the right resource) and custom query filters to return the selected stream configuration metadata, information required later when the delivery stream will be reconfigured:

aws firehose describe-delivery-stream
	--region us-east-1
	--delivery-stream-name cc-project5-delivery-stream

06 The command output should return the stream configuration information:

[
    "DeliveryStreamDescription": {
        "DeliveryStreamType": "KinesisStreamAsSource",
        "HasMoreDestinations": false,
        "LastUpdateTimestamp": 1535049631.780,
        "VersionId": "3",

         ...

        "CreateTimestamp": 1535041354.658,
        "DeliveryStreamARN": "arn:aws:firehose:us-east-1:123456789012:deliverystream/cc-project5-delivery-stream",
        "DeliveryStreamStatus": "ACTIVE",
        "DeliveryStreamName": "cc-project5-delivery-stream"
    }
}

07 Create the necessary configuration file, name it stream-destination-config.json and save it as a JSON file, using the metadata returned at the previous step. The ARN of the AWS CMK used for encryption, returned at step no. 3, should be configured as value for the AWSKMSKeyARN attribute:

{
  "RoleARN": "arn:aws:iam::123456789012:role/firehose_delivery_role",
  "BucketARN": "arn:aws:s3:::cc-firehose-s3-bucket",
  "Prefix": "",
  "BufferingHints": {
    "SizeInMBs": 30,
    "IntervalInSeconds": 300
  },
  "CompressionFormat": "UNCOMPRESSED",
  "EncryptionConfiguration": {
    "KMSEncryptionConfig": {
      "AWSKMSKeyARN": "arn:aws:kms:us-east-1:123456789012:key/abcdabcd-aaaa-bbbb-cccc-abcdabcdabcd"
    }
  },
  "CloudWatchLoggingOptions": {
    "Enabled": true,
    "LogGroupName": "/aws/kinesisfirehose/cc-project5-delivery-stream",
    "LogStreamName": "S3Delivery"
  },
  "ProcessingConfiguration": {
    "Enabled": false,
    "Processors": []
  },
  "S3BackupMode": "Disabled"
}

08 Run update-destination command (OSX/Linux/UNIX) using the name of the Firehose delivery stream that you want to reconfigure (see Audit section part II to identify the right resource) to update the selected stream destination configuration with the stream-destination-config.json file in order to set up encryption using a KMS Customer Master Key with the ARN "arn:aws:kms:us-east-1:123456789012:key/abcdabcd-aaaa-bbbb-cccc-abcdabcdabcd" (the command does not produce an output):

aws firehose update-destination
	--region us-east-1
	--delivery-stream-name cc-project5-delivery-stream
	--current-delivery-stream-version-id 3
	--destination-id destinationId-000000000001
	--extended-s3-destination-update file://stream-destination-config.json

09 Repeat steps no. 5 – 8 to configure Server-Side Encryption with KMS Customer Master Keys for other Firehose delivery streams available in the selected AWS region.

10 Change the AWS region by updating the --region command parameter value and repeat the entire remediation process for other regions.

References

Publication date Oct 15, 2018