Open menu
-->

Enable AWS ElasticSearch Encryption At Rest

Cloud Conformity allows you to automate the auditing process of this resolution page. Register for a 14 day evaluation and check your compliance level for free!

Start a Free Trial Product features
Security

Risk level: High (not acceptable risk)

Ensure that your Amazon ElasticSearch (ES) domains are encrypted in order to meet security and compliance requirements. Encryption of data at rest helps prevent unauthorized users from reading sensitive information available on your ES domains (clusters) and their storage systems. This includes all data stored on the underlying file systems, primary and replica indices, log files, memory swap files and automated snapshots saved to S3. Amazon ElasticSearch handles the encryption/decryption process seamlessly, so you don’t have to modify your applications to access your data. The ElasticSearch at-rest encryption feature uses AWS KMS service to store and manage the encryption keys.

When working with production data that contains sensitive information, it is highly recommended to implement encryption at rest in order to protect it from unauthorized access and fulfill any compliance requirements available within your organization. Note: At-rest encryption can be enabled only for AWS ES domains with ElasticSearch version 5.1 and above.

Audit

To determine if data-at-rest encryption is enabled for your AWS ES domains, perform the following:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to ElasticSearch (ES) dashboard at https://console.aws.amazon.com/es/.

03 Choose the ES domain that you want to examine and click on the domain name (link) to access its configuration page.

04 On the domain configuration page, select the Overview tab and check the current value set for the Encryption at rest attribute. If the attribute value is set to Disabled, the data-at-rest encryption feature is not enabled for the selected Amazon ElasticSearch domain, therefore the data stored on the domain file systems, primary and replica indices, log files, memory swap files and so on, is not protected from unauthorized access.

05 Repeat step no. 3 and 4 to verify the encryption status for other AWS ElasticSearch domains available in the current region.

06 Change the AWS region from the navigation bar and repeat the process for the other regions.

Using AWS CLI

01 Run list-domain-names command (OSX/Linux/UNIX) to list the names of all Amazon ElasticSearch domains currently available within the selected region:

aws es list-domain-names 
	--region us-east-1

02 The command output should return the requested ES domain names:

{
    "DomainNames": [
        {
            "DomainName": "cc-prod-es-domain"
        },
        {
            "DomainName": "cc-project7-domain"
        }
    ]
}

03 Run describe-elasticsearch-domain command (OSX/Linux/UNIX) using the ElasticSearch domain name returned at the previous step as identifier and custom query filters to determine if data-at-rest encryption feature is enabled for the selected domain:

aws es describe-elasticsearch-domain
	--region us-east-1
	--domain-name cc-prod-es-domain
	--query 'DomainStatus.EncryptionAtRestOptions'

04 The command output should return the requested ES feature status:

{
    "Enabled": false
}

If the Enabled flag value returned by the describe-elasticsearch-domain command output is false, as shown in the example above, the data-at-rest encryption is not enabled for the selected Amazon ElasticSearch domain, therefore the data stored on the domain file systems, primary and replica indices, log files, memory swap files and automated snapshots, is not protected from unauthorized access.

05 Repeat step no. 3 and 4 to verify the data-at-rest encryption status for other AWS ES domains available in the current region.

06 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 5 to perform the audit process for other regions.

Remediation / Resolution

To enable at-rest encryption for your existing AWS ElasticSearch domains, you must re-create them with the necessary encryption configuration. To relaunch the required ES domains, perform the following:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to ElasticSearch (ES) dashboard at https://console.aws.amazon.com/es/.

03 Choose the ES domain that you want to examine and click on the domain name (link) to access its configuration page.

04 On the domain configuration page, perform the following actions:

  1. Select the Overview tab and copy the domain configuration information such as Instance count, Instance type, Dedicated master instance type, Dedicated master instance count, Storage Type, EBS volume type, EBS volume size and so on.
  2. Select the VPC tab and copy the network configuration information such as VPC ID, Security groups ID(s), IAM role name and AZs and Subnets IDs.
  3. Click on Modify access policy button from the dashboard top menu and copy the policy document available in the Add or edit the access policy textbox.

05 Go back to the AWS ElasticSearch service dashboard and click Create new domain to launch a new ES domain.

06 On the Define domain page, perform the following actions:

  1. Provide a unique name for the new ES domain in the Elasticsearch domain name box.
  2. Select the right version of the Elasticsearch engine from the Elasticsearch version dropdown list. Note that at-rest encryption can be enabled only for domains with ElasticSearch version 5.1 and above.
  3. Click Next to continue the setup process.

07 On the Configure cluster page, perform the following:

  1. Set the new domain parameters using the configuration details copied at step no. 4 a.
  2. Select Enable encryption at rest checkbox to enable data-at-rest encryption feature for the new ES domain. From the KMS master key dropdown list choose whether to use the AWS managed-key (default encryption key) or to use your own AWS KMS customer-managed key to encrypt the ES data.
  3. Click Next to continue.

08 On the Set up access page, configure the network access to the new ES domain by using the same parameters copied at step no. 4 b. Also, paste the access policy copied at step no. 4 c. into the Add or edit the access policy textbox to use the same access policy as the source domain, then click the Next button to continue.

09 On the Review page, verify the domain configuration details then click Confirm to launch your new AWS ElasticSearch domain with data-at-rest encryption feature enabled.

10 Once the new AWS ES domain is created, upload the data from the source domain to the new (destination) ES cluster.

11 Now it’s safe to remove the source (unencrypted) ElasticSearch domain in order to stop incurring charges for it. To delete the necessary domain, perform the following:

  1. Click on the name of the domain that you want to remove (see Audit section part I to identify the right ElasticSearch resource).
  2. On the selected domain description page, click Delete domain to start the removal process.
  3. Within Delete domain dialog box, check Delete the domain <domain_name> then click the Delete button to confirm the action.

12 Repeat steps no. 3 - 11 to enable at-rest encryption for other AWS ES domains, available in the current region.

13 Change the AWS region from the navigation bar and repeat the entire process for other regions.

Using AWS CLI

01 Run describe-elasticsearch-domain command (OSX/Linux/UNIX) using the name of the ES domain that you want to relaunch (see Audit section part II to identify the right resource) to describe the configuration information for the selected domain:

aws es describe-elasticsearch-domain
	--region us-east-1
	--domain-name cc-prod-es-domain                                                          

02 The command output should return the configuration details for the selected AWS ES domain:

{
    "DomainStatus": {
        "ElasticsearchClusterConfig": {
            "DedicatedMasterEnabled": false,
            "InstanceCount": 2,
            "ZoneAwarenessEnabled": false,
            "InstanceType": "m4.large.elasticsearch"
        },
        "DomainId": "123456789012/cc-prod-es-domain",
        "Created": true,
        "Deleted": false,
        "DomainName": "cc-prod-es-domain",
 
        ...
 
        "SnapshotOptions": {
            "AutomatedSnapshotStartHour": 0
        },
        "ElasticsearchVersion": "5.5",
        "Processing": false,
        "Endpoints": {
            "vpc": "vpc-cc-prod-es-domain-aaaabbbbccccddddeeeeffff.us-east-1.es.amazonaws.com"
        },
        "EncryptionAtRestOptions": {
            "Enabled": false
        },
        "ARN": "arn:aws:es:us-east-1:123456789012:domain/cc-prod-es-domain"
    }
}

03 Run create-elasticsearch-domain command (OSX/Linux/UNIX) using the configuration metadata returned at the previous step to relaunch the selected AWS ElasticSearch domain and enable at-rest encryption feature using --encryption-at-rest-options parameter:

aws es create-elasticsearch-domain
	--region us-east-1
	--domain-name cc-prod-es-new-domain
	--elasticsearch-version 5.5
	--elasticsearch-cluster-config InstanceType=m4.large.elasticsearch,InstanceCount=2
	--ebs-options EBSEnabled=true,VolumeType=standard,VolumeSize=200
	--access-policies file://source-domain-access-policy.json
	--vpc-options SubnetIds=subnet-aaaabbbb,SecurityGroupIds=sg-ccccdddd
	--encryption-at-rest-options Enabled=true,KmsKeyId="aaaabbbb-cccc-dddd-eeee-aaaabbbbcccc"

04 The command output should return the metadata for the new AWS ElasticSearch domain:

{
    "DomainStatus": {
        "ElasticsearchClusterConfig": {
            "DedicatedMasterEnabled": false,
            "InstanceCount": 2,
            "ZoneAwarenessEnabled": false,
            "InstanceType": "m4.large.elasticsearch"
        },
        "DomainId": "123456789012/cc-prod-es-new-domain",
        "AdvancedOptions": {
            "rest.action.multi.allow_explicit_index": "true"
        },
 
        ...
       
        "SnapshotOptions": {
            "AutomatedSnapshotStartHour": 0
        },
        "ElasticsearchVersion": "5.5",
        "Processing": true,
        "Endpoints": {
            "vpc": "vpc-cc-prod-es-new-domain-bbbbccccddddeeeeffff.us-east-1.es.amazonaws.com"
        },
        "EncryptionAtRestOptions": {
            "Enabled": true,
            "KmsKeyId": "aaaabbbb-cccc-dddd-eeee-aaaabbbbcccc"
        },
        "ARN": "arn:aws:es:us-east-1:123456789012:domain/cc-prod-es-new-domain"
    }
}

05 Once the new AWS ES cluster is provisioned, upload the existing data (exported from the source ES cluster) to the newly created cluster.

06 After all the data is uploaded, it is safe to remove the source (unencrypted) ElasticSearch domain in order to stop incurring charges for the resource. To shut it down run delete-elasticsearch-domain command (OSX/Linux/UNIX) using the name of the domain that you want to delete as command parameter:

aws es delete-elasticsearch-domain
	--region us-east-1
	--domain-name cc-prod-es-domain

07 The command output should return the source AWS ElasticSearch domain metadata:

{
    "DomainStatus": {
        "ElasticsearchClusterConfig": {
            "DedicatedMasterEnabled": false,
            "InstanceCount": 2,
            "ZoneAwarenessEnabled": false,
            "InstanceType": "m4.large.elasticsearch"
        },
 
	  ...
 
        "AdvancedOptions": {
            "rest.action.multi.allow_explicit_index": "true",
            "indices.fielddata.cache.size": ""
        },
        "ElasticsearchVersion": "5.5",
        "ARN": "arn:aws:es:us-east-1:123456789012:domain/cc-prod-es-domain"
    }
}

08 Repeat steps no. 1 - 7 to enable at-rest encryption for other AWS ES domains, available in the current region.

09 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 – 8 to perform the process for other regions.

References

Publication date Dec 19, 2017