Elasticsearch Dedicated Master Enabled
Cloud Conformity allows you to automate the auditing process of this resolution page. Register for a 14 day evaluation and check your compliance level for free!
Start a Free Trial Product features
Cloud Conformity allows you to automate the auditing process of this resolution page. Register for a 14 day evaluation and check your compliance level for free!
Start a Free Trial Product featuresEnsure that your AWS Elasticsearch Service (ES) clusters are using dedicated master nodes to improve their environmental stability by offloading all the management tasks from the cluster data nodes.
This rule resolution is part of the Cloud Conformity Security & Compliance tool for AWS
Using Elasticsearch dedicated master nodes to separate management tasks from index and search requests will improve the clusters ability to manage easily different types of workload and make them more resilient in production.
Note 1: Because ES dedicated master nodes do not process search and query requests nor hold any data, the node type chosen for this role typically does not require a large amount of CPU or RAM memory. Cloud Conformity recommends starting with the m3.medium.elasticsearch node type then adjust as necessary.
Note 2: Ensure you allocate at least 3 dedicated master nodes for each Elasticsearch domain (cluster) running in production. The default value for the number of master nodes is set to 3 but this value can be adjusted in the rule settings on the Cloud Conformity console.
To determine if your Elasticsearch clusters are using dedicated master nodes, perform the following:
01 Login to the AWS Management Console.
02 Navigate to Elasticsearch (ES) dashboard at https://console.aws.amazon.com/es/.
03 Click on the ES domain that you want to examine, e.g.
A domain is a collection of resources required to run an Amazon Elasticsearch cluster.
04 On the selected domain description page, click the Configure cluster button from the dashboard top menu to access the cluster configuration page.
05 On the Configure cluster page, in the Node configuration section, verify the Enable dedicated master setting status. If Enable dedicated master checkbox is unchecked:
the selected ES cluster does not have any dedicated mater nodes enabled, therefore the load/search requests and the management tasks are all handled by the cluster data nodes.
06 Repeat steps no. 3 - 5 to verify if other Elasticsearch domains (clusters) available in the current region have dedicated master nodes in use.
07 Change the AWS region from the navigation bar and repeat the process for the other regions.
01 Run list-domain-names command (OSX/Linux/UNIX) to list the names of all AWS Elasticsearch (ES) domains currently available in the selected region:
aws es list-domain-names --region us-east-1
02 The command output should return the requested ES domain name(s):
{
"DomainNames": [
{
"DomainName": "cloudconformity-es-cluster"
},
{
"DomainName": "elastisearch-cluster-prod"
},
{
"DomainName": "ecom-analytics-es-cluster"
}
]
}
03 Run describe-elasticsearch-domain command (OSX/Linux/UNIX) using the ES domain name returned at the previous step and custom query filters to expose the ES dedicated master node(s) configuration for the selected domain (cluster):
aws es describe-elasticsearch-domain --domain-name cloudconformity-es-cluster --region us-east-1 --query 'DomainStatus.ElasticsearchClusterConfig.DedicatedMasterEnabled'
04 The command output should return the dedicated master node(s) configuration status (true for enabled, false for disabled):
>false
05 Repeat steps no. 3 and 4 to determine if other Elasticsearch domains (clusters) available in the current region are using dedicated master nodes.
06 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 5 to perform the audit process for other regions.
To enable dedicated master nodes for your Amazon Elasticsearch clusters, perform the following:
01 Login to the AWS Management Console.
02 Navigate to Elasticsearch (ES) dashboard at https://console.aws.amazon.com/es/.
03 Click on the ES domain that you want to reconfigure (see Audit section part I to identify the right AWS resource).
04 On the selected ES domain description page, click the Configure cluster button from the dashboard top menu to open the cluster configuration page.
05 On the Configure cluster page, in the Node configuration section, perform the following:
06 Click Submit to add the dedicated master nodes to your ES cluster.
07 In the Change cluster configuration dialog box, click OK to confirm the action. The ES domain status should change from Active to Processing and back to Active once the service finishes processing your configuration changes (i.e. provisioning and attaching the dedicated master nodes).
08 Repeat steps no. 3 - 7 to enable dedicated master nodes for other Elasticsearch clusters available in the current region.
09 Change the AWS region from the navigation bar and repeat the process for other regions.
01 Run update-elasticsearch-domain-config command (OSX/Linux/UNIX) using the name of the Elasticsearch domain that you want to reconfigure (see Audit section part II to identify the right ES resource) to enable dedicated master nodes (i.e. DedicatedMasterEnabled=true) for the selected ES cluster:
aws es update-elasticsearch-domain-config --domain-name cloudconformity-es-cluster --region us-east-1 --ebs-options EBSEnabled=true,VolumeType="gp2",VolumeSize=100 --elasticsearch-cluster-config InstanceType="m3.large.elasticsearch",InstanceCount=2,ZoneAwarenessEnabled=true,DedicatedMasterEnabled=true,DedicatedMasterType="m3.medium.elasticsearch",DedicatedMasterCount=3
02 The command output should return the new configuration metadata for the modified Elasticsearch domain (cluster):
{
"DomainConfig": {
"ElasticsearchClusterConfig": {
"Status": {
"PendingDeletion": false,
"State": "Processing",
"CreationDate": 1480581626.057,
"UpdateVersion": 9,
"UpdateDate": 1480615796.77
},
"Options": {
"DedicatedMasterEnabled": true,
"InstanceCount": 2,
"ZoneAwarenessEnabled": true,
"DedicatedMasterType": "m3.medium.elasticsearch",
"InstanceType": "m3.large.elasticsearch",
"DedicatedMasterCount": 3
}
},
...
"AdvancedOptions": {
"Status": {
"PendingDeletion": false,
"State": "Active",
"CreationDate": 1480581626.057,
"UpdateVersion": 5,
"UpdateDate": 1480582118.479
},
"Options": {
"rest.action.multi.allow_explicit_index": "true",
"indices.fielddata.cache.size": ""
}
},
"AccessPolicies": {
"Status": {
"PendingDeletion": false,
"State": "Active",
"CreationDate": 1480581626.057,
"UpdateVersion": 5,
"UpdateDate": 1480582118.479
},
"Options": "{ ... }"
}
}
}
03 Repeat step no. 1 and 2 to enable dedicated master nodes for other Elasticsearch clusters available in the current region.
04 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 3 to perform the entire process for other regions.
Whether your AWS exploration is just starting to take shape, you’re mid-way through a migration or you’re already running complex workloads in the cloud, Cloud Conformity offers full visibility of your infrastructure and provides continuous assurance it’s secure, optimized and compliant.
Chat with us to set up your onboarding session and start a free trial.
Let’s Chat