01 First, execute create-log-group command (OSX/Linux/UNIX) to create the necessary AWS CloudWatch log group within the selected region (the command does not produce an output):
aws logs create-log-group
--region us-east-1
--log-group-name cc-cloudwatch-log-group
02 Run describe-log-groups command (OSX/Linux/UNIX) using the name of the newly created CloudWatch log group and custom query filters to expose the CloudWatch resource ARN:
aws logs describe-log-groups
--region us-east-1
--log-group-name cc-cloudwatch-log-group
--query 'logGroups[*].arn'
03 The command output should return the requested log group ARN:
[
"arn:aws:logs:us-east-1:123456789012:log-group:cc-cloudwatch-log-group:*"
]
04 Now execute put-resource-policy command (OSX/Linux/UNIX) to give Amazon Elasticsearch permissions to write to the CloudWatch log group created at step no. 1:
aws logs put-resource-policy
--region us-east-1
--policy-name es-slow-logs-policy
--policy-document '{ "Version": "2012-10-17", "Statement": [{ "Sid": "", "Effect": "Allow", "Principal": { "Service": "es.amazonaws.com"}, "Action":[ "logs:PutLogEvents"," logs:PutLogEventsBatch","logs:CreateLogStream"],"Resource": "arn:aws:logs:us-east-1:123456789012:log-group:cc-cloudwatch-log-group:*"}]}'
05 The command output should return the command request metadata (including information about the access policy used):
{
"resourcePolicy": {
"policyName": "es-slow-logs-policy",
"policyDocument": "{ \"Version\": \"2012-10-17\", \"Statement\": [{ \"Sid\": \"\", \"Effect\": \"Allow\", \"Principal\": { \"Service\": \"es.amazonaws.com\"}, \"Action\":[ \"logs:PutLogEvents\",\" logs:PutLogEventsBatch\",\"logs:CreateLogStream\"],\"Resource\": \"arn:aws:logs:us-east-1:123456789012:log-group:cc-cloudwatch-log-group:*\"}]}",
"lastUpdatedTime": 1510081902775
}
}
06 Run update-elasticsearch-domain-config command (OSX/Linux/UNIX) to update the cluster configuration and enable the publishing of search and index slow logs for the specified AWS ES domain:
aws es update-elasticsearch-domain-config
--region us-east-1
--domain-name cc-es-cluster-v5
--log-publishing-options "SEARCH_SLOW_LOGS={CloudWatchLogsLogGroupArn=arn:aws:logs:us-east-1:123456789012:log-group:cc-cloudwatch-log-group:*,Enabled=true},INDEX_SLOW_LOGS={CloudWatchLogsLogGroupArn=arn:aws:logs:us-east-1:123456789012:log-group:cc-cloudwatch-log-group:*,Enabled=true}"
07 The command output should return the new configuration metadata for the selected AWS ES domain:
{
"DomainConfig": {
"ElasticsearchClusterConfig": {
"Status": {
"PendingDeletion": false,
"State": "Active",
"CreationDate": 1510072997,
"UpdateVersion": 5,
"UpdateDate": 1510073688
},
"Options": {
"DedicatedMasterEnabled": false,
"InstanceCount": 2,
"ZoneAwarenessEnabled": false,
"InstanceType": "c4.xlarge.elasticsearch"
}
},
...
"LogPublishingOptions": {
"Status": {
"PendingDeletion": false,
"State": "Processing",
"CreationDate": 1510077571,
"UpdateVersion": 23,
"UpdateDate": 1510081936
},
"Options": {
"INDEX_SLOW_LOGS": {
"CloudWatchLogsLogGroupArn": "arn:aws:logs:us-east-1:123456789012:log-group:cc-cloudwatch-log-group:*",
"Enabled": true
},
"SEARCH_SLOW_LOGS": {
"CloudWatchLogsLogGroupArn": "arn:aws:logs:us-east-1:123456789012:log-group:cc-cloudwatch-log-group:*",
"Enabled": true
}
}
}
}
}
08 Repeat steps no. 1 – 7 to enable search and index slow logs publishing to AWS CloudWatch for other AWS ES domains available in the current region.
09 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 8 to perform the entire process for other regions.