Elasticsearch Reserved Instance Recent Purchases

01 Sign in to AWS Management Console.

02 Navigate to Elasticsearch (ES) dashboard at https://console.aws.amazon.com/es/.

03 In the left navigation panel, choose Reserved Instances.

04 Select the Reserved Instance (RI) that you want to examine and check the values listed within Term (year) and Remaining Days columns. Based on these values calculate if the ES resource has been purchased recently (i.e. within the last 7 days). If the selected AWS Elasticsearch Reserved Instance has been purchased recently and you are unaware of this purchase, check your AWS CloudTrail logs or contact Amazon Web Services using the Support Center console to resolve the unwanted Elasticsearch RI purchase issue (see Remediation/Resolution section for more information).

05 Repeat step no. 4 to determine the purchase date for the rest of AWS Elasticsearch RIs available in the current region.

06 Change the AWS region from the navigation bar and repeat the process for other regions.

01 Run describe-reserved-elasticsearch-instances command (OSX/Linux/UNIX) using custom query filters to list the IDs of all Elasticsearch Reserved Instances available in the selected AWS region:

aws es describe-reserved-elasticsearch-instances
	--region us-east-1
	--output table
	--query 'ReservedElasticsearchInstances[*].ReservedElasticsearchInstanceId'

02 The command output should return a table with the requested ES RI IDs:

----------------------------------------
|   ReservedElasticsearchInstanceIds   |
+--------------------------------------+
| abcdabcd-abcd-1234-abcd-abcdabcdabcd |
| aaaaaaaa-bbbb-cccc-dddd-bbbbbbcccccc |
+--------------------------------------+

03 Run describe-reserved-elasticsearch-instances command (OSX/Linux/UNIX) using the ID of the Elasticsearch Reserved Instance that you want to examine as identifier and custom filtering to obtain the date when the selected RI was purchased:

aws es describe-reserved-elasticsearch-instances
	--region us-east-1
	--reserved-elasticsearch-instance-id abcdabcd-abcd-1234-abcd-abcdabcdabcd
	--query 'ReservedElasticsearchInstances[*].StartTime'

04 The command output should return the timestamp (date) at which the reservation started:

[
    "StartTime": 1539274423.430
]

05 The value returned for "StartTime" attribute at the previous step is using the Unix time format, which represents the number of seconds that have passed since midnight UTC of 1 January 1970. To convert the returned value into a human-readable format, run the following command (replace the Unix timestamp with your own timestamp returned at the previous step):

date -d @1507369500.430

06 The command output should return the RI purchase date in a human-readable format:

Thu Oct 11 16:13:43 UTC 2018

07 Repeat steps no. 3 – 6 to determine the purchase date for the rest of Amazon Elasticsearch RIs available in the current region.

08 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 – 7 to perform the entire audit process for other regions.

01 Sign in to AWS Management Console.

02 Navigate to CloudTrail dashboard at https://console.aws.amazon.com/cloudtrail/.

03 In the left navigation panel, select Trails.

04 Under Name column, select the trail name that you need to examine, available in the same AWS region with the identified Elasticsearch RI purchase (reservation).

05 Within Storage location section, check the name of the S3 bucket used to store the trail log data.

06 Navigate to S3 dashboard at https://console.aws.amazon.com/s3/.

07 Select the S3 bucket used for CloudTrail logging and use the date/time bucket name format (e.g. cloudtrail-logging-bucket/AWSLogs/123456789012/CloudTrail/us-east-1/2018/10/11) to open the right log file for analysis.

08 Based on the log file name (i.e. 123456789012_CloudTrail_us-east-1_20181011T1613Z_aaaabbbbccccdddd.json.gz), identify the CloudTrail log file that contains the API activity recorded on the same date as the unwanted Elasticsearch RI purchase request, click the Actions dropdown button from the dashboard top menu and select Open to download and open the log file in your web browser.

09 Once the right CloudTrail log file is opened, search for the following attributes in order to identify the necessary log record:

1. "eventSource":"es.amazonaws.com" – for the name of the AWS service used to place the RI purchase request.
2. "eventName":" PurchaseReservedElasticsearchInstance" – for the name of the AWS API action used to place the RI purchase request.
3. "eventTime":"2018-10-11T16:13:43.112Z" – for the time when the Elasticsearch RI purchase request was placed.

10 Identify the right CloudTrail log record based on the attributes listed at the previous step and verify the "userIdentity" attribute value to determine the unwanted Elasticsearch Reserved Instance purchase request origin and context.

11 Repeat steps no. 7 – 10 to verify the request origin and context for other unwanted Elasticsearch Reserved Node (RI) purchases placed in the selected region.

12 Change the AWS region from the navigation bar and repeat the process for other regions.

01 Run describe-trails command (OSX/Linux/UNIX) to list the names of all CloudTrail trails currently available within the selected AWS region:

aws cloudtrail describe-trails
	--region us-east-1
	--output table
	--query 'trailList[*].Name'

02 The command output should return a table with the requested trail name(s):

---------------------------
|     DescribeTrails      |
+-------------------------+
|    cc-prod-env-trail    |
+-------------------------+

03 Run again describe-trails command (OSX/Linux/UNIX) using the name of the trail returned at the previous step and custom query filters to get the name of the S3 bucket used to store the log files for the selected AWS CloudTrail trail:

aws cloudtrail describe-trails
	--region us-east-1
	--trail-name-list cc-prod-env-trail
	--query 'trailList[*].S3BucketName'

04 The command output should return the name of the requested S3 bucket:

[
    "cc-prod-env-trail-logs"
]

05 Now run list-objects command (OSX/Linux/UNIX) to list the names of all S3 objects available in the selected S3 bucket:

aws s3api list-objects
	--region us-east-1
	--bucket cc-prod-env-trail-logs
	--query 'Contents[].Key'

06 The command output should return the name of each S3 object (i.e. CloudTrail log file) currently available within the selected S3 bucket:

[
"AWSLogs/123456789012/CloudTrail/us-east-1/2018/10/11/123456789012_CloudTrail_us-east-1_20181011T1613Z_aaaabbbbccccdddd.json.gz",
     ...
"AWSLogs/123456789012/CloudTrail/us-east-1/2018/10/17/123456789012_CloudTrail_us-east-1_20181017T1235Z_abcdabcdabcdabcd.json.gz"

]

07 Run get-object command (OSX/Linux/UNIX) to get the appropriate CloudTrail log file (e.g. 20181011T1613Z_aaaabbbbccccdddd.json.gz) from the specified S3 bucket and download it to your machine:

aws s3api get-object
	--region us-east-1
	--bucket cc-prod-env-trail-logs
	--key AWSLogs/123456789012/CloudTrail/us-east-1/2018/10/11/123456789012_CloudTrail_us-east-1_20181011T1613Z_aaaabbbbccccdddd.json.gz 20181011T1613Z_aaaabbbbccccdddd.json.gz

08 The command output should return the GET request metadata:

{
    "AcceptRanges": "bytes",
    "ContentType": "application/json",
    "ContentLength": 5980,

     ...

    "ContentEncoding": "gzip",
    "ServerSideEncryption": "AES256",
    "Metadata": {}
}

09 Extract and open the required CloudTrail log file, downloaded at the previous step (e.g. 20181011T1613Z_aaaabbbbccccdddd.json.gz), in your favorite text editor.

10 Once the log file is opened, search for the following attributes in order to identify the necessary log record:

1. "eventSource":"es.amazonaws.com" – for the name of the AWS service used to place the RI purchase request.
2. "eventName":" PurchaseReservedElasticsearchInstance" – for the name of the AWS API action used to place the RI purchase request.
3. "eventTime":"2018-10-11T16:13:43.112Z" – for the time when the Elasticsearch RI purchase request was placed.

11 Identify the right CloudTrail log record based on the attributes listed at the previous step and verify the "userIdentity" attribute value to determine the unwanted Elasticsearch Reserved Instance purchase request origin and context.

12 Repeat steps no. 7 – 11 to verify the request origin and context for other unwanted Elasticsearch RI purchases available in the selected region.

13 Change the AWS region by updating the --region command parameter value and perform the entire remediation process for other regions.