|   Trend Micro Cloud One™
Open menu

Elasticsearch Reserved Instance Recent Purchases

Cloud Conformity allows you to automate the auditing process of this resolution page. Register for a 14 day evaluation and check your compliance level for free!

Start a Free Trial Product features
Knowledge Base Amazon Web Services Amazon Elasticsearch Service Elasticsearch Reserved Instance Recent Purchases
Last updated: 13 May 2020
Risk level: Low (generally tolerable level of risk)
Rule ID: ES-020

Ensure that all active Amazon Elasticsearch (ES) Reserved Instance purchases are reviewed every 7 days to make sure that no unwanted RI purchase has been placed recently.

This rule can help you work with the AWS Well-Architected Framework

This rule resolution is part of the Cloud Conformity Security & Compliance tool for AWS

Cost
optimisation

By verifying your Elasticsearch Reserved Instance purchases on a regular basis you can detect and cancel any unwanted purchases placed accidentally or intentionally within your AWS account in order to avoid unexpected charges on your AWS bill.

Note: You can change the default threshold value (i.e. 7 days) set for the review time frame within the conformity rule settings, on your Cloud Conformity account console.

Audit

To identify the active Elasticsearch Reserved Instance purchases placed recently within your AWS account for review purposes, perform the following:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to Elasticsearch (ES) dashboard at https://console.aws.amazon.com/es/.

03 In the left navigation panel, choose Reserved Instances.

04 Select the Reserved Instance (RI) that you want to examine and check the values listed within Term (year) and Remaining Days columns. Based on these values calculate if the ES resource has been purchased recently (i.e. within the last 7 days). If the selected AWS Elasticsearch Reserved Instance has been purchased recently and you are unaware of this purchase, check your AWS CloudTrail logs or contact Amazon Web Services using the Support Center console to resolve the unwanted Elasticsearch RI purchase issue (see Remediation/Resolution section for more information).

05 Repeat step no. 4 to determine the purchase date for the rest of AWS Elasticsearch RIs available in the current region.

06 Change the AWS region from the navigation bar and repeat the process for other regions.

Using AWS CLI

01 Run describe-reserved-elasticsearch-instances command (OSX/Linux/UNIX) using custom query filters to list the IDs of all Elasticsearch Reserved Instances available in the selected AWS region: 

aws es describe-reserved-elasticsearch-instances
	--region us-east-1
	--output table
	--query 'ReservedElasticsearchInstances[*].ReservedElasticsearchInstanceId'

02 The command output should return a table with the requested ES RI IDs: 

----------------------------------------
|   ReservedElasticsearchInstanceIds   |
+--------------------------------------+
| abcdabcd-abcd-1234-abcd-abcdabcdabcd |
| aaaaaaaa-bbbb-cccc-dddd-bbbbbbcccccc |
+--------------------------------------+

03 Run describe-reserved-elasticsearch-instances command (OSX/Linux/UNIX) using the ID of the Elasticsearch Reserved Instance that you want to examine as identifier and custom filtering to obtain the date when the selected RI was purchased: 

aws es describe-reserved-elasticsearch-instances
	--region us-east-1
	--reserved-elasticsearch-instance-id abcdabcd-abcd-1234-abcd-abcdabcdabcd
	--query 'ReservedElasticsearchInstances[*].StartTime'

04 The command output should return the timestamp (date) at which the reservation started: 

[
    "StartTime": 1539274423.430
]

05 The value returned for "StartTime" attribute at the previous step is using the Unix time format, which represents the number of seconds that have passed since midnight UTC of 1 January 1970. To convert the returned value into a human-readable format, run the following command (replace the Unix timestamp with your own timestamp returned at the previous step): 

date -d @1507369500.430

06 The command output should return the RI purchase date in a human-readable format: 

Thu Oct 11 16:13:43 UTC 2018

If the date returned by the command output indicates a recent Elasticsearch Reserved Instance purchase request (i.e. request placed within the last 7 days) and you are unaware of this purchase, check your AWS CloudTrail logs or contact Amazon Web Services using the Support Center console to solve the unwanted Elasticsearch RI purchase problem.

07 Repeat steps no. 3 – 6 to determine the purchase date for the rest of Amazon Elasticsearch RIs available in the current region.

08 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 – 7 to perform the entire audit process for other regions.

Remediation / Resolution

Case A: Verify Amazon CloudTrail logs from the date when the Elasticsearch RI purchase request was placed to determine the request origin and context. To find and analyze the necessary API logging data, perform the following actions:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to CloudTrail dashboard at https://console.aws.amazon.com/cloudtrail/.

03 In the left navigation panel, select Trails.

04 Under Name column, select the trail name that you need to examine, available in the same AWS region with the identified Elasticsearch RI purchase (reservation).

05 Within Storage location section, check the name of the S3 bucket used to store the trail log data.

06 Navigate to S3 dashboard at https://console.aws.amazon.com/s3/.

07 Select the S3 bucket used for CloudTrail logging and use the date/time bucket name format (e.g. cloudtrail-logging-bucket/AWSLogs/123456789012/CloudTrail/us-east-1/2018/10/11) to open the right log file for analysis.

08 Based on the log file name (i.e. 123456789012_CloudTrail_us-east-1_20181011T1613Z_aaaabbbbccccdddd.json.gz), identify the CloudTrail log file that contains the API activity recorded on the same date as the unwanted Elasticsearch RI purchase request, click the Actions dropdown button from the dashboard top menu and select Open to download and open the log file in your web browser.

09 Once the right CloudTrail log file is opened, search for the following attributes in order to identify the necessary log record:

  1. "eventSource":"es.amazonaws.com" – for the name of the AWS service used to place the RI purchase request.
  2. "eventName":" PurchaseReservedElasticsearchInstance" – for the name of the AWS API action used to place the RI purchase request.
  3. "eventTime":"2018-10-11T16:13:43.112Z" – for the time when the Elasticsearch RI purchase request was placed.

10 Identify the right CloudTrail log record based on the attributes listed at the previous step and verify the "userIdentity" attribute value to determine the unwanted Elasticsearch Reserved Instance purchase request origin and context.

11 Repeat steps no. 7 – 10 to verify the request origin and context for other unwanted Elasticsearch Reserved Node (RI) purchases placed in the selected region.

12 Change the AWS region from the navigation bar and repeat the process for other regions.

Using AWS CLI

01 Run describe-trails command (OSX/Linux/UNIX) to list the names of all CloudTrail trails currently available within the selected AWS region: 

aws cloudtrail describe-trails
	--region us-east-1
	--output table
	--query 'trailList[*].Name'

02 The command output should return a table with the requested trail name(s): 

---------------------------
|     DescribeTrails      |
+-------------------------+
|    cc-prod-env-trail    |
+-------------------------+

03 Run again describe-trails command (OSX/Linux/UNIX) using the name of the trail returned at the previous step and custom query filters to get the name of the S3 bucket used to store the log files for the selected AWS CloudTrail trail: 

aws cloudtrail describe-trails
	--region us-east-1
	--trail-name-list cc-prod-env-trail
	--query 'trailList[*].S3BucketName'

04 The command output should return the name of the requested S3 bucket: 

[
    "cc-prod-env-trail-logs"
]

05 Now run list-objects command (OSX/Linux/UNIX) to list the names of all S3 objects available in the selected S3 bucket: 

aws s3api list-objects
	--region us-east-1
	--bucket cc-prod-env-trail-logs
	--query 'Contents[].Key'

06 The command output should return the name of each S3 object (i.e. CloudTrail log file) currently available within the selected S3 bucket: 

[
"AWSLogs/123456789012/CloudTrail/us-east-1/2018/10/11/123456789012_CloudTrail_us-east-1_20181011T1613Z_aaaabbbbccccdddd.json.gz",
     ...
"AWSLogs/123456789012/CloudTrail/us-east-1/2018/10/17/123456789012_CloudTrail_us-east-1_20181017T1235Z_abcdabcdabcdabcd.json.gz"

]

07 Run get-object command (OSX/Linux/UNIX) to get the appropriate CloudTrail log file (e.g. 20181011T1613Z_aaaabbbbccccdddd.json.gz) from the specified S3 bucket and download it to your machine: 

aws s3api get-object
	--region us-east-1
	--bucket cc-prod-env-trail-logs
	--key AWSLogs/123456789012/CloudTrail/us-east-1/2018/10/11/123456789012_CloudTrail_us-east-1_20181011T1613Z_aaaabbbbccccdddd.json.gz 20181011T1613Z_aaaabbbbccccdddd.json.gz

08 The command output should return the GET request metadata: 

{
    "AcceptRanges": "bytes",
    "ContentType": "application/json",
    "ContentLength": 5980,

     ...

    "ContentEncoding": "gzip",
    "ServerSideEncryption": "AES256",
    "Metadata": {}
}

09 Extract and open the required CloudTrail log file, downloaded at the previous step (e.g. 20181011T1613Z_aaaabbbbccccdddd.json.gz), in your favorite text editor.

10 Once the log file is opened, search for the following attributes in order to identify the necessary log record:

  1. "eventSource":"es.amazonaws.com" – for the name of the AWS service used to place the RI purchase request.
  2. "eventName":" PurchaseReservedElasticsearchInstance" – for the name of the AWS API action used to place the RI purchase request.
  3. "eventTime":"2018-10-11T16:13:43.112Z" – for the time when the Elasticsearch RI purchase request was placed.

11 Identify the right CloudTrail log record based on the attributes listed at the previous step and verify the "userIdentity" attribute value to determine the unwanted Elasticsearch Reserved Instance purchase request origin and context.

12 Repeat steps no. 7 – 11 to verify the request origin and context for other unwanted Elasticsearch RI purchases available in the selected region.

13 Change the AWS region by updating the --region command parameter value and perform the entire remediation process for other regions.

Case B: To mitigate unwanted AWS Elasticsearch Reserved Instance purchase requests you can contact Amazon Web Services and request the RI purchase cancellation. To create the necessary case using the AWS Support Center console, perform the following:

Note: Requesting Amazon to cancel unwanted Elasticsearch Reserved Instance purchase requests using AWS Management Console or AWS API via Command Line Interface (CLI) is not currently supported.

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to AWS Support Center page at https://console.aws.amazon.com/support/.

03 On Support Center page, in the My support cases panel, click Create case to open the support case form.

04 On the Create Case page, perform the following actions:

  1. Under Regarding, select Account and Billing Support option.
  2. Choose Billing from the Service dropdown list to send your request to AWS Billing and Cost Management service.
  3. Select Reserved Instances from the Category dropdown list.
  4. Inside the Subject box, enter a subject for your request such as "Cancel unwanted AWS Elasticsearch Reserved Instance purchase".
  5. Within Description textbox, provide the reason why do you need to cancel your recent Elasticsearch RI purchase and explain in detail how and when this unwanted purchase request was placed. This will help AWS support to evaluate properly your request.
  6. Under Contact method, select a preferred contact method that AWS support team can use to respond to your request.
  7. Click Submit to send the cancellation request for your unwanted Elasticsearch Reserved Instance purchase to Amazon Web Services.

References

Publication date Oct 29, 2018

Related Elasticsearch rules

Unlock the Remediation Steps

Gain free unlimited access to our full Knowledge Base

Over 600 rules & best practices for and

Get started for FREE

A verification email will be sent to this address
We keep your information private. Learn more.

Thank you!

Please click the link in the confirmation email sent to

You are auditing:

Elasticsearch Reserved Instance Recent Purchases

Risk level: Low