Open menu
-->

Idle AWS EBS Volumes

Cloud Conformity allows you to automate the auditing process of this resolution page. Register for a 14 day evaluation and check your compliance level for free!

Start a Free Trial Product features
Cost
optimisation

Identify any non-root Amazon EBS volumes that appear to be idle and remove them from your account to help lower the cost of your monthly AWS bill. By default, an EBS volume is considered "idle" when meets the following criteria:

  • The total number of VolumeReadOps and VolumeWriteOps recorded per day for the last 7 days has been less than 1 (one) on average.

The AWS CloudWatch metrics used to detect idle EBS volumes are:
  • VolumeReadOps and VolumeWriteOps - the total number of disk I/O operations in a specified period of time (Units: Count).

Note 1: This rule can be applied to all AWS EBS volume types (gp2, io1, st1 and sc1). However, the rule excludes EBS boot (root) volumes.
Note 2: You can easily change the default threshold for this rule on Cloud Conformity console and set your own value for each condition, i.e. VolumeReadOps, VolumeWriteOps and recording time range (days), in order to configure the volumes idleness based on your requirements.
Note 3: For this rule Cloud Conformity assumes that your EBS volumes are tagged using a "Role" tag that provides visibility into their usage profile and help you decide whether it`s safe or not to delete them.
Note 4: Backup your data - once an EBS volume is deleted, the data available on it will be completely lost. Since EBS snapshots are much more cost-effective because the volume data is stored as an object by the AWS S3 service, creating volume snapshots before removal is highly recommended.

This rule resolution is part of the Cloud Conformity Cost Optimisation Package

Idle EBS volumes represent a good candidate to reduce your monthly AWS costs and avoid accumulating unnecessary usage charges.

Audit

To identify any non-root idle AWS EBS volumes currently available in your AWS account, perform the following:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to EC2 dashboard at https://console.aws.amazon.com/ec2/.

03 In the left navigation panel, under ELASTIC BLOCK STORE section, choose Volumes.

04 On the EBS volumes listing page, click inside the attributes filter box located under the dashboard top menu, choose State parameter from the dropdown list and select In use. This filtering method will return only the running (in use) EBS volumes provisioned within the current region.

05 Select the EBS volume that you want to examine.

06 Select the Description tab from the dashboard bottom panel and check the Attachment information attribute value. If the attachment information for the selected EBS volume contains the /dev/sda1

EBS volume contains the /dev/sda1

or /dev/xvda

EBS volume contains the /dev/xvda

device name, the verified EBS resource is a root volume and the rule audit process must be restarted with the step no. 5, otherwise continue with the next step.

07 Select the Monitoring tab from the dashboard bottom panel.

08 Within the CloudWatch metrics section, perform the following actions:

  1. Click on the Read Throughput (Ops/s) usage graph thumbnail to open the volume Read I/O usage details box. Inside the CloudWatch Monitoring Details dialog box, set the following parameters:
    • From the Statistic dropdown list, select Sum.
    • From the Time Range list, select Last 1 Week.
    • From the Period dropdown list, select 1 Hour.

    Once the monitoring data is loaded, verify the EBS volume read throughput (i.e. VolumeReadOps) usage recorded in the last 7 days. If the total number of VolumeReadOps has been less than 1, e.g. If the total number of VolumeReadOps has been less than 1, the selected EBS volume qualifies as candidate for the idle resource. Click Close to return to the dashboard.

09 Now determine the role assigned to the selected EBS volume in order to decide whether it's safe or not to delete the volume. To verify the volume Role tag value, perform the following:

  1. Select the Tags tab from the dashboard bottom panel.
  2. Check the Role tag value, available in the Value column, or any Role-like tag value that can provide information about the usage profile of the EBS volume (e.g. staging-web-server-volume) in order to decide if the volume can be removed or not.

If all the conditions outlined at step no. 7 and 8 are met, the selected AWS EBS volume is considered "idle" and can be safely removed from your AWS account in order to stop incurring charges for the EBS resource.

10 Repeat steps no. 5 – 9 to verify the usage profile (i.e. role) and the associated CloudWatch metrics (VolumeReadOps and VolumeWriteOps) for other EBS volumes available in the current region.

11 Change the AWS region from the navigation bar and repeat the audit process for the other regions.

Using AWS CLI

01 Run describe-volumes command (OSX/Linux/UNIX) using custom query filters based on JMESPath query language (integrated by AWS CLI), to list the IDs of all non-root, running (i.e. in use) EBS volumes, available within the selected region:

aws ec2 describe-volumes
	--region us-east-1
	--filters Name=status,Values=in-use
	--output table
	--query 'Volumes[].Attachments[?(Device!=`/dev/xvda`) && (Device!=`/dev/sda1`)].VolumeId | []'

02 The command output should return a table with the requested volume IDs:

---------------------------
|     DescribeVolumes     |
+-------------------------+
|  vol-05fd4936c6b5bc413  |
|  vol-0c005f19fa4ac7c0e  |
|  vol-0acb9f6580f38caf5  |
+-------------------------+

03 Run get-metric-statistics command (OSX/Linux/UNIX) to get the statistics recorded by AWS CloudWatch for the VolumeReadOps metric representing the EBS volume read throughput. The following command example returns the total read throughput usage captured within the 7-day time frame for an EBS volume identified by the ID vol-05fd4936c6b5bc413, using 1 hour as the granularity for the returned datapoints:

aws cloudwatch get-metric-statistics
	--region us-east-1
	--metric-name VolumeReadOps
	--start-time 2017-03-04T12:15:00
	--end-time 2017-03-11T12:15:00
	--period 3600
	--namespace AWS/EBS
	--statistics Sum
	--dimensions Name=VolumeId,Value=vol-05fd4936c6b5bc413

04 The command output should return the volume read throughput usage data requested:

{
    "Datapoints": [
        {
            "Timestamp": "2017-03-04T12:15:00Z",
            "Sum": 0.1050,
            "Unit": "Count"
        },
        {
            "Timestamp": "2017-03-04T13:15:00Z",
            "Sum": 0.10425,
            "Unit": "Count"
        },
        {
            "Timestamp": "2017-03-04T14:15:00Z",
            "Sum": 0.033499999999999995,
            "Unit": "Count"
        },

        ...

        {
            "Timestamp": "2017-03-11T10:15:00Z",
            "Sum": 0.027833333333333333,
            "Unit": "Count"
        },
        {
            "Timestamp": "2017-03-11T11:15:00Z",
            "Sum": 0.033499999999999995,
            "Unit": "Count"
        },
        {
            "Timestamp": "2017-03-11T12:15:00Z",
            "Sum": 0.033799999999999995,
            "Unit": "Count"
        }
    ],
    "Label": "VolumeReadOps"
}

If the total number of VolumeReadOps has been less than 1 (one), the selected EBS volume qualifies as candidate for the idle volume.

05 Run again get-metric-statistics command (OSX/Linux/UNIX) to retrieve the statistics recorded by Amazon CloudWatch service for the VolumeWriteOps metric, representing the EBS volume write throughput. The following command example returns the total write throughput usage recorded within the 7-day time frame for an EBS volume identified by the ID vol-05fd4936c6b5bc413, using 1 hour as the granularity for the returned datapoints:

aws cloudwatch get-metric-statistics
	--region us-east-1
	--metric-name VolumeWriteOps
	--start-time 2017-03-04T12:15:00
	--end-time 2017-03-11T12:15:00
	--period 3600
	--namespace AWS/EBS
	--statistics Sum
	--dimensions Name=VolumeId,Value=vol-05fd4936c6b5bc413

06 The command output should return the volume write throughput usage details requested:

{
    "Datapoints": [
        {
            "Timestamp": "2017-03-04T12:15:00Z",
            "Sum": 0.0,
            "Unit": "Count"
        },
        {
            "Timestamp": "2017-03-04T13:15:00Z",
            "Sum": 0.0,
            "Unit": "Count"
        },
        {
            "Timestamp": "2017-03-04T14:15:00Z",
            "Sum": 0.0,
            "Unit": "Count"
        },

        ...

        {
            "Timestamp": "2017-03-11T10:15:00Z",
            "Sum": 0.056733333333333333,
            "Unit": "Count"
        },
        {
            "Timestamp": "2017-03-11T11:15:00Z",
            "Sum": 0.054099999999999995,
            "Unit": "Count"
        },
        {
            "Timestamp": "2017-03-11T12:15:00Z",
            "Sum": 0.038999999999999995,
            "Unit": "Count"
        }
    ],
    "Label": "VolumeWriteOps"
}

If the total number of VolumeWriteOps has been less than 1, the selected EBS volume qualifies as candidate for idleness.

07 Run describe-tags command (OSX/Linux/UNIX) to describe the tags for the selected EBS volume. These tags are used to determine the volume role within your application stack (e.g. staging-server-block-volume) in order to decide whether it's safe or not to delete the volume:

aws ec2 describe-tags
	--region us-east-1
	--filters "Name=resource-id,Values=vol-05fd4936c6b5bc413"

08 The command output should return the EBS volume tags available. Check for any "Role" or Role-like tags to determine the volume usage profile:

{
    "Tags": [
        {
            "ResourceType": "volume",
            "ResourceId": "vol-05fd4936c6b5bc413",
            "Value": "Staging",
            "Key": "Environment"
        },
        {
            "ResourceType": "volume",
            "ResourceId": "vol-05fd4936c6b5bc413",
            "Value": "cc-webapp-server-volume",
            "Key": "Name"
        },
        {
            "ResourceType": "volume",
            "ResourceId": "vol-05fd4936c6b5bc413",
            "Value": "staging-server-block-volume",
            "Key": "Role"
        }
    ]
}

If the data returned for the steps no. 3 - 8 satisfy the conformity rule conditions (i.e. total read/write throughput usage recorded within a week), the selected EBS volume is considered "idle" and can be safely removed from the AWS account to reduce the EBS monthly costs.

09 Repeat steps no. 3 – 8 to verify the usage profile and the associated CloudWatch metrics (VolumeReadOps and VolumeWriteOps) for other EBS volumes provisioned within the current region.

10 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 9 to perform the entire audit process for other regions.

Remediation / Resolution

Option 1: Delete any AWS EBS non-root volumes that are currently running in the idle mode. To remove the specified volumes, perform the following actions:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to EC2 dashboard at https://console.aws.amazon.com/ec2/.

03 In the left navigation panel, under ELASTIC BLOCK STORE section, choose Volumes.

04 Select the EBS volume that you want to remove (check Audit section part I to identify the right resource).

05 Click the Actions dropdown button from the dashboard top menu and select Create Snapshot to take a full snapshot of the volume before removal.

06 Within the Create Snapshot dialog box, provide a unique name and a short description (optional) for your new EBS volume snapshot, then click Create to initiate the backup process. Now click Close to return to the EBS dashboard.

07 Use again the Actions dropdown button from the dashboard top menu and select Delete Volume.

08 In the Delete Volume dialog box, review the selected EBS volume details then click Yes, Delete to confirm the action.

09 Repeat steps no. 4 – 8 to delete any other idle EBS volumes available within the current region.

10 Change the AWS region from the navigation bar and repeat the remediation/resolution process for other regions.

Using AWS CLI

01 Run create-snapshot command (OSX/Linux/UNIX) to create a final snapshot for the specified idle EBS volume before removal (check Audit section part II to identify the right volume):

aws ec2 create-snapshot
	--region us-east-1
	--volume-id vol-05fd4936c6b5bc413
	--description "CC Staging Server Block Volume Snapshot"

02 The command output should return the EBS volume snapshot metadata:

{
    "Description": "CC Staging Server Block Volume Snapshot",
    "Encrypted": false,
    "VolumeId": "vol-05fd4936c6b5bc413",
    "State": "pending",
    "VolumeSize": 30,
    "Progress": "",
    "StartTime": "2017-03-11T14:49:11.000Z",
    "SnapshotId": "snap-0b374ce6ac5338e52",
    "OwnerId": "123456789012"
}

03 Run delete-volume command (OSX/Linux/UNIX) to remove the selected idle EBS volume from your AWS account. The following command example deletes a non-root EBS volume identified by the ID vol-05fd4936c6b5bc413, available within the US East (N. Virginia) region (the command does not produce an output):

aws ec2 delete-volume
	--region us-east-1
	--volume-id vol-05fd4936c6b5bc413

04 Repeat steps no. 1 – 3 to remove any other idle EBS volumes provisioned within the current region.

05 Change the AWS region by updating the --region command parameter value and repeat the entire process for other regions.

Option 2: Disable the rule check. If the selected idle EBS volume is needed (its role within your application stack/environment is important), you may want turn off the conformity rule check for the specified EBS volume from the Cloud Conformity console.

References

Publication date Mar 13, 2017