RDS General Purpose SSD

Risk level: Medium (should be achieved)

Rule ID: RDS-010

Ensure that your RDS instances are using General Purpose SSDs instead of Provisioned IOPS SSDs for cost-effective storage that fits a broad range of database workloads. Unless you are running mission-critical applications that require more than 10000 IOPS or 160 MiB/s of throughput per database, Cloud Conformity recommends converting your Provisioned IOPS RDS instances to General Purpose instances in order to lower the cost of your monthly AWS bill while keeping the same I/O performance. For example, if the requirements for your MySQL database are 150GB storage size and 1500 IOPS (input/output operations per second) you can save roughly $110 per month using instances with General Purpose SSD storage instead of ones with Provisioned IOPS SSD storage as the former (1500 IOPS, 150GB storage) costs $234.63/month while the latter (1500 IOPS, 500GB storage) costs only $123.38/month (as of November 2016).

This rule can help you work with the AWS Well-Architected Framework

This rule resolution is part of the Cloud Conformity Security & Compliance tool for AWS

Cost
optimisation

Using General Purpose (GP) SSD database storage instead of Provisioned IOPS (PIOPS) SSD storage represents a good strategy to cut down on AWS RDS costs because for GP SSDs you only pay for the storage compared to PIOPS SSDs where you pay for both storage and IOPS. Converting existing PIOPS-based databases to GP is often possible by configuring larger storage which gives higher baseline performance of IOPS for a lower cost.

Note: Backing up your RDS database instances before converting them to General Purpose SSD is highly recommended because once these instances are modified, all their automated backups (snapshots) are removed.

Audit

To determine the type of your existing RDS database instances, perform the following:

Using AWS Console

01 Login to the AWS Management Console.

02 Navigate to RDS dashboard at https://console.aws.amazon.com/rds/.

03 In the left navigation panel, under RDS Dashboard, click Instances.

04 Select the RDS instance that you want to examine.

05 Click Instance Actions button from the dashboard top menu and select See Details.

06 On the Details tab, in the Instance and IOPS section, check the Storage Type property value. If the current value is set to Provisioned IOPS (SSD):

the storage type used is Provisioned IOPS SSD, therefore the selected RDS instance is not using the most cost-effective storage type available. To convert a PIOPS-based database instance to a GP one, follow the steps outlined in the Remediation/Resolution section of the conformity rule.

07 Repeat steps no. 4 - 6 to verify the storage type of other RDS database instances provisioned in the current region.

08 Change the AWS region from the navigation bar and repeat the process for the other regions.

Using AWS CLI

01 Run describe-db-instances command (OSX/Linux/UNIX) using custom query filters to list the names (identifiers) of all RDS database instances available in the selected AWS region:

aws rds describe-db-instances
	--region us-east-1
	--output table
	--query 'DBInstances[*].DBInstanceIdentifier'

02 The command output should return a table with the requested identifiers:

-------------------------
|  DescribeDBInstances  |
+-----------------------+
|  cc-mysql-db          |
|  cc-mysql-aurora      |
|  cc-postgresql        |
+-----------------------+

03 Run again describe-db-instances command (OSX/Linux/UNIX) using the RDS database instance identifier and appropriate filtering to determine the storage type used by the selected resource:

aws rds describe-db-instances
	--region us-east-1
	--db-instance-identifier cc-mysql-db
	--query 'DBInstances[*].StorageType'

04 The command output should return the storage type used by the RDS instance:

[
    "io1"
]

If the value returned by the command output is "io1", the storage type used is Provisioned IOPS SSD, therefore the selected RDS instance is not using the most cost-effective storage type available (i.e. General Purpose SSD).

05 Repeat steps no. 3 and 4 to verify the storage type of other RDS database instances provisioned in the current region.

06 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 5 to perform the audit process for other regions.

Remediation / Resolution

To convert your Provisioned IOPS SSD based RDS instances to General Purpose SSD based instances, you need to modify your instances storage type configuration by perform the following:

Using AWS Console

01 Login to the AWS Management Console.

02 Navigate to RDS dashboard at https://console.aws.amazon.com/rds/.

03 In the left navigation panel, under RDS Dashboard, click Instances.

04 Select the RDS instance that you want to modify (see Audit section part I to identify the right resource).

05 Click Instance Actions button from the dashboard top menu and select Take Snapshot to create a database snapshot (backup).

06 On the Take DB Snapshot page, in the Snapshot Name box, enter a unique name for the database backup then click Take Snapshot to send the request.

07 Go back to the Instances page and select again the RDS instance that you want to modify.

08 Click Instance Actions button from the dashboard top menu and select Modify.

09 On the Modify DB Instance: <instance identifier> page, perform the following actions:

Select General Purpose SSD from the Storage Type dropdown list.
In the Allocated Storage box, enter the instance storage size in GB required to match the IOPS number previously set for the RDS database instance.
At the bottom of the page select Apply Immediately checkbox to apply the storage type change immediately. (!) IMPORTANT: when you convert (modify) the instance storage type, an immediate outage will occur if a custom parameter group is used. If the selected database instance is used in production consider leaving Apply Immediately option disabled before applying the changes in order to avoid any downtime. If Apply Immediately is not selected, the instance storage type change will be processed during the next maintenance window.

10 Click the Continue button to continue the process.

11 Review the storage type changes and click Modify DB Instance. During the modification process the instance status should change from available to modifying and back to available (it may take few minutes depending on the instance configuration).

12 Repeat steps no. 4 - 11 to convert other Provisioned IOPS SSD instances provisioned in the current region to General Purpose SSD instances.

13 Change the AWS region from the navigation bar and repeat the process for other regions.

Using AWS CLI

01 First, run create-db-snapshot command (OSX/Linux/UNIX) to take the necessary instance snapshot (backup):

aws rds create-db-snapshot
	--region us-east-1
	--db-instance-identifier cc-mysql-db
	--db-snapshot-identifier cc-mysql-db-io1-snapshot

02 The command output should return the database instance snapshot metadata:

{
    "DBSnapshot": {
        "Engine": "mysql",
        "Status": "creating",
        "AvailabilityZone": "us-east-1a",
        "PercentProgress": 0,
        "MasterUsername": "ccmysqlusr",
        "Encrypted": false,
        "LicenseModel": "general-public-license",
        "StorageType": "io1",
        "VpcId": "vpc-2fb56548",
        "DBSnapshotIdentifier": "cc-mysql-db-io1-snapshot",
        "InstanceCreateTime": "2015-10-20T18:25:07.280Z",
        "OptionGroupName": "default:mysql-5-6",
        "AllocatedStorage": 150,
        "EngineVersion": "5.6.27",
        "SnapshotType": "manual",
        "Iops": 1500,
        "Port": 3306,
        "DBInstanceIdentifier": "cc-mysql-db"
    }
}

03 Run modify-db-instance command (OSX/Linux/UNIX) to modify the selected RDS instance configuration in order to convert its current storage type to General Purpose SSD. (!) IMPORTANT: the following example is using the Apply Immediately option to apply the change asynchronously and trigger an immediate outage if a custom RDS parameter group is used. In order to avoid any downtime in production, skip adding the --apply-immediately command parameter and the RDS service will apply the storage type change during the next maintenance window. For the --allocated-storage parameter enter the instance storage size in GB required to match the IOPS number previously set for the RDS database instance:

aws rds modify-db-instance
	--region us-east-1
	--db-instance-identifier cc-mysql-db
	--allocated-storage 500
	--storage-type gp2
	--apply-immediately

04 The command output should reveal the configuration metadata for the modified RDS database instance:

{
    "DBInstance": {
        "PubliclyAccessible": true,
        "MasterUsername": "ccmysqlusr",
        "MonitoringInterval": 0,
        "LicenseModel": "general-public-license",
        "VpcSecurityGroups": [
            {
                "Status": "active",
                "VpcSecurityGroupId": "sg-b3dfd6c9"
            }
        ],
        "InstanceCreateTime": "2015-10-20T18:25:07.280Z",
        "CopyTagsToSnapshot": false,
        "OptionGroupMemberships": [
            {
                "Status": "in-sync",
                "OptionGroupName": "default:mysql-5-6"
            }
        ],
        "PendingModifiedValues": {},
        "Engine": "mysql",
        "MultiAZ": false,
        "LatestRestorableTime": "2016-10-25T19:30:00Z",
        "DBSecurityGroups": [],

        ...


        "ReadReplicaDBInstanceIdentifiers": [],
        "AllocatedStorage": 500,
        "BackupRetentionPeriod": 7,
        "DBName": "cc_mysql_dbn",
        "PreferredMaintenanceWindow": "sun:03:34-sun:04:04",
        "Endpoint": {
            "HostedZoneId": "Z2R2ITUGPM61AM",
            "Port": 3306,
            "Address": "cc-mysql-db.doacrvjugw55.us-east-1 ... "
        },
        "DBInstanceStatus": "available",
        "EngineVersion": "5.6.27",
        "AvailabilityZone": "us-east-1a",
        "DomainMemberships": [],
        "StorageType": "gp2",
        "DbiResourceId": "db-QEXMBFQGBV5DNZN3IOO3B2Y3OA",
        "CACertificateIdentifier": "rds-ca-2015",
        "Iops": 1500,
        "StorageEncrypted": false,
        "DBInstanceClass": "db.t2.micro",
        "DBInstanceIdentifier": "cc-mysql-db"
    }
}

05 Once the process is complete (it may take few minutes depending on the instance configuration), run describe-db-instances command (OSX/Linux/UNIX) using the database instance identifier to check if the storage type has been successfully changed:

aws rds describe-db-instances
	--region us-east-1
	--db-instance-identifier cc-mysql-db
	--query 'DBInstances[*].StorageType'

06 The command output should return the storage type currently used by the RDS instance:

[
    "gp2"
]

07 Repeat steps no. 1 - 6 to convert other Provisioned IOPS SSD instances provisioned in the current region to General Purpose SSD instances.

08 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 7 to perform the entire process for other regions.

References

AWS Command Line Interface (CLI) Documentation
rds
describe-db-instances
create-db-snapshot
modify-db-instance

Publication date Oct 26, 2016

Unlock the Remediation Steps

Gain free unlimited access
to our full Knowledge Base

Over 750 rules & best practices
for and

You are auditing:

RDS General Purpose SSD

Risk level: Medium

Audit

Using AWS Console

Using AWS CLI

Remediation / Resolution

Using AWS Console

Using AWS CLI

References

Related RDS rules