Open menu

AWS ElastiCache Cluster In VPC

Cloud Conformity allows you to automate the auditing process of this resolution page. Register for a 14 day evaluation and check your compliance level for free!

Start a Free Trial Product features
Home Knowledge Base ElastiCache AWS ElastiCache Cluster In VPC
Last updated: 06 February 2017
Security

Risk level: Medium (should be achieved)

Ensure that your ElastiCache clusters are provisioned within the AWS EC2-VPC platform instead of EC2-Classic platform (outdated from 2013-12-04) for better flexibility and control over the cache clusters security, availability, traffic routing and more

This rule resolution is part of the Cloud Conformity Tool

Creating and managing Amazon ElastiCache clusters using EC2-VPC platform instead of EC2-Classic can bring multiple advantages such as better networking infrastructure (network isolation, subnets and private IP addresses), much more flexible control over access security (control over VPC security group membership, network ACLs, security group outbound/egress traffic filtering) and the capability to run cache clusters on single-tenant hardware.

Audit

To determine the EC2 platform (EC2-Classic or EC2-VPC) used to launch your ElastiCache clusters, perform the following:

Using AWS Console

01 Login to the AWS Management Console.

02 Navigate to ElastiCache dashboard at https://console.aws.amazon.com/elasticache/.

03 Within the Resources section, check the ElastiCache Supported Platforms attribute value(s):

  1. If the Supported Platforms value is VPC: Supported Platforms value is VPC, your AWS account supports only the EC2-VPC platform and all your ElastiCache clusters are launched within a Virtual Private Cloud (VPC), therefore the platform check for your account stops here.
  2. If the Supported Platforms attribute value is set to EC2 and VPC, your AWS account supports both EC2-Classic and EC2-VPC platforms. To identify any cache clusters launched using EC2-Classic, continue the audit process with the next step.

04 In the navigation panel, under ElastiCache Dashboard, click Memcached to access the clusters created with Memcached in-memory cache engine or Redis to access the clusters created with Redis engine.

05 Choose the Memcached/Redis cluster that you want to examine then click on its identifier link, listed in the Name column.

06 On the selected cluster configuration page, select the Description tab and check for the Subnet Group attribute value listed in the Cluster Details section. If the Subnet Group does not have a value (name), the selected Amazon ElastiCache cluster is not running within a Virtual Private Cloud (VPC), therefore a migration to EC2-VPC platform is highly recommended (see Remediation/Resolution section for the migration process).

07 Repeat step no. 5 and 6 to verify the platform used by other ElastiCache clusters available in the current region.

08 Change the AWS region from the navigation bar and repeat the process for other regions.

Using AWS CLI

01 Run describe-cache-clusters command (OSX/Linux/UNIX) using custom query filters to list the identifiers (ID) of all ElastiCache clusters available in the selected region: 

aws elasticache describe-cache-clusters
    --region us-east-1
    --output table
    --query 'CacheClusters[*].CacheClusterId'

02 The command output should return a table with the requested cluster identifiers: 

-------------------------
| DescribeCacheClusters |
+-----------------------+
| webcachecluster1      |
| memcachedcluster      |
+-----------------------+

03 Run again describe-cache-clusters command (OSX/Linux/UNIX) using the ID (identifier) of the cluster that you want to examine and the custom query filters to expose the name of the Subnet Group used by the selected AWS ElastiCache cluster: 

aws elasticache describe-cache-clusters
    --region us-east-1
    --cache-cluster-id webcachecluster1
    --query 'CacheClusters[*].CacheSubnetGroupName'

04 The command output should return the name of the Subnet Group attached to the cache cluster (if existent): 

[
    ""
]

If the command response is an empty string (i.e. ""), as shown in the example above, the selected Amazon ElastiCache cluster does not have a Subnet Group attached, therefore is not running within an AWS Virtual Private Cloud (VPC).

05 Repeat step no. 3 and 4 to verify the platform used (EC2-Classic or EC2-VPC) by other ElastiCache clusters deployed in the current region.

06 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 5 to perform the audit process for other regions.

Remediation / Resolution

To migrate your EC2-Classic ElastiCache clusters to a Virtual Private Cloud, you must re-create those clusters within a VPC environment. To relaunch the necessary clusters, perform the following:

Note: As example, this guide will explain how to migrate an ElastiCache Redis cache cluster from EC2-Classic platform to EC2-VPC within the same AWS region.

Using AWS Console

01 Sign in to the AWS Management Console.

02 Create the Virtual Private Cloud (VPC) where the EC2-Classic cache clusters will be migrated. To set up the VPC environment, perform the following actions:

  1. Navigate to VPC dashboard at https://console.aws.amazon.com/vpc/. Make sure you create the VPC environment in the same AWS region with the EC2-Classic clusters.
  2. In the Resources section, click Start VPC Wizard to initiate the setup process.
  3. Choose the VPC with a Single Public Subnet option then click the Select button.
  4. On the VPC with a Single Public Subnet configuration page, enter a name for your new Virtual Private Cloud in the VPC name box and leave the default configuration settings unchanged.
  5. Click Create VPC to set up your new VPC. Once the VPC is successfully created click OK to close the status window and return to the VPC dashboard.

03 Before relaunch the ElastiCache cluster, you need to create a VPC security group for the new cluster. To create the necessary security group, perform the following:

  1. Navigate to EC2 dashboard at https://console.aws.amazon.com/ec2/.
  2. In the navigation panel, under NETWORK & SECURITY section, select Security Groups.
  3. Click the Create Security Group button from the dashboard top menu to start the setup process.
  4. In the Create Security Group dialog box, provide the following details:
    • In the Security group name box, enter a name for the new security group.
    • In the Description box, provide a description for new security group.
    • From the VPC dropdown list, select the VPC ID/name created at step no. 2.
    • Inside the Inbound tab, click Add Rule to add the inbound rule(s), required by your ElastiCache cluster configuration.
    • Click the Create button to create the security group.

04 Navigate to ElastiCache dashboard at https://console.aws.amazon.com/elasticache/.

05 In the left navigation panel, under ElastiCache Dashboard, click Redis to access the clusters created with the Redis engine.

06 Choose the cache cluster that you want to examine then click on its identifier link (see Audit section part I to identify the right ElastiCache resource).

07 On the selected cluster configuration page, select the Description tab and copy the resource attributes such as Name, Node Type, Number of Nodes, Engine and Engine Version, and Cache Port. This information is required for the next step (i.e. cache cluster relaunch).

08 Now it’s time to re-create your EC2-Classic cache cluster within the AWS VPC deployed at step no. 2. To relaunch the necessary ElastiCache cluster, perform the following:

  1. Go back to the ElastiCache service dashboard and click Create to launch the cache cluster setup.
  2. On the Create your Amazon ElastiCache cluster page, perform the following actions:
    • Select Redis from the Cluster Engine section then paste configuration attributes of the existing cluster copied at step no. 7 inside the Redis settings section fields.
    • Click Advanced Redis settings tab to expand the cluster advanced settings panel then select the VPC security group created at step no. 3. Set the rest of the configuration options based on your needs.
    • Click the Create button to launch your new Amazon ElastiCache cluster. Once the cache cluster has been successfully created, its status should change from creating to available.

09 Once you have replaced the EC2-Classic cluster endpoint with the EC2-VPC cluster endpoint (e.g. webcachecluster1.ruxyga.ng.0002.use1.cache.amazonaws.com), it is safe to shut down and delete the old cache cluster in order to stop incurring charges for it. To remove the necessary ElastiCache cluster from your AWS account, perform the following:

  1. Select the cache cluster that you want to remove and click the Delete button from the dashboard top menu.
  2. In the Delete Cluster confirmation box, select Yes from the Create final backup dropdown menu, provide a name for the cluster backup, then click Delete.

Using AWS CLI

01 First, run create-vpc command (OSX/Linux/UNIX) to create the new Virtual Private Cloud (VPC) where the ElastiCache cluster will be re-created. The following command example creates a shared tenancy VPC with the CIDR block 10.0.0.0/16: 

aws ec2 create-vpc
    --region us-east-1
    --cidr-block 10.0.0.0/16

02 The command output should return the new VPC metadata (including the VPC ID): 

{
    "Vpc": {
        "VpcId": "vpc-ca27e381",
        "InstanceTenancy": "default",
        "State": "pending",
        "DhcpOptionsId": "dopt-e5d918f5",
        "CidrBlock": "10.0.0.0/16",
        "IsDefault": false
    }
}

03 Run create-internet-gateway command (OSX/Linux/UNIX) to create an AWS Internet Gateway for use with the newly created VPC (required): 

aws ec2 create-internet-gateway
    --region us-east-1

04 The command output should return the Internet Gateway metadata (including its ID): 

{
    "InternetGateway": {
        "Tags": [],
        "InternetGatewayId": "igw-506a0b24",
        "Attachments": []
    }
}

05 Run attach-internet-gateway command (OSX/Linux/UNIX) to attach the new Internet Gateway to your VPC created at step no. 3 (the command does not produce an output): 

aws ec2 attach-internet-gateway
    --region us-east-1
    --internet-gateway-id igw-506a0b24
    --vpc-id vpc-ca27e381

06 Now run create-subnet command (OSX/Linux/UNIX) to set up a subnet for the existing VPC. The cache cluster will be launched within this subnet (required): 

aws ec2 create-subnet
    --region us-east-1
    --vpc-id vpc-ca27e381
    --cidr-block 10.0.1.0/24

07 The command output should return the subnet metadata (including the subnet ID): 

{
    "Subnet": {
        "VpcId": "vpc-ca27e381",
        "CidrBlock": "10.0.1.0/24",
        "State": "pending",
        "AvailabilityZone": "us-east-1a",
        "SubnetId": "subnet-da296f89",
        "AvailableIpAddressCount": 251
    }
}

08 Run create-route-table command (OSX/Linux/UNIX) to create a route table for your new VPC (required): 

aws ec2 create-route-table
    --region us-east-1
    --vpc-id vpc-ca27e381

09 The command output should return the VPC route table metadata (including its ID - highlighted): 

{
    "RouteTable": {
        "Associations": [],
        "RouteTableId": "rtb-50611442",
        "VpcId": "vpc-ca27e381",
        "PropagatingVgws": [],
        "Tags": [],
        "Routes": [
            {
                "GatewayId": "local",
                "DestinationCidrBlock": "10.0.0.0/16",
                "State": "active",
                "Origin": "CreateRouteTable"
            }
        ]
    }
}

10 Run associate-route-table command (OSX/Linux/UNIX) to associate the VPC subnet created at step no. 6 with the new route table (required): 

aws ec2 associate-route-table
  --region us-east-1
  --route-table-id rtb-50611442
  --subnet-id subnet-da296f89

11 The command output should return the VPC route table association ID: 

{
    "AssociationId": "rtbassoc-a3f461e1"
}

12 Run create-route command (OSX/Linux/UNIX) to add a new route within the VPC route table installed earlier (required): 

aws ec2 create-route
  --region us-east-1
  --route-table-id rtb-50611442
  --destination-cidr-block 0.0.0.0/0
  --gateway-id igw-506a0b24

13 The command output should return the status of request (true for success, an error message if the request fails): 

{
    "Return": true
}

14 Now that your VPC is ready, you need to create the necessary security group for the new cluster. To set up the VPC security group, perform the following:

  1. Run create-security-group command (OSX/Linux/UNIX) to create a security group within the VPC created at step no. 1. The following command example creates a security group called ElastiCacheSecurityGroup inside the VPC identified with the ID vpc-ca27e381, within the US East AWS region: 
    aws ec2 create-security-group
  --region us-east-1
  --group-name ElastiCacheSecurityGroup
  --description "Redis Cache Cluster Security Group"
  --vpc-id vpc-ca27e381
  2. The command output should return the new security group ID: 
    {
    "GroupId": "sg-f29492e0"
}
  3. Run authorize-security-group-ingress command (OSX/Linux/UNIX) to add one or more inbound rules to the security group created at the previous step (no command output is returned): 
    aws ec2 authorize-security-group-ingress
  --region us-east-1
  --group-id sg-f29492e0
  --protocol tcp
  --port 6379
  --cidr 53.165.46.104/32

15 Now gather the configuration details from the existing EC2-Classic ElastiCache cluster, details required for the next step (i.e. cache cluster relaunch). Run describe-cache-clusters command (OSX/Linux/UNIX) using the ID of the cluster that you want to re-create (see Audit section part II to identify the right resource) to describe the selected cluster configuration details: 

aws elasticache describe-cache-clusters
  --region us-east-1
  --cache-cluster-id webcachecluster1

16 The command output should return the EC2-Classic cache cluster configuration metadata: 

{
    "CacheClusters": [
        {
            "Engine": "redis",
            "CacheClusterId": "webcachecluster1",
            "NumCacheNodes": 2,
            "CacheClusterCreateTime": "2013-01-30T09:25:26.712Z",
            "AutoMinorVersionUpgrade": true,
            "CacheClusterStatus": "available",
            "PreferredAvailabilityZone": "us-east-1a",

            ...

            "CachePort": 6379,
            "CacheSubnetGroupName": "",
            "EngineVersion": "2.6.13",
            "PendingModifiedValues": {},
            "PreferredMaintenanceWindow": "sat:03:00-sat:04:00",
            "CacheNodeType": "cache.m3.medium"
        }
    ]
}

17 Re-create your EC2-Classic cache cluster within the AWS VPC deployed at step no. 1 with create-cache-cluster command (OSX/Linux/UNIX), using the existing ElastiCache cluster configuration attributes returned at the previous step: 

aws elasticache create-cache-cluster
  --region us-east-1
  --cache-cluster-id vpccachecluster
  --az-mode single-az
  --preferred-availability-zone "us-east-1a"
  --num-cache-nodes 2
  --cache-node-type cache.m3.medium
  --engine redis
  --engine-version "2.6.13"
  --security-group-ids "sg-f29492e0"
  --port 6379
  --auto-minor-version-upgrade

18 The command output should return the newly created EC2-VPC cache cluster metadata: 

{
    "CacheCluster": {
        "Engine": "redis",
        "CacheParameterGroup": {
            "CacheNodeIdsToReboot": [],
            "CacheParameterGroupName": "default.redis2.6",
            "ParameterApplyStatus": "in-sync"
        },
        "CacheClusterId": "vpccachecluster",
        "CacheSecurityGroups": [],
        "NumCacheNodes": 2,
        "AutoMinorVersionUpgrade": true,
        "CacheClusterStatus": "creating",
        "PreferredAvailabilityZone": "us-east-1a",

        ...

        "SecurityGroups": [
            {
                "Status": "active",
                "SecurityGroupId": "sg-f29492e0"
            }
        ],
        "CacheSubnetGroupName": "default",
        "EngineVersion": "2.6.13",
        "PendingModifiedValues": {},
        "PreferredMaintenanceWindow": "tue:03:30-tue:04:30",
        "CacheNodeType": "cache.m3.medium"
    }
}

19 Once the EC2-Classic cluster endpoint have been replaced with the EC2-VPC one (e.g. webcachecluster1.rybucx.ng.0002.use1.cache.amazonaws.com), it is safe to shut down and delete the old cache cluster to stop incurring charges for the resource. To remove the EC2-Classic ElastiCache cluster from your AWS account, run delete-cache-cluster command (OSX/Linux/UNIX): 

aws elasticache delete-cache-cluster
  --region us-east-1
  --cache-cluster-id webcachecluster1
  --final-snapshot-identifier webcachecluster1-final-snapshot

20 The command output should return the old cache cluster metadata (including the resource current status, i.e. "deleting"): 

{
    "CacheClusters": [
        {
            "Engine": "redis",
            "CacheClusterId": "webcachecluster1",
            "NumCacheNodes": 2,
            "CacheClusterCreateTime": "2013-01-30T09:25:26.712Z",
            "AutoMinorVersionUpgrade": true,
            "CacheClusterStatus": "deleting",

            ...

            "CachePort": 6379,
            "CacheSubnetGroupName": "",
            "EngineVersion": "2.6.13",
            "PendingModifiedValues": {},
            "PreferredMaintenanceWindow": "sat:03:00-sat:04:00",
            "CacheNodeType": "cache.m3.medium"
        }
    ]
}

References

Publication date Feb 6, 2017

Related ElastiCache rules