Amazon Web Services (AWS) S3 is the brilliant storage service used by many of the largest businesses in the world, and one that’s been linked to many data breaches recently. We talk about what’s new, how to keep your S3 buckets completely secure and safe with a few checks and share insights from our team of AWS security experts looking to improve the way businesses govern their cloud footprints.
Amazon S3 is object storage built to store and retrieve any amount of data from anywhere. It is designed to deliver 99.999999999% durability and provides comprehensive security and compliance capabilities that meet even the most stringent regulatory requirements. It gives customers flexibility in the way they manage data for cost optimization, access control, and compliance. S3 provides query-in-place functionality, allowing you to run powerful analytics directly on your data at rest in S3.
S3 News 2018: Greater Customization and Control Over Your Data
Asia Pacific Savings
Great news for those using Asia Pacific (Tokyo) and Asia Pacific (Sydney) regions as AWS announced reduced pricing for all S3 and EC2 data transfers. Seeing up to 34 per cent and 28 per cent reductions respectively, this will certainly help to get users better value for their money.
Cheaper Storage for Another Backup
Earlier this year, AWS released a new storage type to its list giving users the same durability but with a 20 per cent lower price tag. The new One Zone-IA band is designed to store data in a single Availability Zone, suiting data that requires a secondary back-up or that can be easily recreated.
Making the most out of tags to keep your storage customized and unique to your needs, AWS have enabled Cross-Region Replication to use Object Tags. Instead of duplicating an entire bucket, which can be expensive and rather unuseful, you now have greater control over exactly what you backup and where.
AWS Hardware-Cloud Storage Solution
For those that are still keen for a hybrid cloud storage solutions, AWS have partnered with Dell to create pre-configured hardware that easily connects on-prem devices to AWS Storage services. The EMC PowerEdge server, which can be bought online at amazon.com, provides low-latency data access allowing you to do all the usual Disaster Recovery practices such as tiering data to the cloud, storage and run data workflows. A truly, full circle AWS service.
S3 Select Revamp
AWS have announced quite a few new S3 Select changes this year. Back in April, the service was extended to allow users to retrieve subsets of data from S3 objects using SQL expressions, massively improving performance. Then just in September, they announced that the service now supports Apache Parquet format, JSON Arrays, and BZIP2 compression for CSV and JSON objects. The new additions give greater options for retrieving and querying specific data, so users can rightfully drill down into their information much more effortlessly.
Given the many, many data breaches over the past two years involving publicly accessible S3 buckets, let’s look at the good habits for completely safe and secure storage.
By default, S3 storage is private and access has to be granted, however, it’s often with the use of Access Control Lists (ACLs) that mistakes can be made. Here’s a checklist for them:
- Are you happy for anyone to access the bucket or object? If not, do not give ‘Read’ access to the Everyone group.
- Do you want anyone to be able to add or delete expensive or sensitive objects? Probably not! Ensure ‘Write’ access to the Everyone group is never granted.
- Equally, never allow ‘Write’ access to the Any Authenticated AWS User group. This group includes quite literally anyone with an active AWS account to change the bucket’s contents; use IAM users instead.
- If very specific access controls are needed, create custom permissions in your own ACL.
Enable access logging so you can keep track of all changes in your S3 buckets
For highly sensitive data needing encryption, S3 buckets support encryption at rest.
Cloud Conformity’s Golden Top Tips
With hindsight, we would all have encryption turned on for our S3 buckets on day one! When encryption is turned on for an existing bucket, the pre-existing objects in that bucket will not be encrypted, only new objects added to the bucket will be.
1) - So, one way to encrypt existing objects is to manually encrypt them again using AWS CLI, AWS SDK’s, AWS web console or REST API’s, then copying the object back to the same bucket.
For example, if you are planning to encrypt the S3 data at rest using S3 server-side encryption (SSE-S3) use the AWS CLI to encrypt objects already in an S3 bucket by copying and replacing the object back to the same bucket. An example of this command would be as follows:
aws s3 cp s3://bucket-name/prefix-a/ s3://bucket-name/prefix-a/ — recursive — sse AES256
2) - We recommend testing this on another bucket to ensure you have the syntax correct before you run the command against your production buckets.
3) - Be careful of data charges for copying objects in and out of buckets and versioning implications; if versioning is turned on you can inadvertently double the size of a bucket.