[AWS - Design Resilient Architectures] 1.4 Choose appropriate resilient storage

Storage Services:

• S3
• Glacier
• CloudFront
• Elastic Block Store (EBS)
• Storage Gateway
• Snow family
• Database

 

Overview

Block storage:

• Used on local networks
  • iSCSI
  • Fibre Channel
• AWS can use block storage with virtual machines within the AWS cloud using EBS

File storage:

• object storage in S3

Selecting Storage:

• Size
• Performance
• Cost

---

 

S3

• It is Object storage. object can be file and any chunk of data.
• Distributes across at least three Availability zones
  • 　　Except: 1A (1 zone, least expensive)
• Support encryption and automatic data classification
• Big data analytics

Getting data into S3

• API
• Amazon Direct Connect (establish private connectivity between AWS and your datacenter)
• Storage Gateway (hybrid cloud storage service that gives you on-premises access to virtually unlimited cloud storage)
• Kinesis Firehose
• Transfer Acceleration (with cloudFront, fast but more expensive)
• Snow Family

 

S3 Concepts

• Bucekts, Regions, Objects, Keys, URLs
• Eventual consistency
• Works great for static website hosting

[Pro tip]: Objects in S3 buckets have eventual consistency. (might delay few seconds/minutes, but eventually)

Objects in Elastic Block Stores are consitent.

 

---

 

Creating a bucket

• Fully qualified domain name (FQDN): means that Bucket names must be globally unique.

 

Properties.Versioning

• Once you enabled it, you cannot disable it, only suspense it.

 

Properties.Static website hosting

• You need to have a DNS to redirect to that url which makes access more friendly

 

Properties.Encyption

• SSE: AWS managing key for you, it is easy 
• KMS: You have to managing the keys, more work to do

Management.lifecycle rules

• If you want some objects in bucket have different lifecycle rules, you can use TAG / Prefix to manage them

[Pro tip]

• minium size of object in S3 is zero bytes. 
• Prefixes act as folder
• An object can have up to 10 tags

 

---

 

 

Elastic Block Store (EBS)

Just like your laptop, it can store all kinds of files and application. On AWS, EC2, just like a laptop instance, to save the EC2 instance, we need Elastic Block Store.

Regularly back up your EBS volumes using Amazon EBS snapshots, and create an Amazon Machine Image (AMI) from your instance to save the configuration as a template for launching future instances. 

Overview

• Used for durable storage in EC2 instances. (EC2 can be shutted down and bring up, need to save all the information)
• Block-level storage from one AWS service to another

EBS Volume Type

Used for EC2 instance.

• Magnetic $ (default one)
• SSD (Solid-state drive) $$ - faster
  • General purpose
  • Provisioned IOPS: PIOPS: provisioned input/output operations per second

Pro tips: When it says you need IOPS for 12,000 or some number, you need PIOPS, 'provisioned' means 'guaranteed' that you get that level of IOPS. 

If just use 'General purpose' IOPS, you might just get 1000 or 3000... but not up to 12,000.

• EBS-optimized instance should be used

Pro tip: If you use SSD storage for your EBS and you wnat to take advantage of the performance capabilites, you have to use EBS-optimized instance. 

If don't, you will paying for SSD, but not get any performance.

 

protecting EBS Data

• Snapshots: backup your storage
• Volumn recovery: Attaching volumes from one instance to another. Just like you attaching one hard drive for your laptop.
• Encryption methods

---

 

Elastic File System (EFS)

•  Shareable: Different from EBS, only accosite with one instance, EFS can be shared to multi instances
• Hierarchical stucture: Different form S3 has prefix as "folder" structure, EFS has actual folder.
• Canbe accessed through NFSv4
• EC2 instances can use EFS shares
• EFS us bit supported on Windows instances

---

 

 

Comparison

 

Storage access security

 You can add S3 policy in generator:

So what 'Principal' should be: basiclly it is the User's ARN: you can find it in IAM

Amazon Resource Name: is the S3 bucket ARN:

 

---

 

Storage Performance

Amazon EBS volume types

General Purpose SSD's Max throughput per volume is 10,000MiB/s, if you need anything large than that, you need to use Provisioned IOPS SSD.

Hard disk drives (HDD)'s Max throughput per volume is 500 MiB/s. 

Pro Tips: Anytime you need 500 < 10,000, you have to use SSD, if need > 10,000 then PIOPS, < 500 then HDD.

 

A Gibibyte is not the same as gigabyte:

gibibyte is more accurate metric for cost managment.

 

Amazon S3 Storage Classes