[AWS Design Cost-Optimized Architectures] 4.1 Identify cost-effective storage solutions

S3 Features

Prefiees and delimiters

For example: the file name in S3 can be: `marking/plans/kpi_2021_1.pdf`.

The point is make it looks like a folder structure, but it is not.

 

Storage classes

 

Object lifecycle management

For example, after store into 'S3 standard' without been used, can move the objects to 'Infrequent access stoage', in order to reduce the cost.

 

Encryption

• Server side encryption: Store the file as (256-bit AES) encrypted version, when accessing it, will decrypte it automatically.
• Client side encryption: You encrypt the file first, then update it. You have to maintain your key and decrypt that file locally.

Versioning

• But default it is not enabled

Multi-Factor Authentication (MFA) Delete / upload

• You cannot delete a file without auth yourself

• upload: split the objects into chunk, so it upload faster

Range GETs

• You just need a piece of information of a large file

Cross Region replication

• Copy the newly uploaded (not existing) object into multi region

Logging

Event notfications

---

 

Glacier

• Archival data storage
• Fractions of a penny per GB/month
• Three access methods
• Expedited (3-5mins)
• Standard (3-5hours)
• Bulk (5-12 hours)
• Define the region to store the data
• Data stored with AES-256 encryption by default

Glacier Integeration

• S3 cold data can be automatically moved into Glacier (S3 lifecycle management)
• Snow devices can be used to import data
• Storage Gateway can connect to Glacier

 

Glacier Concepts

• Archives: You save 'archives' into Glacier. in S3, it is called 'Objects', but once move into Glacier, it is called 'Archives'.
• Vaults: Vaults is the containers where we put 'archives'. In S3, we call 'Bulket', in Glacier, we call 'Vaults'
• Vault locks
• Data retrieval
  • Up to 5% retrieved at no charge, no rollover
  • Vault can be configured (who can retriveal, limit of retriveal) to limit cost

Create a Vault

• A single AWS account can create up to 1000 valuts pre Region
• Only empty vaylts can be deleted
• Glacier supports multipart uplodas of archives, so a large archive is not required to be updated in a single action.

 

---

 

Storage Gateway: Integrating on-premies storage

• Software appliance creates the gateway (it is just a VM you download)
• Provides three types of storage solutions:
  • File-based NFS
  • Volume-based: Internet SCSI protocol
  • Tape-based
• The file gateway provides an interface to S3 buckets

 

EC2 Pricing

On Demand

• On-demand
  • Charged for usage time at a flat rate

Three things can impact the cost:

1. Have the instance running

2. Have the appropriate storage

3. Have the appropriate netowrk throughput

• • Billed in 60-second increments rounded usage

Reserved

• Reserved for a period of time

Spot

• Bid on unsued compute time
• Overnight batch jobs is a good usecase
• Up to a 90% discount over on-demand

• On-demand pricing incurs charges based on usage and is billed in 60 second increments.
• Reserved pricing is based on at least a 1 year reservation and can be less expensive than on-demand when estimates are correct
• Spot pricing is the least expensive beause you are using unused ocmpute time.

 

EBS

• Presistent block storage
• Choose between SSD / PIOPS based on your case
• EBS need to pay for that as always, None of Tx instance type (free) are available for this.
• You need to make sure IOPS match the instance type you select

Docs

For example, if you need IOPS > 32,000, you have to choose `c4.8xlarge` instance type.

 

Tenacy: Shared 

• Multiple customers share the time and space on the physical mahcine.
• Default instance behavior

PROS & CONS

• Pros
  • Reduced costs
  • Simpler deployment
• Cons
  • Lower performance
  • Less control

• The shared tenancy model indicates that multiple instances from multiple customers will be on the same hardware
• Shared tenancy is the default behavior of an instance
• Using shared tenancy can reduce costs, but it may not comply with internal security policies

 

Tenacy: Decicated hosts

• physical machine for you
  • Running VM
• Used by one customer
• Must be explicitly configured
• Not Free

PROS & CONS

• Pros
  • More accurate licensing management
  • More detailed reporting
  • Compliance management
  • Determine host placement during instance restarts
• CONS
  • Cost more
  • Bring your own licenings (you need to prepare your own licensing)

• Performance is NOT the main reason you want to use decicated hosts, because you can choose the wirte instance class type to give you good performance.
• May allow for the installation of some applications that have licenses boudn to the hardware

 

Tenacy: Decicated instances

• Runs on a physical machine
• Only instance running on that machine
• On restart, may be moved to another physical machine (main difference from decicated hosts)
• Used by one customer
• Must be explicitly configured
• Not available in free tier

PROS & CONS

• PROS
  • Runs on hardware dedicated to the customer
  • Provides performance advantage of dedicated host (no other service from my account eat my CPU)
• CONS
  • Less accurate licensing management