[Cloud Architect] 2. Resiliency
Tradeoffs to consider
You can build a system that is singular with no failover options, and you can build systems that are Multi-AZ, Multi-Region with automated lightning fast failover. It is up to you to first determine what level of redundancy is appropriate and then figure out how to build that system.
When approaching a new product or service, you should consider how important the system will be. Will the existence of the company depend on it staying up, or is it just helping a team vote on lunch choices? Will it bring in 10 million dollars a month, or is it providing a service for free? Ultimately, there are tradeoffs that need to be considered, and understanding them is key to understanding how resilient to make a system.
Not all AWS services are redundant by default. Some services have varying degrees of redundancy and it usually costs more to make a service redundant.
Server-Based Services
Server bases services are those that are "instance" based. Services like RDS and ElastiCache are instanced based in that you can run one instance, but you will not have any fault tolerance. In order to gain high availability, you need to tell the service to provision a second instance for the primary instance to failover to, should there be an issue with it. This model is similar to traditional data center environments.
A good way to tell if a service is a server/instance based service is if the service is a pre-existing product that AWS has create a service with (MongoDB, Redis, MySQL, Postgres).
To get Multi-AZ availability, you need to configure a Subnet Group within the service. A subnet is attached to an AZ, and creating a grouping of subnets within the service tells the service where it can place the primary and standby instances of a service.
Redundancy In AWS
Subnet Groups are key to creating Multi-AZ redundancy in server-based services. Subnet Groups define the different availability zones that your service will run in, and having multiple instances allow for fast failover if a single AZ were to go down.
Multi-Region redundancy is more tricky. Depending on the service, it is harder, or not possible to run a service with failover between regions.
- Some services such as ElasticSearch, when you need multi-region redundancy, there is no option from AWS that provided to you. You have to do backup in another region, then enable you quick restore in another region.
- Some service such as RDS, you have option which AWS provide to you, to enable multi-AZ deployment; For multi-region, you can create a read replica, but it require on code level to do switching the endpoint for primary database.
DynamoDB Streams And Global Tables
DynamoDB Streams capture all changes made to a DynamoDB Table. This includes only actions that modify the table, not actions that only read from the table.
DynamoDB Global Tables take advantage of DynamoDB Streams the create Multi-Region active/active DynamoDB Tables. This allows you to modify a table in multiple regions and have those changes reflected in all regions.
Multi-Region, active/active data stores are a big deal and extremely useful for use cases that require it.
S3
S3 is one of the very first AWS services and is an object store. You create buckets and you can store an unlimited number of objects in a bucket.
Over the years, AWS has added many features to S3. There are now many different levels of storage options within S3. These options vary between the cost of use and time or frequency to retrieve objects stored there. AWS has features to execute Lambda functions on the creation, updating or deleting of objects. You can also automatically move or delete objects based on the time that they have been in a bucket with a lifecycle policy.
- Unlimited total size
- Inexpensive
- Durable
Compute Services
AWS offers several options for compute resources. EC2, Lambda and various container services are a few of them. These resources are similar to server based services in that they are not Multi-AZ by default, but with some work, they can be made to be.
EC2 instances can be run as a group with an AutoScaling Group and that AutoScaling Group can be capable of running in multiple AZs. If the EC2 instances are serving incoming HTTP requests, they can be put behind an Elastic Load Balancer, which can also be configured to span AZs.
AWS Lambda has different connectivity modes. If you have a Lambda that does not need to access your VPCs, then it can run independently. If your Lambda does need to access a VPC resource, then it needs to be configured to do so, and part of this configuration can be to allow the Lambda to connect to different AZs (subnets).
Edge Cases
You need to consider trade off for the servcies you use.
For example, if you need a task done in multi region, it might be much cheaper use Lambda, deploy the same function in each region, then run the function in each region; compare with use multi-region functionalities of other services.
When considering which services to utilize in a project, be sure to consider the usage patterns. Patterns that are "spikey" or seldomly used will be better suited for serverless services where you don't have to pay for idle resources. If you need a service to run in multiple regions, be sure to consider which services are capable of this.
Lesson Recap
- Server-based Services
- DynamoDB
- S3
- Compute Services
In this lesson, we looked at a variety of AWS data storage and compute options. AWS provides many options. Thinking through your use case, usage patterns, cost constraints and availability requirements are the key to successfully choosing the right services.
Lesson Objectives
You will be able to:
- Take advantage of different high availability options on AWS
- Create multi-AZ services
- Create multi-region services
- Identify what availability options exist for which AWS services
- Take advantage of resilient features in AWS services
Glossary
- Active/Active: A system that is running actively in multiple instances, typically in a distributed manner where complete functionality is available in more than one area.
- Snapshot: A complete copy of a dataset at a specific point in time.
- Server-based Services: Services that are existing applications that AWS provides as "managed services" and run on individual server instances.
- DynamoDB: AWS developed non-relational database
- DynamoDB Global Tables: Multi-Region DynamoDB Tables.
- S3: AWS developed object store that can store an unlimited amount of data.
- Compute Services: AWS services that provide generic compute capacity.
