[Terraform] 08 - Deploying Django to AWS ECS
文章：Deploying Django to AWS ECS with Terraform
代码：https://github.com/testdrivenio/django-ecs-terraform
参考：How to deploy AWS ECS Fargate Containers Step by Step using Terraform
顺便说一句，这个也不错：Test-Driven Development with Django, Django REST Framework, and Docker【测试驱动，才专业】
Django 部分
一、创建项目: <startproject>
$ mkdir django-ecs-terraform && django-ecs-terraform $ mkdir app && cd app $ python3.8 -m venv env $ source env/bin/activate (env)$ pip install django==3.1 (env)$ django-admin.py startproject hello_django . (env)$ python manage.py migrate (env)$ python manage.py runserver
-
DB's migrate
Ref: 关于django 数据库迁移(migrate)应该知道的一些事
前者是将model层转为迁移文件migration，后者将新版本的迁移文件执行，更新数据库。
这两中命令调用默认为全局，即对所有最新更改的model或迁移文件进行操作。如果想对部分app进行操作，就要在其后追加app name。
So, 生成了db.sqlite3。
-
settings.py
DEBUG = True ALLOWED_HOSTS = ['*']
二、配置容器
-
Dockerfile
# pull official base image FROM python:3.8.5-slim-buster # set work directory WORKDIR /usr/src/app # set environment variables ENV PYTHONDONTWRITEBYTECODE 1 ENV PYTHONUNBUFFERED 1 # install dependencies RUN pip install --upgrade pip COPY ./requirements.txt . RUN pip install -r requirements.txt # copy project COPY . .
docker内启动安装：requirements.txt
requirements.txtapp下内容集体拷贝入容器。
jeffrey@unsw-ThinkPad-T490:app$ docker run -it django-ecs bash root@0b4ea7d72577:/usr/src/app# root@0b4ea7d72577:/usr/src/app# root@0b4ea7d72577:/usr/src/app# root@0b4ea7d72577:/usr/src/app# ls Dockerfile db.sqlite3 env hello_django manage.py requirements.txt
-
Run docker
$ docker build -t django-ecs . $ docker run \ -p 8007:8000 \ --name django-test \ django-ecs \ gunicorn hello_django.wsgi:application --bind 0.0.0.0:8000
Ref: 把 Django 当作普通 WSGI 应用在 Gunicorn 中运行
这样会创建一个进程，包含了一个监听在 127.0.0.1:8000 的线程。前提是你的项目在 Python path 中，要满足这个条件，最简单的方法是在 manage.py 文件所在的目录中运行这条命令。
-
Close container
$ docker stop django-test $ docker rm django-test
-
上传容器
ECR的基本操作。
Terraform 部分
一、AWS Resources
Next, let's configure the following AWS resources:
-
- Networking:
- VPC
- Public and private subnets
- Routing tables
- Internet Gateway
- Key Pairs
- Security Groups
- Load Balancers, Listeners, and Target Groups
- IAM Roles and Policies
- ECS:
- Task Definition (with multiple containers)
- Cluster
- Service
- Launch Config and Auto Scaling Group
- Health Checks and Logs
- Networking:
二、Networking
Ref: List of AWS regions and availability zones
路由表：一个public，一个private。
PUBLIC |
aws_route_table.public-route-table |
aws_subnet.public-subnet-1 |
aws_subnet.public-subnet-2 |
||
PRIVATE |
aws_route_table.private-route-table |
aws_subnet.private-subnet-1 |
aws_subnet.private-subnet-2 |
思考：与 [AWS] Terraform: 03 - VPC + NAT 的区别
vpc.tf
# Production VPC resource "aws_vpc" "production-vpc" { cidr_block = "10.0.0.0/16" enable_dns_support = true enable_dns_hostnames = true }
# Public subnets resource "aws_subnet" "public-subnet-1" { cidr_block = var.public_subnet_1_cidr vpc_id = aws_vpc.production-vpc.id availability_zone = var.availability_zones[0] } resource "aws_subnet" "public-subnet-2" { cidr_block = var.public_subnet_2_cidr vpc_id = aws_vpc.production-vpc.id availability_zone = var.availability_zones[1] } # Private subnets resource "aws_subnet" "private-subnet-1" { cidr_block = var.private_subnet_1_cidr vpc_id = aws_vpc.production-vpc.id availability_zone = var.availability_zones[0] } resource "aws_subnet" "private-subnet-2" { cidr_block = var.private_subnet_2_cidr vpc_id = aws_vpc.production-vpc.id availability_zone = var.availability_zones[1] }
#------------------------------------------------------
# Route tables for the subnets resource "aws_route_table" "public-route-table" { vpc_id = aws_vpc.production-vpc.id }# Associate the newly created route tables to the subnets resource "aws_route_table_association" "public-route-1-association" { route_table_id = aws_route_table.public-route-table.id subnet_id = aws_subnet.public-subnet-1.id } resource "aws_route_table_association" "public-route-2-association" { route_table_id = aws_route_table.public-route-table.id subnet_id = aws_subnet.public-subnet-2.id }
# Internet Gateway for the public subnet resource "aws_internet_gateway" "production-igw" { vpc_id = aws_vpc.production-vpc.id }
nat.tf
# Route tables for the subnets resource "aws_route_table" "public-route-table" { vpc_id = aws_vpc.production-vpc.id } resource "aws_route_table" "private-route-table" { vpc_id = aws_vpc.production-vpc.id } # Associate the newly created route tables to the subnets resource "aws_route_table_association" "private-route-1-association" { route_table_id = aws_route_table.private-route-table.id subnet_id = aws_subnet.private-subnet-1.id } resource "aws_route_table_association" "private-route-2-association" { route_table_id = aws_route_table.private-route-table.id subnet_id = aws_subnet.private-subnet-2.id } # Elastic IP resource "aws_eip" "elastic-ip-for-nat-gw" { vpc = true associate_with_private_ip = "10.0.0.5" depends_on = [aws_internet_gateway.production-igw] }
# NAT gateway resource "aws_nat_gateway" "nat-gw" { allocation_id = aws_eip.elastic-ip-for-nat-gw.id subnet_id = aws_subnet.public-subnet-1.id depends_on = [aws_eip.elastic-ip-for-nat-gw] }
aws_route
将路由表和网关绑定。
resource "aws_route" "nat-gw-route" { route_table_id = aws_route_table.private-route-table.id nat_gateway_id = aws_nat_gateway.nat-gw.id destination_cidr_block = "0.0.0.0/0" } # Route the public subnet traffic through the Internet Gateway resource "aws_route" "public-internet-igw-route" { route_table_id = aws_route_table.public-route-table.id gateway_id = aws_internet_gateway.production-igw.id destination_cidr_block = "0.0.0.0/0" }
三、Security Groups
一些不同之处，或是需要注意的地方：
允许ecs通过5432端口访问rds服务。
# RDS Security Group (traffic ECS -> RDS) resource "aws_security_group" "rds" { name = "rds-security-group" description = "Allows inbound access from ECS only" vpc_id = aws_vpc.production-vpc.id ingress { protocol = "tcp" from_port = "5432" to_port = "5432" security_groups = [aws_security_group.ecs.id] } egress { protocol = "-1" from_port = 0 to_port = 0 cidr_blocks = ["0.0.0.0/0"] } }
Load balancer监听443端口的https请求。
端口：443，服务：Https
四、Application Load Balancer (ALB)
-
Target group 配置
- Classic Load Balancer (ELB)
- Application Load Balancer (ALB、ELBv2) --> 有了target group的概念。
AWS 提供的的建議是：對於 TCP/SSL 或是 EC2 Classic 的應用使用 ELB，其他的情境就使用 ALB。
# Production Load Balancer resource "aws_lb" "production" { name = "${var.ecs_cluster_name}-alb" load_balancer_type = "application" internal = false security_groups = [aws_security_group.load-balancer.id] subnets = [aws_subnet.public-subnet-1.id, aws_subnet.public-subnet-2.id] } # Target group resource "aws_alb_target_group" "default-target-group" { name = "${var.ecs_cluster_name}-tg" port = 80 protocol = "HTTP" vpc_id = aws_vpc.production-vpc.id health_check { path = var.health_check_path # --> ./ping port = "traffic-port" healthy_threshold = 5 unhealthy_threshold = 2 timeout = 2 interval = 5 matcher = "200" } }
So, we configured our load balancer and listener to listen for HTTP requests on port 80. This is temporary. After we verify that our infrastructure and application are set up correctly, we'll update the load balancer to listen for HTTPS requests on port 443.
Take note of the path URL for the health check: /ping/.
# Listener (redirects traffic from the load balancer to the target group) resource "aws_alb_listener" "ecs-alb-http-listener" { load_balancer_arn = aws_lb.production.id port = "443" protocol = "HTTPS" ssl_policy = "ELBSecurityPolicy-2016-08" certificate_arn = var.certificate_arn depends_on = [aws_alb_target_group.default-target-group] default_action { type = "forward" target_group_arn = aws_alb_target_group.default-target-group.arn } }
certificate_arn-(可选)AWS管理的证书的ARN。 AWS Certificate Manager是唯一受支持的来源。 在需要边缘优化的域名时使用。
-
如何获得 certificate arn?
# ACM (SSL certificate) - Specify ARN of an existing certificate or new one will be created and validated using Route53 DNS certificate_arn = "arn:aws:acm:eu-west-1:135367859851:certificate/70e008e1-c0e1-4c7e-9670-7bb5bd4f5a84"
Goto: [AWS] ACM: AWS Certificate Manager
五、IAM Roles
参考: [AWS] Terraform: 05 - IAM: group, user, role, and ec2 init
定义角色，以及其策略。
resource "aws_iam_role" "ecs-host-role" { name = "ecs_host_role_prod" assume_role_policy = file("policies/ecs-role.json") }
# ----------------------------------------------------------
resource "aws_iam_role_policy" "ecs-instance-role-policy" { name = "ecs_instance_role_policy" policy = file("policies/ecs-instance-role-policy.json") role = aws_iam_role.ecs-host-role.id }
resource "aws_iam_instance_profile" "ecs" { name = "ecs_instance_profile_prod" path = "/" role = aws_iam_role.ecs-host-role.name }
如何理解ecs的这两个role？（需要进一步探究）
resource "aws_iam_role" "ecs-service-role" { name = "ecs_service_role_prod" assume_role_policy = file("policies/ecs-role.json") }
# ---------------------------------------------------------- resource "aws_iam_role_policy" "ecs-service-role-policy" { name = "ecs_service_role_policy" policy = file("policies/ecs-service-role-policy.json") role = aws_iam_role.ecs-service-role.id }
六、Cloudwatch + Logs
resource "aws_cloudwatch_log_group" "django-log-group" { name = "/ecs/django-app" retention_in_days = var.log_retention_in_days } resource "aws_cloudwatch_log_stream" "django-log-stream" { name = "django-app-log-stream" log_group_name = aws_cloudwatch_log_group.django-log-group.name }
# ---------------------------------------------------------------
resource "aws_cloudwatch_log_group" "nginx-log-group" { name = "/ecs/nginx" retention_in_days = var.log_retention_in_days } resource "aws_cloudwatch_log_stream" "nginx-log-stream" { name = "nginx-log-stream" log_group_name = aws_cloudwatch_log_group.nginx-log-group.name }
七、Key Pair
resource "aws_key_pair" "production" { key_name = "${var.ecs_cluster_name}_key_pair" public_key = file(var.ssh_pubkey_file) }
八、ECS
resource "aws_ecs_cluster" "production" { name = "${var.ecs_cluster_name}-cluster" } data "template_file" "app" { template = file("templates/django_app.json.tpl") vars = { docker_image_url_django = var.docker_image_url_django docker_image_url_nginx = var.docker_image_url_nginx region = var.region rds_db_name = var.rds_db_name rds_username = var.rds_username rds_password = var.rds_password rds_hostname = aws_db_instance.production.address allowed_hosts = var.allowed_hosts } } resource "aws_ecs_task_definition" "app" { # 开机运行任务 family = "django-app" container_definitions = data.template_file.app.rendered depends_on = [aws_db_instance.production] volume { name = "static_volume" host_path = "/usr/src/app/staticfiles/" } }
resource "aws_ecs_service" "production" { name = "${var.ecs_cluster_name}-service" cluster = aws_ecs_cluster.production.id task_definition = aws_ecs_task_definition.app.arn iam_role = aws_iam_role.ecs-service-role.arn desired_count = var.app_count depends_on = [aws_alb_listener.ecs-alb-http-listener, aws_iam_role_policy.ecs-service-role-policy] load_balancer { target_group_arn = aws_alb_target_group.default-target-group.arn container_name = "nginx" container_port = 80 } }
九、Auto Scaling
触发条件是什么呢？
参考：[AWS] Terraform: 06 - Autoscaling and ELB
resource "aws_autoscaling_group" "ecs-cluster" { name = "${var.ecs_cluster_name}_auto_scaling_group" min_size = var.autoscale_min max_size = var.autoscale_max desired_capacity = var.autoscale_desired health_check_type = "EC2" launch_configuration = aws_launch_configuration.ecs.name vpc_zone_identifier = [aws_subnet.public-subnet-1.id, aws_subnet.public-subnet-2.id] }
以下，定义在了 ecs.tf 中。
resource "aws_launch_configuration" "ecs" { name = "${var.ecs_cluster_name}-cluster" image_id = lookup(var.amis, var.region) instance_type = var.instance_type security_groups = [aws_security_group.ecs.id] iam_instance_profile = aws_iam_instance_profile.ecs.name key_name = aws_key_pair.production.key_name associate_public_ip_address = true user_data = "#!/bin/bash\necho ECS_CLUSTER='${var.ecs_cluster_name}-cluster' > /etc/ecs/ecs.config" }
/* continue */
