[Django] 01 - How to design tests for REST API

Let's go: https://testdriven.io/courses/tdd-django/

Ref: Django drf 教程【中文教程】

Ref: Django Rest-framework【十四罗汉】

Ref: https://www.django-rest-framework.org/tutorial/1-serialization/【官方文档】 

 

DRF

Django REST Framework

Pytest

We'll use Pytest instead of unittest for writing unit and integration tests to test the Django API. 

 

 

新建项目

一、基本安装

Upgraded to Django 3.0.2 and Python 3.8.1.

$ mkdir django-tdd-docker && cd django-tdd-docker
$ mkdir app && cd app

  $ sudo apt-get install python3.8-venv
  $ sudo python3.8 -m venv env

$ python3.8 -m venv env
$ source env/bin/activate

(env)$ pip install django==3.0.5 djangorestframework==3.11.0

(env)$ django-admin.py startproject drf_project .

(env)$ python manage.py startapp movies

 

二、打基础为好

(env) jeffrey@unsw-ThinkPad-T490:tutorial$ django-admin.py startproject tutorial .
(env) jeffrey@unsw-ThinkPad-T490:tutorial$ ls
manage.py  tutorial

(env) jeffrey@unsw-ThinkPad-T490:tutorial$ cd tutorial/
(env) jeffrey@unsw-ThinkPad-T490:tutorial$ ls
asgi.py  __init__.py  settings.py  urls.py  wsgi.py


(env) jeffrey@unsw-ThinkPad-T490:tutorial$ django-admin.py startapp quickstart
(env) jeffrey@unsw-ThinkPad-T490:tutorial$ ls
asgi.py  __init__.py  quickstart  settings.py  urls.py  wsgi.py

./quickstart文件夹 中的内容如下：

admin.py  apps.py  __init__.py  migrations  models.py  tests.py  views.py

 

三、目录结构

• WSGI & ASGI

Ref: WSGI&ASGI

WSGI：基于HTTP协议模式的，不支持WebSocket，

ASGI：其诞生则是为了解决Python常用的WSGI不支持当前Web开发中的一些新的协议标准。(就是个补丁)

同时，ASGI对于WSGI原有的模式的支持和WebSocket的扩展，即ASGI是WSGI的扩展。

 

• 目录结构

基本的三个部分：app部分、manage.py、project 主体部分。

其实添加了两个 APP：drf_project, movies。

├── .gitignore
├── app
│   ├── .dockerignore
│   ├── .env.dev
│   ├── Dockerfile
│   ├── drf_project
│   │   ├── __init__.py
│   │   ├── asgi.py
│   │   ├── settings.py
│   │   ├── urls.py
│   │   └── wsgi.py
│   ├── manage.py
│   ├── movies
│   │   ├── __init__.py
│   │   ├── admin.py
│   │   ├── apps.py
│   │   ├── migrations
│   │   │   ├── 0001_initial.py
│   │   │   └── __init__.py
│   │   ├── models.py
│   │   ├── tests.py
│   │   └── views.py
│   └── requirements.txt
└── docker-compose.yml

 

• settings.py 文件

Ref: Django框架文件解析--setting.py

  

四、启动服务

先把数据库准备好，然后运行服务即可。

python manage.py migrate
python manage.py runserver

 

五、创建 APP

python manage.py startapp blog

 

apps.py

from django.apps import AppConfig

class BlogConfig(AppConfig):
    name = 'blog'

 

admin.py

from django.contrib import admin

# Register your models here

 

models.py

from django.db import models

# Create your models here.

 

tests.py

from django.test import TestCase

# Create your tests here.

 

views.py

from django.shortcuts import render

# Create your views here.

 

 

 

数据库操作

一、为 app 配置 db

基于两个例子，一个是cms徒手创建一个博客。另一个是tdd的例子，这里作为相互映衬，帮助学习。

• 创建 model

这些都会反映在控制面板的UI上。

from django.db import models

from django.utils import timezone
from django.contrib.auth.models import User

# Create your models here.
class Post(models.Model):
  STATUS_CHOICES = (
    ('draft', 'Draft'),
    ('published', 'Published'),
  )
  title           = models.CharField(max_length=250)
  slug            = models.CharField(max_length=250)
  content         = models.TextField()
  seo_title       = models.CharField(max_length=250)
  seo_description = models.CharField(max_length=160)

  author = models.ForeignKey(User, related_name='blog_posts')

  published = models.DateTimeField(default=timezone.now)
  created   = models.DateTimeField(auto_now_add=True)
  updated   = models.DateTimeField(auto_now=True)
  status    = models.CharField(max_length=9, choices=STATUS_CHOICES, default=published)

  def __str__(self):
    return self.title

 

• 创建一个 app 以及对应的 db

 

[数据库迁移]

1. 首先在settings.py中添加上app。

2. 然后：python manage.py makemigrations blog，生成了./imgrations/

如何理解 0001_initial.py？如下命令。

(blog) jeffrey@unsw-ThinkPad-T490:cms$ python manage.py sqlmigrate blog 0001
BEGIN;
--
-- Create model Post
--
CREATE TABLE "blog_post" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "title" varchar(250) NOT NULL, "slug" varchar(250) NOT NULL, "content" text NOT NULL, "seo_title" varchar(250) NOT NULL, "seo_description" varchar(160) NOT NULL, "published" datetime NOT NULL, "created" datetime NOT NULL, "updated" datetime NOT NULL, "status" varchar(9) NOT NULL, "author_id" integer NOT NULL REFERENCES "auth_user" ("id"));
CREATE INDEX "blog_post_4f331e2f" ON "blog_post" ("author_id");
COMMIT;

3. 最后：python manage.py migrate

 

• 创建 超级用户 并 注册 db model

(blog) jeffrey@unsw-ThinkPad-T490:cms$ python manage.py createsuperuser
Username (leave blank to use 'jeffrey'): 
Email address: jeffrey@yahoo.com
Password: 
Password (again): 
Superuser created successfully.

并注册 in admin.py

from django.contrib import admin
from  .models import Post
# Register your models here.

admin.site.register(Post)

多出一个Post在以下界面中。

 

二、model 对应的 控制面板

体会这种，model <--> admin UI 对应的美妙之处。

默认数据库sqlite的内容： 

 

三、Django Shell

We will learn how to query the database. This step is very important and it may be a bit boring but if you understand how to query I database in Django you basically will never have any issues with Django.

这里，通过 命令行对数据库进行操作。

 

• 添加一个表

(blog) jeffrey@unsw-ThinkPad-T490:cms$ python manage.py shell
Python 3.7.0 (default, Jun 28 2018, 13:15:42) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)

>>> from django.contrib.auth.models import User
>>> from blog.models import Post

>>> user = User.objects.get(username='jeffrey')

# 添加一条
>>> post = Post.objects.create(title='Practice Title', slug='practice-title', content='This is the content', author=user)
>>> post.save()

# 再做一次修改
>>> post.title = 'Tom'
>>> post.save()
>>> Post.objects.all()
<QuerySet [<Post: code>, <Post: Tom>]>

 

• 删除这个表

看上去 pk和id作为key是一个效果。

>>> post = Post.objects.get(pk=2)
>>> post
<Post: Tom>

>>> post = Post.objects.get(pk=1)
>>> post
<Post: code>

>>> post = Post.objects.get(id=2)
>>> post
<Post: Tom>

>>> post.delete()
(1, {'blog.Post': 1})

>>> Post.objects.all()
<QuerySet [<Post: code>]>

 

 

 

数据库微服务 

一、预备知识

Ref: PostgreSQL: The World's Most Advanced Open Source Relational Database

Ref: PostgreSQL新手入门

MySQL 和 Postgresql 的最大区别应该就是多线程和多进程设计的对决吧。虽然双方的出发点不一样，也各有优势，但是个人觉得多进程相对于多线程来说，管理和通信方面要方便不少，理解上也更简洁明了。

[未来需要解决的一个问题] Ref: How to Deploy a django app into aws with RDS - Part 3

 

• psycopg2

psycopg2 库是 python 用来操作 postgreSQL 数据库的第三方库。

Download: https://www.psycopg.org/

movies的容器中要配置安装相关package。

所以，需在 docker-compose.yml 中配置好。

 

 

二、 docker image 配置

• Docker-compose

Ref: Docker Compose 配置文件详解

docker-compose的若干关键字：

build：它可以指定 Dockerfile 所在文件夹的路径。Compose 将会利用它自动构建这个镜像，然后使用这个镜像启动服务容器。

command：容器启动后默认执行的命令。

volumes：挂载一个目录或者一个已存在的数据卷容器。

version: '3.7'

services:
  movies:
    build: ./app
    command: python manage.py runserver 0.0.0.0:8000
    volumes:
      - ./app/:/usr/src/app/
    ports:
      - 8000:8000
    env_file:
      - ./app/.env.dev　　# 保存了各种环境变量
    depends_on:
      - movies-db

  movies-db:
    image: postgres:12-alpine
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    environment:
      - POSTGRES_USER=movies
      - POSTGRES_PASSWORD=movies
      - POSTGRES_DB=movies_dev

volumes:
  postgres_data:

 

有两个 container 在运行。

$ docker-compose up -d --build
$ docker images

postgres                   12-alpine           b5a8143fc58d        27 hours ago        158MB
python                     3.8.2-alpine        6c32e2504283        6 months ago        107MB

 

• movies镜像 的 环境配置

与db docker 相对应的环境变量配置，下载： .env.dev

DEBUG=1
SECRET_KEY=foo
DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1]

SQL_ENGINE=django.db.backends.postgresql
SQL_USER=movies
SQL_PASSWORD=movies
SQL_DATABASE=movies_dev
SQL_HOST=movies-db
SQL_PORT=5432
DATABASE=postgres

 

• movies镜像 配置 postgres 接口

Django的db配置：app/drf_project/settings.py

如下，os.environ.get(<key>，<默认值>)

 

 

三、docker container 执行

• 测试 postgres 

(1) movies 要在等待中查看下 db 的启动情况，运行良好后，再沟通。

if [ "$DATABASE" = "postgres" ]
then
    echo "Waiting for postgres..."

    while ! nc -z $SQL_HOST $SQL_PORT; do
      sleep 0.1
    done

    echo "PostgreSQL started"
fi

python manage.py flush --no-input
python manage.py migrate

exec "$@"

并相应的添加内容到 Dockerfile 中，以及环境变量设置。

# copy entrypoint.sh
COPY ./entrypoint.sh /usr/src/app/entrypoint.sh
RUN chmod +x /usr/src/app/entrypoint.sh

# copy project
COPY . /usr/src/app/

# run entrypoint.sh
ENTRYPOINT ["/usr/src/app/entrypoint.sh"]

 

(2) docker-compose exec 命令：在其中一个运行的容器中执行命令。

$ docker-compose exec movies    python manage.py migrate --noinput
$ docker-compose exec movies-db psql --username=movies --dbname=movies_dev 

  

• 容器状态

启动，并查看两个container的状态。

$ docker-compose up -d --build

Ensure http://localhost:8000/ still works.

jeffrey@unsw-ThinkPad-T490:app$ docker ps
CONTAINER ID        IMAGE                      COMMAND                  CREATED             STATUS              PORTS                    NAMES
ed0c58ad1f34        django-tdd-docker_movies   "/usr/src/app/entryp…"   4 hours ago         Up 4 hours          0.0.0.0:8000->8000/tcp   django-tdd-docker_movies_1
d5619e55aa0b        postgres:12-alpine         "docker-entrypoint.s…"   4 hours ago         Up 4 hours          5432/tcp                 django-tdd-docker_movies-db_1

 

 

 

Pytest

一、目录结构

tests/
├── __init__.py
├── movies
│   └── test_models.py
└── test_foo.py

 

 

二、test ping 

路由 REST API 搭建套路。

(1) test invoke --> request

Goto: Django路由之reverse反解析

# app/tests/test_foo.py

import json
from django.urls import reverse

def test_ping(client):
    url      = reverse("ping")
    response = client.get(url)
    content  = json.loads(response.content)

    assert response.status_code == 200
    assert content["ping"] == "pong!"

(2) request --> urls.py

# app/drf_project/urls.py

from django.contrib import admin
from django.urls import path

from .views import ping


urlpatterns = [
    path('admin/', admin.site.urls),
    path('ping/', ping, name="ping"),
]

(3) urls.py --> views.py

# app/drf_project/views.py

from django.http import JsonResponse


def ping(request):
    data = {"ping": "pong!"}
    return JsonResponse(data)

 

 

三、fixture 固件

暂时没有用到。此处表示，每一次session测试前自动设置一些db的环境变量。

import os

from django.conf import settings

import pytest


DEFAULT_ENGINE = "django.db.backends.postgresql_psycopg2"


@pytest.fixture(scope="session")
def django_db_setup():
    settings.DATABASES["default"] = {
        "ENGINE": os.environ.get("DB_TEST_ENGINE", DEFAULT_ENGINE),
        "HOST": os.environ["DB_TEST_HOST"],
        "NAME": os.environ["DB_TEST_NAME"],
        "PORT": os.environ["DB_TEST_PORT"],
        "USER": os.environ["DB_TEST_USER"],
        "PASSWORD": os.environ["DB_TEST_PASSWORD"],
    }

  

End.