[Django] 05 - Test stage in CI

第一部分

Test in CI

Using the image that we created in the build stage along with the Postgres service, we run Pytest, Flake8, Black, and isort in the test stage.

  script:
    - cd app
    - pytest -p no:warnings --cov=.
    - flake8 .
    - black --check --exclude=migrations .
    - isort ./*/*.py --check-only

思路：关掉大部分info print。如果还有print，则表示测试没有过。

第二部分

一、Code Coverage and Quality

文件：requirements.txt

Django==3.0.5
dj-database-url==0.5.0
djangorestframework==3.11.0
gunicorn==20.0.4
psycopg2-binary==2.8.5
pytest-cov==2.8.1
pytest-django==3.9.0
pytest==5.4.1
whitenoise==5.0.1
six==1.11.0

文件：.coveragerc file

[run]
omit =
  *apps.py,
  *settings.py,
  *urls.py,
  *wsgi.py,
  *asgi.py,
  manage.py,
  *migrations/*,
  *tests/*,
branch = True

[执行]

django-tdd-docker$ docker-compose exec movies pytest -p no:warnings --cov=.
======================================================= test session starts =========================================================
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.10.0, pluggy-0.13.1
django: settings: drf_project.settings (from ini)
rootdir: /usr/src/app, inifile: pytest.ini
plugins: django-3.9.0, cov-2.8.1
collected 10 items                                                                                                                                                                         

tests/movies/test_models.py .                                                                                                  [ 10%]
tests/movies/test_views.py ....                                                                                                [ 50%]
tests/test_foo.py ..                                                                                                           [ 70%]
tests/movies/test_serializers.py ..                                                                                            [ 90%]
tests/movies/test_views.py .                                                                                                   [100%]

----------- coverage: platform linux, python 3.8.2-final-0 -----------
Name                      Stmts   Miss Branch BrPart  Cover
-----------------------------------------------------------
drf_project/__init__.py       0      0      0      0   100%
drf_project/views.py          4      0      0      0   100%
movies/__init__.py            0      0      0      0   100%
movies/admin.py              11      0      4      0   100%
movies/models.py             12      0      0      0   100%
movies/serializers.py         7      0      0      0   100%
movies/tests.py               1      0      0      0   100%
movies/views.py              24      2      2      0    92%
-----------------------------------------------------------
TOTAL                        59      2      6      0    97%

二、测试：三驾马车

Ref: https://testdriven.io/courses/tdd-django/remaining-routes/

如果代码除了小问题，那么按照如下实操一遍。

flake8 格式检查

$ docker-compose exec movies flake8 .

./tests/test_foo.py:10:1: E302 expected 2 blank lines, found 1
./tests/movies/test_views.py:1:1: F401 'json' imported but unused
./tests/movies/test_views.py:63:1: E302 expected 2 blank lines, found 1
./tests/movies/test_views.py:65:80: E501 line too long (87 > 79 characters)
./movies/views.py:2:1: F401 'django.shortcuts.render' imported but unused
./movies/views.py:16:1: E303 too many blank lines (5)
./movies/admin.py:10:1: E303 too many blank lines (3)
./movies/models.py:12:10: E221 multiple spaces before operator
./movies/models.py:13:10: E221 multiple spaces before operator
./movies/models.py:14:9: E221 multiple spaces before operator
./movies/tests.py:1:1: F401 'django.test.TestCase' imported but unused
./movies/migrations/0001_initial.py:21:80: E501 line too long (114 > 79 characters)
./movies/migrations/0001_initial.py:22:80: E501 line too long (88 > 79 characters)
./movies/migrations/0001_initial.py:23:80: E501 line too long (103 > 79 characters)
./movies/migrations/0001_initial.py:24:80: E501 line too long (196 > 79 characters)
./movies/migrations/0001_initial.py:25:80: E501 line too long (329 > 79 characters)
./movies/migrations/0001_initial.py:26:80: E501 line too long (103 > 79 characters)
./movies/migrations/0001_initial.py:27:80: E501 line too long (102 > 79 characters)
./movies/migrations/0001_initial.py:28:80: E501 line too long (103 > 79 characters)
./movies/migrations/0001_initial.py:29:80: E501 line too long (165 > 79 characters)
./movies/migrations/0001_initial.py:30:80: E501 line too long (203 > 79 characters)
./movies/migrations/0001_initial.py:31:80: E501 line too long (117 > 79 characters)
./movies/migrations/0001_initial.py:32:80: E501 line too long (266 > 79 characters)
./movies/migrations/0001_initial.py:33:80: E501 line too long (229 > 79 characters)
./movies/migrations/0002_movie.py:16:80: E501 line too long (114 > 79 characters)
./drf_project/settings.py:21:2: W291 trailing whitespace
./drf_project/settings.py:24:2: W291 trailing whitespace
./drf_project/settings.py:27:2: W291 trailing whitespace
./drf_project/settings.py:28:27: E231 missing whitespace after ','
./drf_project/settings.py:33:80: E501 line too long (86 > 79 characters)
./drf_project/settings.py:47:22: E261 at least two spaces before inline comment
./drf_project/settings.py:48:14: E261 at least two spaces before inline comment
./drf_project/settings.py:95:80: E501 line too long (85 > 79 characters)
./drf_project/settings.py:108:80: E501 line too long (91 > 79 characters)
./drf_project/settings.py:111:80: E501 line too long (81 > 79 characters)
./drf_project/settings.py:114:80: E501 line too long (82 > 79 characters)
./drf_project/settings.py:117:80: E501 line too long (83 > 79 characters)

black

Next, let's add Black, which is used for formatting your code so that "code looks the same regardless of the project you're reading". This helps to speed up code reviews. "Formatting becomes transparent after a while and you can focus on the content instead."

针对代码风格不一致问题，导致的维护成本过高，针对性的镇定代码风格统一标准，是很有必要的。目前市面上用的比较多的python代码格式化工具有YAPF、Black。

Black，号称不妥协的代码格式化工具，它检测到不符合规范的代码风格直接就帮你全部格式化好，根本不需要你确定，直接替你做好决定。从而节省关注代码规范的时间和精力，关注编程。

$ docker-compose exec movies black --exclude=migrations .

isort

发现“不同”。

$ docker-compose up -d --build
$ docker-compose exec movies /bin/sh -c "isort ./*/*.py --check-only"
$ docker-compose exec movies /bin/sh -c "isort ./*/*.py --diff"

更新 by “不同”。

$ docker-compose exec movies /bin/sh -c "isort ./*/*.py"

最后验证，不应该再有问题，毕竟都自动修改过了呢。

$ docker-compose exec movies flake8 .
$ docker-compose exec movies black --check --exclude=migrations .
$ docker-compose exec movies /bin/sh -c "isort ./*/*.py --check-only"

一些问题搜集，如何解决呢？

$ http GET http://localhost:8000/api/movies/1/

Traceback (most recent call last):
  File "/usr/lib/command-not-found", line 28, in <module>
    from CommandNotFound import CommandNotFound
  File "/usr/lib/python3/dist-packages/CommandNotFound/CommandNotFound.py", line 19, in <module>
    from CommandNotFound.db.db import SqliteDatabase
  File "/usr/lib/python3/dist-packages/CommandNotFound/db/db.py", line 5, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'

三、http测试

就是直接测试 rest api。

jeffrey@unsw-ThinkPad-T490:django-tdd-docker$ http GET http://localhost:8000/api/movies/
HTTP/1.1 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Length: 2
Content-Type: application/json
Date: Fri, 18 Dec 2020 10:10:50 GMT
Server: WSGIServer/0.2 CPython/3.8.2
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

[]


jeffrey@unsw-ThinkPad-T490:django-tdd-docker$ http --json POST http://localhost:8000/api/movies/ title=Fargo genre=comedy year=1996
HTTP/1.1 201 Created
Allow: GET, POST, HEAD, OPTIONS
Content-Length: 145
Content-Type: application/json
Date: Fri, 18 Dec 2020 10:11:21 GMT
Server: WSGIServer/0.2 CPython/3.8.2
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
    "created_date": "2020-12-18T10:11:21.689819Z",
    "genre": "comedy",
    "id": 1,
    "title": "Fargo",
    "updated_date": "2020-12-18T10:11:21.689837Z",
    "year": "1996"
}


jeffrey@unsw-ThinkPad-T490:django-tdd-docker$ http GET http://localhost:8000/api/movies/
HTTP/1.1 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Length: 147
Content-Type: application/json
Date: Fri, 18 Dec 2020 10:11:32 GMT
Server: WSGIServer/0.2 CPython/3.8.2
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

[
    {
        "created_date": "2020-12-18T10:11:21.689819Z",
        "genre": "comedy",
        "id": 1,
        "title": "Fargo",
        "updated_date": "2020-12-18T10:11:21.689837Z",
        "year": "1996"
    }
]


jeffrey@unsw-ThinkPad-T490:django-tdd-docker$ http GET http://localhost:8000/api/movies/1/
HTTP/1.1 200 OK
Allow: GET, PUT, DELETE, HEAD, OPTIONS
Content-Length: 145
Content-Type: application/json
Date: Fri, 18 Dec 2020 10:11:59 GMT
Server: WSGIServer/0.2 CPython/3.8.2
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
    "created_date": "2020-12-18T10:11:21.689819Z",
    "genre": "comedy",
    "id": 1,
    "title": "Fargo",
    "updated_date": "2020-12-18T10:11:21.689837Z",
    "year": "1996"
}


jeffrey@unsw-ThinkPad-T490:django-tdd-docker$ http --json PUT http://localhost:8000/api/movies/1/ title=Fargo genre=thriller year=1996
HTTP/1.1 200 OK
Allow: GET, PUT, DELETE, HEAD, OPTIONS
Content-Length: 147
Content-Type: application/json
Date: Fri, 18 Dec 2020 10:13:01 GMT
Server: WSGIServer/0.2 CPython/3.8.2
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
    "created_date": "2020-12-18T10:11:21.689819Z",
    "genre": "thriller",
    "id": 1,
    "title": "Fargo",
    "updated_date": "2020-12-18T10:13:01.847127Z",
    "year": "1996"
}


jeffrey@unsw-ThinkPad-T490:django-tdd-docker$ http DELETE http://localhost:8000/api/movies/1/
HTTP/1.1 204 No Content
Allow: GET, PUT, DELETE, HEAD, OPTIONS
Content-Length: 0
Date: Fri, 18 Dec 2020 10:13:31 GMT
Server: WSGIServer/0.2 CPython/3.8.2
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY



jeffrey@unsw-ThinkPad-T490:django-tdd-docker$ http GET http://localhost:8000/api/movies/
HTTP/1.1 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Length: 2
Content-Type: application/json
Date: Fri, 18 Dec 2020 10:13:37 GMT
Server: WSGIServer/0.2 CPython/3.8.2
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

[]

Pytest Monkeypatching

Ref: 猴子补丁(Monkey Patching)【初识】

Ref: https://testdriven.io/courses/tdd-django/pytest-monkeypatching/

/* 暂时略 */

End.

posted @ 2020-12-16 18:54 郝壹贰叁阅读(180) 评论(0) 编辑收藏举报

刷新页面返回顶部

机器学习水很深

We all have two lives. The second one starts when we realize that we only have one. --- Tom Hiddleston