Django与celery集成:异步任务原理和过程
0.原理和架构
a.客户发送请求到django;
b.django产生任务(要执行的函数);
c.django把任务丢给celery的broker
d.celery的worker从broker拿到任务并且执行;
e.worker执行后保存结果到后端数据库;
1.在django里面配置celery的目录结构
PS D:\djangotest\myrecrument> tree D:. ├─.idea │ └─inspectionProfiles ├─celery │ └─__pycache__ │ - manage.py ├─interview │ ├─management │ │ └─commands │ │ └─__pycache__ │ ├─migrations │ │ └─__pycache__ │ └─__pycache__ ├- db.sqlite3 ├─myrecrument │ └─settings.py └─tmp
2.关联django和celery
2. cat D:\djangotest\myrecrument\myrecrument\celery.py
3. cat D:\djangotest\myrecrument\myrecrument\settings.py
CELERY_BROKER_URL = 'redis://106.45.274.145:30013/0' CELERY_RESULT_BACKEND = 'redis://106.45.274.145:30013/1' CELERY_ACCEPT_CONTENT = ['application/json'] CELERY_RESULT_SERIALIZER = 'json' CELERY_TASK_SERIALIZER = 'json' CELERY_TIMEZONE = 'Asia/Shanghai' CELERYD_MAX_TASKS_PER_CHILD = 10 CELERYD_LOG_FILE = os.path.join(BASE_DIR, "logs", "celery_work.log") CELERYBEAT_LOG_FILE = os.path.join(BASE_DIR, "logs", "celery_beat.log")
4.配置异步发送钉钉消息D:\djangotest\myrecrument\interview> cat .\dingtalk.py
#coding=utf-8 from dingtalkchatbot.chatbot import DingtalkChatbot from django.conf import settings def send(message, at_mobiles=[]): # 引用 settings里面配置的钉钉群消息通知的WebHook地址: webhook = settings.DINGTALK_WEB_HOOK # 初始化机器人小丁, # 方式一:通常初始化方式 xiaoding = DingtalkChatbot(webhook) # 方式二:勾选“加签”选项时使用(v1.5以上新功能) # xiaoding = DingtalkChatbot(webhook, secret=secret) # Text消息@所有人 xiaoding.send_text(msg=('面试通知: %s' %message), at_mobiles = ['14743373423'] )
5.D:\djangotest\myrecrument\interview> cat .\task.py #配置任务task
from __future__ import absolute_import, unicode_literals
from celery import shared_task
from .dingtalk import send
@shared_task
def send_dingtalk_message(message):
send(message)
6.在interview的admin.py D:\djangotest\myrecrument\interview> cat .\admin.py #应用tasks
from .tasks import send_dingtalk_message ... def notify_interviewer(modeladmin,request,queryset): candidates = ' ' for obj in queryset: candidates = obj.username + ';' + candidates send_dingtalk_message.delay(candidates)
7.启动监控flower:D:\djangotest\myrecrument\celery> celery -A tasks flower broker='redis://redis:30013/0'
8.启动celery的worker:D:\djangotest\myrecrument> celery -A myrecrument worker --loglevel=INFO -P eventlet
9.启动项目:D:\djangotest\myrecrument> python .\manage.py runserver 0.0.0.0:8000
10.flower的效果
参考:https://docs.celeryproject.org/en/stable/django/first-steps-with-django.html First steps with Django