在django中使用celery异步任务和定时任务

在django中使用celery

 

Django版本1.9.8

pip install django-celery

Redis要指定版本，默认的3.x版本有问题

Pip uninstall redis

pip install redis==2.10.6

创建django的app celery_course，并配置运行起来，作为celery的测试应用

Python manage.py startapp celery_course

将celery_course这个app拖入apps这个集合文件夹中

把celery_course加入settings.py配置中

定义views这个视图

Celery_course/views.py

# _*_ coding:utf-8 _*_
from django.http import JsonResponse
from celery_course.tasks import CourseTask
from django.views.generic.base import View


class DoView(View):
    def get(self, request):
        # 执行异步任务
        print 'start to request'
        # CourseTask.delay()
        CourseTask.apply_async(args=('hello',),queue='work_queue')
        print 'end do request'
        return JsonResponse({'result': 'ok'})

定义url

Mxonline/urls.py

url(r'^celery_course/', include('celery_course.urls', namespace="celery_course")),

Celery_course/urls.py
from django.conf.urls import url
from .views import DoView

urlpatterns = [
    # 课程机构首页
    url(r'^do/$', DoView.as_view(), name="do"),
]

启动django应用，访问

Python manage.py runserver 0.0.0.0:8000

 

能正常访问，说明django配置没问题

 

接下来配置celery相关的内容

定义任务tasks.py

Celery_course/tasks.py

# _*_ coding:utf-8 _*_
# __author__ == 'jack'
import time
from celery.task import Task


class CourseTask(Task):
    name = 'celery-course-task'

    def run(self, *args, **kwargs):
        print 'start celery_course task'
        time.sleep(5)
        print 'args={}, kwargs={}'.format(args, kwargs)
        print 'end course task '

配置celery

Mxonline/celeryconfig.py

# _*_ coding:utf-8 _*_
# __author__ == 'jack'
import djcelery
from datetime import timedelta
djcelery.setup_loader()


CELERY_IMPORTS = (
    'celery_course.tasks',
)

CELERY_QUEUES = {
    'beat_tasks': {
        'exchange': 'beat_tasks',
        'exchange_type': 'direct',
        'binding_key': 'beat_tasks',
    },
    'work_queue': {
        'exchange': 'work_queue',
        'exchange_type': 'direct',
        'binding_key':'work_queue'
    }
}

# 默认队列
CELERY_DEFAULT_QUEUE = 'work_queue'

# 有些情况可以防止死锁
CELERYD_FORCE_EXECV = True

# 设置并发的worker数量
CELERYD_CONCURRENCY = 4

# 允许重试
CELERY_ACKS_LATE = True

# 每个worker最多执行100个任务被销毁，可以防止内存泄露
CELERYD_MAX_TASKS_PER_CHILD = 100

# 单个任务的最大运行时间
CELERYD_TASK_TIME_LIMIT = 12 * 30

# 配置定时任务
CELERYBEAT_SCHEDULE = {
    'task1': {
        'task': 'celery-course-task',
        'schedule': timedelta(seconds=5),
        'options':{
            'queue': 'beat_tasks'
        }
    }
}

将celery和django联系起来，修改settings.py配置加入djcelery

导入celeryconfig配置

 

# 导入celery相关的配置
from .celeryconfig import *
BROKER_BACKEND = 'redis'
BROKER_URL = 'redis://localhost:6379/1'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/2'

 

启动django

启动worker

(mxonline) D:\python\mxonline>python manage.py celery worker --loglevel=info

启动beat定时任务

(mxonline) D:\python\mxonline>python manage.py celery beat --loglevel=info