https协议+三种数据库实现限流+安全防护

1 http 和 https

1.1 http 和 https 的区别

• http 明文（报文明文）可以篡改
• https 加密（报文密文）

1.2 如何访问https

• pip install django-sslserver
• 终端中 python manage.py runsslserver 启动项目

2 项目中的限流操作

• 用来存储存ip地址，明确用户身份，可以用来防爬虫
• 使用场景：钉钉机器人发送短信验证，不能在5秒内，向服务端发起三次以上的请求

2.1 mysql实现

• mysql中存两个数据，ip+时间戳

2.2.1 装饰器

• 装饰器实际上是一个闭包，延伸了作用域的方法，可以再在内部访问一个方法外部的变量（外部的变量是全局变量）。
• 用法：语法常

from django.http import HttpResponse
from userapp.models import InfoVisit, BlackList


#  the Decorator to make sure the only one ip to visit us.(mysql)

def visit_info_mysql_decorator(func):
    def inner(request, *args, **kwargs):
        # try:

        # don't think about that nginx may course 127.0.0.1:8000 at this time.

        #     if request.META['HTTP_X_FORWARDED_FOR']:
        #         ip = request.META['HTTP_X_FORWARDED_FOR']
        # except Exception as e:
        #      print('ok')

        ip = request.META['REMOTE_ADDR']
        obj = InfoVisit.objects.filter(net_ip=ip, is_delete=False).first()
        if obj:
            visit_time = obj.visit_time
            import datetime
            timediff = datetime.datetime.now() - visit_time
            if timediff.total_seconds() > 5:
                InfoVisit.objects.create(net_ip=ip, is_delete=False)
                obj.is_delete = True
                obj.save()
            else:
                return HttpResponse('恶意访问')
        else:
            print('first visit')
            InfoVisit.objects.create(net_ip=ip, is_delete=False)
        return func(request, *args, **kwargs)
    return inner

2.2.2 中间件

from django.http import HttpResponse
from django.utils.deprecation import MiddlewareMixin
from userapp.models import InfoVisit, BlackList


# 此页面所有拦截都是针对钉钉机器人，用户小于5秒访问第二次的！

# mysql -----------------middleware

class VisitInfoMysqlMiddleWare(MiddlewareMixin):
    def process_request(self, request):
        path = request.path_info
        print(123,path)
        if path in ('/user/dingding/'):
            ip = request.META['REMOTE_ADDR']
            print(234, ip)
            obj = InfoVisit.objects.filter(net_ip=ip, is_delete=False).first()
            if obj:
                visit_time = obj.visit_time
                import datetime
                timediff = datetime.datetime.now() - visit_time
                if timediff.total_seconds() > 5:
                    InfoVisit.objects.create(net_ip=ip, is_delete=False)
                    obj.is_delete = True
                    obj.save()
                else:
                    return HttpResponse('恶意访问')
            else:
                print('first visit')
                InfoVisit.objects.create(net_ip=ip, is_delete=False)

2.2 redis 实现

• redis 存key-values，判断key存不存在，不需要存value（存内存）

2.2.1 装饰器

from django.http import HttpResponse
from userapp.models import InfoVisit

#  the Decorator to make sure the only one ip to visit us.(redis)

import redis
r = redis.Redis(host='127.0.0.1', port=6379, db=5)
def visit_info_redis_decorator(func):
    def inner(request, *args, **kwargs):
        # try:
        # don't think about that nginx may course 127.0.0.1:8000 at this time.

        #     if request.META['HTTP_X_FORWARDED_FOR']:
        #         ip = request.META['HTTP_X_FORWARDED_FOR']
        # except Exception as e:
        #      print('ok')

        ip = request.META['REMOTE_ADDR']
        value = r.get(ip)
        if value:
            return HttpResponse('恶意访问')
        else:
            r.set(ip, 'value')
            r.expire(ip, 5)
            # print('first visit')
        return func(request, *args, **kwargs)
    return inner

2.2.2 中间件

# -*- coding: utf-8 -*-
from django.http import HttpResponse
from django.utils.deprecation import MiddlewareMixin
from userapp.models import InfoVisit
# 此页面所有拦截都是针对钉钉机器人，用户小于5秒访问第二次的！


# redis ---------------------middleware
import redis
r = redis.Redis(host='127.0.0.1', port=6379, db=5)
class VisitInfoRedisMiddleWare(MiddlewareMixin):
    def process_request(self, request):
        path = request.path_info
        # print(123,path)
        if path in ('/user/dingding/'):
            ip = request.META['REMOTE_ADDR']
            value = r.get(ip)
            if value:
                return HttpResponse('恶意访问')
            else:
                r.set(ip, 'value')
                r.expire(ip, 5)

2.3 mongodb 实现

• mongodb中的过期索引，只能实现60秒一次的检测功能，所以并不准确，是mongodb对限流操作实现的一大弊端，可以通过以下代码自行检验
• 还遇到了一个坑，就是FQ软件开启的时候，不能用代理下载，访问国外网站！
• 安装的时候，注意安全软件关闭或者给予权限
• windows + r =========> services.msc 可以查看启动

# 指定bin目录下检测启动
C:\Program Files\MongoDB\Server\4.0\bin>mongo
MongoDB shell version v4.0.4
connecting to: mongodb://127.0.0.1:27017
Implicit session: session { "id" : UUID("e22f6748-06d6-446b-b3dc-46bc7b9e1cde") }
MongoDB server version: 4.0.4
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
        http://docs.mongodb.org/
Questions? Try the support group
        http://groups.google.com/group/mongodb-user
Server has startup warnings:
2020-12-18T23:33:51.408+0800 I CONTROL  [initandlisten]
2020-12-18T23:33:51.408+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2020-12-18T23:33:51.408+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2020-12-18T23:33:51.408+0800 I CONTROL  [initandlisten]
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).

The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.

To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---
# 出现 < 代表安装成功
# 可以进行简单的计算
> 2+3
5
# 存在runoob即使用，不存在就创建（详细参考P2的mongodb攻略）
> use runoob
switched to db runoob
> db
runoob
> show dbs
admin   0.000GB
config  0.000GB
local   0.000GB
> db
runoob

• 因为不够准确，推荐不使用过期索引！以下代码采用了过期索引，仅供参考

2.2.1 装饰器

# -*- coding: utf-8 -*-
# 此页面所有拦截都是针对钉钉机器人，小于5秒访问第二次的！

from django.http import HttpResponse
from userapp.models import InfoVisit

#  the Decorator to make sure the only one ip to visit us.(mongodb)

import pymongo
client = pymongo.MongoClient(host='localhost', port=27017)
db = client['runoob']
set = db['sub']

def visit_info_mongo_decorator(func):

    import datetime
    def inner(request, *args, **kwargs):
        ip = request.META['REMOTE_ADDR']
        value = set.find_one({'ip': ip})
        print(123,value)
        if value:
            return HttpResponse('恶意访问')
        else:
            timestamp = datetime.datetime.now()
            print(123123)
            set.ensure_index('Date', expireAfterSeconds=10)
            set.insert({'ip': ip, 'Date': timestamp})
            print('first visit')
        return func(request, *args, **kwargs)
    return inner

2.2.2 中间件

# -*- coding: utf-8 -*-
from django.http import HttpResponse
from django.utils.deprecation import MiddlewareMixin
from userapp.models import InfoVisit
# 此页面所有拦截都是针对钉钉机器人，用户小于5秒访问第二次的！


# mongo ---------------------middleware
import pymongo
client = pymongo.MongoClient(host='localhost', port=27017)
db = client['runoob']
set = db['sub']
class VisitInfoMongoMiddleWare(MiddlewareMixin):
    def process_request(self, request):
        path = request.path_info
        if path in ('/user/dingding/'):
            import datetime
            ip = request.META['REMOTE_ADDR']
            value = set.find_one({'ip': ip})
            print(123, value)
            if value:
                return HttpResponse('恶意访问')
            else:
                timestamp = datetime.datetime.now()
                print(123123)
                set.ensure_index('Date', expireAfterSeconds=10)
                # 没办法实时监测，因为一分钟才检查一次
                set.insert({'ip': ip, 'Date': timestamp})
                print('first visit')

2.4 总结

• 限流最好用的就是redis，和mysql结合实现安全防护，详看本章 4

3 在函数中调用的方式

3.1 装饰器

from django.utils.decorators import method_decorator
class DingDingView(APIView):
    @method_decorator(visit_info_mysql_decorator, name='post')
    @method_decorator(visit_info_redis_decorator, name='post')
    @method_decorator(visit_info_mongo_decorator, name='post')
    def post(self, request):
        username = request.data.get('username')
        # 这里实现一个场景，就是只有在登录的情况下才可以发送验证码（本来想使用token，但是累觉没必要，前端输入的同时，表单绑定记录下来就好了！何必难为自己T.T，做一次小测试，具体情况具体分析就可以了）
        num = message_code()
        dingding_robot(num)
        request.session['num'] = num
        request.session.set_expiry(300)
        if not request.session.session_key:
            request.session.create()
        return Response(
            {'msg': '发送成功', 'code':200, 'num':num, 'session_key': request.session.session_key}
        )

3.2 中间件

• settings中注册

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'corsheaders.middleware.CorsMiddleware',
    'django.middleware.common.CommonMiddleware',
    # 'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    # mysql middleware
    'userapp.middlewares.VisitInfoMysqlMiddleWare',
    # redis middleware
    'userapp.middlewares.VisitInfoRedisMiddleWare',
    # mongo middleware
    'userapp.middlewares.VisitInfoMongoMiddleWare'
    ]

4 安全防护

• 安全防护是基于限流操作的，为了防止爬虫，我们要设置安全防护。但是也要注意到的是，同一台电脑，可以由多个用户访问，所以设置黑名单的时候，要设置用户名
• 在同一个局域网内，可以通过ip访问不同的服务器，启动代码是

python manage.py runserver 0.0.0.0:8000
# 局域网

• 设计表结构的时候，采用了唯一索引：唯一索引作用是排重，同时减少select语句的io操作，针对username做唯一索引，ip一样，用户名一样才针对查询。

4.1 黑名单表设计

• 此处采用了 django 表结构的设计

class BlackList(models.Model):
    username = models.CharField(max_length=30)
    net_ip = models.CharField(max_length=40)
    lock_time = models.IntegerField(default=3600)

    class Meta:
        index_together = ['username', 'net_ip']
        unique_together = ['username', 'net_ip']
        db_table = 'black_list'
        
# 联合索引必须一起添加，否则要报错

4.2 redis + mysql实现操作

• redis 进行限流
• mysql存储黑名单信息

4.2.1 middleware

# redis ---------------------middleware
import redis
r = redis.Redis(host='127.0.0.1', port=6379, db=5, decode_responses=True)
class VisitInfoRedisMiddleWare(MiddlewareMixin):
    def process_request(self, request):
        path = request.path_info
        if path in ('/user/dingding/'):
            username = request.data.get('username')
            ip = request.META['REMOTE_ADDR']
            # 为空要报错哦
            if BlackList.objects.filter(username=username, net_ip=ip):
                return HttpResponse('账号已经被停用')
            else:
                value = r.get(ip)
                print('this is value', value)
                # value------->type:str
                if value is not None and int(value) > 2:
                    BlackList.objects.create(username=username, net_ip=ip)
                    return HttpResponse('恶意访问,您即将被封号')
                else:
                    r.incrby(ip, 1)
                    # 这里我思考了一个问题，就是自增会不会刷新过期时间，实际上是不会的，查询资料发现
                    # 要刷新过期时间还需要新的解决方案，符合逻辑！
                    r.expire(ip, 10)

4.2.2 director

#  the Decorator to make sure the only one ip to visit us.(redis)

import redis
r = redis.Redis(host='127.0.0.1', port=6379, db=5, decode_responses=True)
def visit_info_redis_decorator(func):
    def inner(request, *args, **kwargs):
        username = request.data.get('username')
        ip = request.META['REMOTE_ADDR']
        # 为空要报错哦
        if BlackList.objects.filter(username=username, net_ip=ip):
            return HttpResponse('账号已经被停用')
        else:
            value = r.get(ip)
            print('this is value', value)
            # value------->type:str
            if value is not None and int(value) > 2:
                BlackList.objects.create(username=username, net_ip=ip)
                return HttpResponse('恶意访问,您即将被封号')
            else:
                r.incrby(ip, 1)
                # 这里我思考了一个问题，就是自增会不会刷新过期时间，实际上是不会的，查询资料发现
                # 要刷新过期时间还需要新的解决方案，符合逻辑！
                r.expire(ip, 10)
        return func(request, *args, **kwargs)
    return inner