django+uwsgi+nginx+pandas 导出excel超时问题

django+uwsgi+nginx+pandas 导出excel超时问题

一、问题现象和日志报错

　　之前在项目实现了excel导入导出：django导入导出excel实践，之前一直稳定运行，突然得知导出用户信息时出现nginx错误报告：

　　查看nginx日志，报错信息如下所示：

upstream timed out (110: Connection timed out) while reading response header from upstream, client: 119.157.163.211,
server: localhost, request: "GET /xxxxx/export/ HTTP/1.1", upstream: "uwsgi://127.0.0.1:7080", host: "xxxx.com:8100", referrer: "http://xxxxx.com:8100/xxxxxxx/"
　　查看uwsgi日志，报错信息如下所示：

Tue Jul 30 10:25:03 2019 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /stark/crm/customer/export/ (ip 113.57.163.211) !!!
Tue Jul 30 10:25:03 2019 - uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 341] during GET /stark/crm/customer/export/ (113.57.163.211)
OSError: write error

二、解决方法

1、调整uwsgi配置
　　ignore-singpipe 使uWSGI不显示SIGPIPE错误；

　　ignore-write-errors 使它不显示诸如uwsgi_response_writev_headers_and_body_do的错误；

　　disable-write-exception 防止 OSError写入时生成。

 　　因此在uwsgi.ini中添加如下配置：

    ignore-sigpipe=true
    ignore-write-errors=true
    disable-write-exception=true	
    
2、调整nginx配置  
        uwsgi_send_timeout 1080;        # 指定连接到后端uWSGI的超时时间。
        uwsgi_connect_timeout 1080;     # 指定向uWSGI传送请求的超时时间，完成握手后向uWSGI传送请求的超时时间。
        uwsgi_read_timeout 1080;        # 指定接收uWSGI应答的超时时间，完成握手后接收uWSGI应答的超时时间。
        如：
        	 server {
                listen  8889;
                server_name  127.0.0.1;
                charset UTF-8;
                client_max_body_size 75M;
                location / { 
                        include uwsgi_params;                   
                        uwsgi_pass 127.0.0.1:9000;           
                        uwsgi_param UWSGI_SCRIPT xxxx.wsgi;
                        uwsgi_param UWSGI_CHDIR  /home/xxxx/;
                        uwsgi_send_timeout 1080;        
                        uwsgi_connect_timeout 1080;
                        uwsgi_read_timeout 1080;             
                    }   	
                }
3.重启nginx
	nginx -s reload

三.pandas报错

1.import pandas._libs.groupby as libgroupby

解决办法

更新pandas， pip3 install -U pandas

2.AttributeError: 'XlsxWriter' object has no attribute 'save'

解决办法

将save()改为close()