[z]Django UnicodeEncodeError when uploading files

说明一下，我是将Django部署到Debian6.0上，使用nginx和uwsgi配置服务，一切就绪后，发现中文文件名的文件无法访问，且在上传中文文件名的文件时出现以下错误：

UnicodeEncodeError at /admin/tech/resource/2/

'ascii' codec can't encode characters in position 58-65: ordinal not in range(128)

Request Method:     POST
Request URL:     http://www.aero-sim.com:8000/admin/tech/resource/2/
Django Version:     1.8
Exception Type:     UnicodeEncodeError
Exception Value:     

'ascii' codec can't encode characters in position 58-65: ordinal not in range(128)

Exception Location:     /usr/local/python2.7.9/lib/python2.7/genericpath.py in exists, line 18
Python Executable:     /usr/local/python2.7.9/bin/uwsgi
Python Version:     2.7.9
Python Path:     

['.',
 '',
 '/usr/local/python2.7.9/lib/python2.7/site-packages/pip-6.0.8-py2.7.egg',
 '/usr/local/python2.7.9/lib/python2.7/site-packages/MySQL_python-1.2.3-py2.7-linux-i686.egg',
 '/usr/local/python2.7.9/lib/python2.7/site-packages/django_bootstrap3-5.4.0-py2.7.egg',
 '/usr/local/python2.7.9/lib/python2.7/site-packages/django_admin_bootstrapped-2.5.0-py2.7.egg',
 '/usr/local/python2.7.9/lib/python27.zip',
 '/usr/local/python2.7.9/lib/python2.7',
 '/usr/local/python2.7.9/lib/python2.7/plat-linux2',
 '/usr/local/python2.7.9/lib/python2.7/lib-tk',
 '/usr/local/python2.7.9/lib/python2.7/lib-old',
 '/usr/local/python2.7.9/lib/python2.7/lib-dynload',
 '/usr/local/python2.7.9/lib/python2.7/site-packages']

Server time:     Wed, 29 Apr 2015 08:47:52 +0000

找到以下解决办法：

第一步：将系统编码改成 zh_CN.UTF8，参考http://www.111cn.net/sys/nginx/63978.htm

# nano /etc/profile

添加以下代码，然后保存重启系统！

export LANG=zh_CN.UTF8
export LC_ALL=zh_CN.UTF8

第二步：更改uwsgi ini配置文件

在*.ini文件中添加下面一行代码

env LC_ALL=zh_CN.UTF-8 

害惨我了，之前配置成env=LC_ALL=zh_CN.UTF-8了

第三步：更改nginx配置文件

将charset改为 utf-8

重启nginx服务，

service nginx restart

启动uwsgi服务

 uwsgi --emperor /etc/uwsgi/vassals --uid root --gid root

OK，搞定！

以下是外网资料，供参考

http://tech.barszcz.info/2012/05/17/django-unicodeencodeerror-when-uploading-files/

Often when you try to upload files with non-ASCII file names in your Django application you get:

UnicodeEncodeError: 'ascii' codec can't encode characters in position N-N: ordinal not in range(128)

It seems that it is quite common problem so I decided to list all solutions I found and I tried in one place.

First check your currently set locale and available locales using locale and locale -a commands.

If you want to see what locales are used in your program modify your view…

import locale
import sys
loc_info = {'loc_loc': locale.getlocale(), 'loc_def': locale.getdefaultlocale(), 'sys_fenc': sys.getfilesystemencoding(), 'sys_denc': sys.getdefaultencoding()}
# this must be passed to template: 'loc_info': loc_info

… and template:

{{loc_info.loc_loc}} - {{loc_info.loc_def}} - {{loc_info.sys_fenc}} - {{loc_info.sys_denc}}

I will use pl_PL.UTF-8 locale in examples below.

Apache

Edit /etc/apache2/envvars by adding this two lines:

export LANG='pl_PL.UTF-8'
export LC_ALL='pl_PL.UTF-8'

Nginx

Add charset utf-8; line in http section in main Nginx config file (/etc/nginx/nginx.conf) or in section server in your virtual server config file.

Read more about Nginx HttpCharsetModule.

Gunicorn + Supervisor

Add line environment=LANG=pl_PL.UTF-8,LC_ALL=pl_PL.UTF-8 to [supervisord] section in main Supervisor config file (/etc/supervisor/supervisord.conf) or to [program:program_name] section in your program config file.

Read more about supervisord config section and subprocess environment on Supervisor configuration documentation.

uWSGI

Add env setting with value LC_ALL=pl_PL.UTF-8 to your uWSGI config file.

env LC_ALL=zh_CN.UTF-8 

害惨我了，之前配置成env=LC_ALL=zh_CN.UTF-8了

Bonus: storing files in directory mounted using SSHFS

If you mount a directory using SSHFS you also have to remember about source and target file names encoding. You should use two options of iconv module: from_code (default value: UTF-8) and to_code (default value: ISO-8859-2) . If you use /etc/fstab file to mount your directory you should add something like that: modules=iconv,from_code=UTF-8,to_code=UTF-8 to options section.

2 Comments

RSS feed for comments on this post.

27 March 2013 at 15:02
Simi says:

Sir, your guide have just saved me from the encoding hell. Much appreciation.

A clear and working solution for an encoding problem like this is really hard to search for on the internet. My guess is that most English speaking people won’t be encountering these problems much like we non-native do (I’m from Asia). My hats off to you my Polish sir.

A note to Ubuntu users using Upstart, just add these lines in /etc/init/yourservice.conf (assuming your encoding is en_US):
env LANG=en_US.UTF-8
env LC_ALL=en_us.UTF-8
before calling exec or script.

Reply
16 July 2014 at 15:27
irek says:

cheers for this, the supervisor.conf i would have never figured out myself
much appreciated