Python2和Python3比较分析

一直有看到网上有讨论Python2和Python3的比较，最近公司也在考虑是否在spark-python大数据开发环境中升级到python3。通过本篇博文记录Python2.7.13和Pthon3.5.3的各方面比较。

环境配置

这里继续使用我们在之前博文里配置的环境。
因为是比较Python2和Python3差异，所以单纯升级Python版本无法解决，我通过pyenv和virtualenv两个工具来实现隔离的测试环境。
参考文档：使用pyenv和virtualenv搭建python虚拟环境、使用 pyenv 可以在一个系统中安装多个python版本
配置的步骤如下：

  • 最开始是更新Tkinter，不然后续要重新再来一次，不要问我为什么知道…
  1. sudo yum install tkinter -y
  2. sudo yum install tk-devel tcl-devel -y
  • 更新pyenv依赖软件
  1. sudo yum install readline readline-devel readline-static -y
  2. yum install openssl openssl-devel openssl-static -y
  3. yum install sqlite-devel -y
  4. yum install bzip2-devel bzip2-libs -y
  • 下载安装pyenv，并下载python2.7.13和python3.5.3
  1. git clone https://github.com/yyuu/pyenv.git ~/.pyenv
  2. chgmod 777 -R ~/.pyenv
  3. echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bash_profile
  4. echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bash_profile
  5. echo 'eval "$(pyenv init -)"' >> ~/.bash_profile
  6. exec $SHELL
  7. source ~/.bash_profile 
  9. pyenv  install --list
  10. pyenv install -v 2.7.13
  11. pyenv install -v 3.5.3
  • 下载安装pyenv-virtualenv，并安装两个隔离环境
  1. git clone https://github.com/yyuu/pyenv-virtualenv.git ~/.pyenv/plugins/pyenv-virtualenv   
  2. echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bash_profile
  3. source ~/.bash_profile
  4. pyenv virtualenv 2.7.13 py2
  5. pyenv virtualenv 3.5.3 py3

好，到此基本搞定两个隔离的python环境，测试如下，我们可以发现当前的python环境从centos7默认的2.7.5切换到2.7.13再切换到3.5。

  1. [kejun@localhost ~]$ python -V
  2. Python 2.7.5
  3. [kejun@localhost ~]$ pyenv activate py2
  4. (py2) [kejun@localhost ~]$ python -V
  5. Python 2.7.13
  6. (py2) [kejun@localhost ~]$ pyenv deactivate
  7. [kejun@localhost ~]$ pyenv activate py3
  8. (py3) [kejun@localhost ~]$ python -V
  9. Python 3.5.

详细测试：

我们安装了常用的数据分析第三方工具包，并做了安装测试和样例测试，样例测试的脚本见最下。

分类工具名用途
数据收集scrapy网页采集，爬虫
数据收集scrapy-redis分布式爬虫
数据收集seleniumweb测试，仿真浏览器
数据处理beautifulsoup网页解释库，提供lxml的支持
数据处理lxmlxml解释库
数据处理xlrdexcel文件读取
数据处理xlwtexcel文件写入
数据处理xlutilsexcel文件简单格式修改
数据处理pywin32excel文件的读取写入及复杂格式定制
数据处理Python-docxWord文件的读取写入
数据分析numpy基于矩阵的数学计算库
数据分析pandas基于表格的统计分析库
数据分析scipy科学计算库，支持高阶抽象和复杂模型
数据分析statsmodels统计建模和计量经济学工具包
数据分析scikit-learn机器学习工具库
数据分析gensim自然语言处理工具库
数据分析jieba中文分词工具库
数据存储MySQL-pythonmysql的读写接口库
数据存储mysqlclientmysql的读写接口库
数据存储SQLAlchemy数据库的ORM封装
数据存储pymssqlsql server读写接口库
数据存储redisredis的读写接口
数据存储PyMongomongodb的读写接口
数据呈现matplotlib流行的数据可视化库
数据呈现seaborn美观的数据可是湖库，基于matplotlib
工具辅助jupyter基于web的python IDE，常用于数据分析
工具辅助chardet字符检查工具
工具辅助ConfigParser配置文件读写支持
工具辅助requestsHTTP库，用于网络访问
  1. # encoding=utf-8
  2. import sys
  3. import platform
  4. import traceback
  5. import gc
  6. import ctypes  
  9. STD_OUTPUT_HANDLE= -11  
  10. FOREGROUND_BLACK = 0x0  
  11. FOREGROUND_BLUE = 0x01 # text color contains blue.  
  12. FOREGROUND_GREEN= 0x02 # text color contains green.  
  13. FOREGROUND_RED = 0x04 # text color contains red.  
  14. FOREGROUND_INTENSITY = 0x08 # text color is intensified.  
  16. class WinPrint:
  17.     """
  18.     提供给Windows打印彩色字体使用
  19.     """
  21.     std_out_handle = ctypes.windll.kernel32.GetStdHandle(STD_OUTPUT_HANDLE)  
  23.     def set_cmd_color(self, color, handle=std_out_handle):  
  24.         bool = ctypes.windll.kernel32.SetConsoleTextAttribute(handle, color)  
  25.         return bool  
  27.     def reset_color(self):  
  28.         self.set_cmd_color(FOREGROUND_RED | FOREGROUND_GREEN | FOREGROUND_BLUE)  
  30.     def print_red_text(self, print_text):  
  31.         self.set_cmd_color(FOREGROUND_RED | FOREGROUND_INTENSITY)  
  32.         print (print_text)  
  33.         self.reset_color()  
  35.     def print_green_text(self, print_text):  
  36.         self.set_cmd_color(FOREGROUND_GREEN | FOREGROUND_INTENSITY)  
  37.         print (print_text)  
  38.         self.reset_color()  
  40. class UnixPrint:
  41.     """
  42.     提供给Centos打印彩色字体
  43.     """
  44.     def print_red_text(self, print_text): 
  45.         print('\033[1;31m%s\033[0m'%print_text)
  47.     def print_green_text(self, print_text): 
  48.         print('\033[1;32m%s\033[0m'%print_text)
  50. py_env = "Python2" if sys.version.find("2.7") > -1 else "Python3"
  51. sys_ver = "Windows" if platform.system().find("indows") > -1 else "Centos"
  52. my_print = WinPrint() if platform.system().find("indows") > -1 else UnixPrint()
  55. def check(sys_ver, py_env):
  56.     """
  57.     装饰器，统一输入输出
  58.     顺便测试带参数的装饰器，非必须带参数
  59.     """
  60.     def _check(func):
  61.         def __check():
  62.             try:
  63.                 func()
  64.                 my_print.print_green_text(
  65.                     "[%s,%s]: %s pass." % (sys_ver, py_env, func.__name__))
  66.             except:
  67.                 traceback.print_exc()
  68.                 my_print.print_red_text(
  69.                     "[%s,%s]: %s fail." % (sys_ver, py_env, func.__name__))
  70.         return __check
  71.     return _check
  74. def make_requirement(filepath, filename):
  75.     """
  76.     处理pip requirements文件
  77.     """
  78.     result = []
  79.     with open(filepath + "\\" + filename, "r") as f:
  80.         data = f.readlines()
  82.         for line in data:
  83.             if line.find("==") > -1:
  84.                 result.append(line.split("==")[0] + "\n")
  85.             else:
  86.                 result.append(line + "\n")
  87.     with open(filepath + "\\" + filename.split(".")[0] + "-clean.txt",
  88.               "w") as f1:
  89.         f1.writelines(result)
  92. @check(sys_ver, py_env)
  93. def test_scrapy():
  94.     from scrapy import signals
  95.     from selenium import webdriver
  96.     from scrapy.http import HtmlResponse
  97.     from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
  98.     from selenium.webdriver.common.keys import Keys
  99.     from selenium.common.exceptions import NoSuchElementException
  100.     from selenium.common.exceptions import TimeoutException
  101.     from selenium.webdriver.support import expected_conditions as EC
  102.     from selenium.webdriver.common.by import By
  103.     from selenium.webdriver.support.ui import WebDriverWait
  106. @check(sys_ver, py_env)
  107. def test_matplotlib():
  108.     import matplotlib.pyplot as plt
  109.     l = [1, 2, 3, 4, 5]
  110.     h = [20, 14, 38, 27, 9]
  111.     w = [0.1, 0.2, 0.3, 0.4, 0.5]
  112.     b = [1, 2, 3, 4, 5]
  113.     fig = plt.figure()
  114.     ax = fig.add_subplot(111)
  115.     rects = ax.bar(l, h, w, b)
  116.     # plt.show()
  119. @check(sys_ver, py_env)
  120. def test_beautifulSoup():
  121.     from bs4 import BeautifulSoup
  122.     html_str = "<html><meta/><head><title>Hello</title></head><body onload=crash()>Hi all<p></html>"
  123.     soup = BeautifulSoup(html_str, "lxml")
  124.     # print (soup.get_text())
  127. @check(sys_ver, py_env)
  128. def test_lxml():
  129.     from lxml import html
  130.     html_str = "<html><meta/><head><title>Hello</title></head><body onload=crash()>Hi all<p></html>"
  131.     html.fromstring(html_str)
  134. @check(sys_ver, py_env)
  135. def test_xls():
  136.     import xlrd
  137.     import xlwt
  138.     from xlutils.copy import copy
  139.     excel_book2 = xlwt.Workbook()
  140.     del excel_book2
  141.     excel_book1 = xlrd.open_workbook("1.xlsx")
  142.     del excel_book1
  143.     import docx
  144.     doc = docx.Document("1.docx")
  145.     # print (doc)
  146.     del doc
  147.     gc.collect()
  150. @check(sys_ver, py_env)
  151. def test_data_analysis():
  152.     import pandas as pd
  153.     import numpy as np
  154.     data_list = np.array([x for x in range(100)])
  155.     data_serial = pd.Series(data_list)
  156.     # print (data_serial)
  157.     from scipy import fft
  158.     b = fft(data_list)
  159.     # print (b)
  162. @check(sys_ver, py_env)
  163. def test_statsmodels():
  164.     import statsmodels.api as sm
  165.     data = sm.datasets.spector.load()
  166.     data.exog = sm.add_constant(data.exog, prepend=False)
  167.     # print data.exog
  170. @check(sys_ver, py_env)
  171. def test_sklearn():
  172.     from sklearn import datasets
  173.     iris = datasets.load_iris()
  174.     data = iris.data
  175.     # print(data.shape)
  178. @check(sys_ver, py_env)
  179. def test_gensim():
  180.     import warnings
  181.     warnings.filterwarnings(action='ignore', category=UserWarning, module='gensim')
  182.     from gensim import corpora
  183.     from collections import defaultdict
  184.     documents = ["Human machine interface for lab abc computer applications",
  185.                  "A survey of user opinion of computer system response time",
  186.                  "The EPS user interface management system",
  187.                  "System and human system engineering testing of EPS",
  188.                  "Relation of user perceived response time to error measurement",
  189.                  "The generation of random binary unordered trees",
  190.                  "The intersection graph of paths in trees",
  191.                  "Graph minors IV Widths of trees and well quasi ordering",
  192.                  "Graph minors A survey"]
  193.     stoplist = set('for a of the and to in'.split())
  194.     texts = [[word for word in document.lower().split() if word not in stoplist]
  195.              for document in documents]
  196.     frequency = defaultdict(int)
  197.     for text in texts:
  198.         for token in text:
  199.             frequency[token] += 1
  200.     texts = [[token for token in text if frequency[token] > 1]
  201.              for text in texts]
  202.     dictionary = corpora.Dictionary(texts)
  203.     dictionary.save('deerwester.dict')
  206. @check(sys_ver, py_env)
  207. def test_jieba():
  208.     import jieba
  209.     seg_list = jieba.cut("我来到了北京参观天安门。", cut_all=False)
  210.     # print("Default Mode: " + "/ ".join(seg_list))  # 精确模式
  213. @check(sys_ver, py_env)
  214. def test_mysql():
  215.     import MySQLdb as mysql
  216.     #测试pet_shop连接
  217.     db = mysql.connect(host="xx", user="yy", passwd="12345678", db="zz")
  218.     cur = db.cursor()
  219.     sql="select id from role;"
  220.     cur.execute(sql)
  221.     result = cur.fetchall()
  222.     db.close()
  223.     # print (result)
  226. @check(sys_ver, py_env)
  227. def test_SQLAlchemy():
  228.     from sqlalchemy import Column, String, create_engine,Integer
  229.     from sqlalchemy.orm import sessionmaker
  230.     from sqlalchemy.ext.declarative import declarative_base
  231.     engine = create_engine('mysql://xxx/yy',echo=False)
  232.     DBSession = sessionmaker(bind=engine)
  233.     Base = declarative_base()
  234.     class rule(Base):
  235.         __tablename__="role"
  236.         id=Column(Integer,primary_key=True,autoincrement=True)
  237.         role_name=Column(String(100))
  238.         role_desc=Column(String(255))
  239.     new_rule=rule(role_name="test_sqlalchemy",role_desc="forP2&P3")
  240.     session=DBSession()
  241.     session.add(new_rule)
  242.     session.commit()
  243.     session.close()
  246. @check(sys_ver, py_env)
  247. def test_redis():
  248.     import redis 
  249.     pool = redis.Redis(host='127.0.0.1', port=6379) 
  251. @check(sys_ver, py_env)
  252. def test_requests():
  253.     import requests
  254.     r=requests.get(url="http://www.cnblogs.com/kendrick/")
  255.     # print (r.status_code)
  258. @check(sys_ver, py_env)
  259. def test_PyMongo():
  260.     from pymongo import MongoClient
  261.     conn=MongoClient("localhost",27017)
  263. if __name__ == "__main__":
  264.     print ("[%s,%s]  start checking..." % (sys_ver, py_env))
  265.     test_scrapy()
  266.     test_beautifulSoup()
  267.     test_lxml()
  268.     test_matplotlib()
  269.     test_xls()
  270.     test_data_analysis()
  271.     test_sklearn()
  272.     test_mysql()
  273.     test_SQLAlchemy()
  274.     test_PyMongo()
  275.     test_gensim()
  276.     test_jieba()
  277.     test_redis()
  278.     test_requests()
  279.     test_statsmodels()
  280.     print ("[%s,%s]  finish checking." % (sys_ver, py_env))
posted @ 2017-03-23 16:08  柯君  阅读(587)  评论(0)  编辑  收藏  举报