小小的技巧，弯道超车！pythonic more 


import urllib2 
from multiprocessing.dummy import Pool as ThreadPool 

urls = [
'http://www.python.org', 
'http://www.python.org/about/',
'http://www.onlamp.com/pub/a/python/2003
]

pool = ThreadPool(4) 
results = pool.map(urllib2.urlopen, urls)
pool.close() 
pool.join()

对，你没有看错，只要一行代码就可以把普通的任务变成并行任务。不用手动管理线程，一切都由map自动完成。

这里演示的是多线程，如果要多进程的话只需把 from multiprocessing.dummy 改成 from multiprocessing 。

 

装饰器

装饰器（Decorators）是Python里一个很重要的概念，它能够使得Python代码更加简洁，用一句话概括：装饰器是修改其他函数功能的函数。PySnooper的调用主要依靠装饰器的方式，所以，了解装饰器的基本概念和使用方法更有助于理解PySnooper的使用。在这里，我先简单介绍一下装饰器的使用，如果精力有限，了解装饰器的调用方式即可。

对于Python，一切都是对象，一个函数可以作为一个对象在不同模块之间进行传递，举个例子，

def one(func):
    print("now you are in function one.")
    func()
​
    
def two():
    print("now you are in function two")
​
    
one(two)
​
# 输出
>>> now you are in function one.
>>> now you are in function two.

其实这就是装饰器的核心所在，它们封装一个函数，可以用这样或那样的方式来修改它。换一种方式表达上述调用，可以用@+函数名来装饰一个函数。

def one(func):
    print("now you are in function one.")
    def warp():
        func()
    return warp
​
​
@one
def two():
    print("now you are in function two.")
​
    
two()
​
# 输出
>>> now you are in function one.
>>> now you are in function two.

此外，在调用装饰器时还可以给函数传入参数：

def one(func):
    print("now you are in function one.")
    def warp(*args):
        func(*args)
    return warp
​
​
@one
def two(x, y):
    print("now you are in function two.")
    print("x value is %d, y value is %d" % (x, y))
​
    
two(5, 6)
​
# 输出
>>> now you are in function one.
>>> now you are in function two.
>>> x value is 5, y value is 6

另外，装饰器本身也可以接收参数，

def three(text):
    def one(func):
        print("now you are in function one.")
        def warp(*args):
            func(*args)
        return warp
    print("input params is {}".format(text))
    return one
​
​
@three(text=5)
def two(x, y):
    print("now you are in function two.")
    print("x value is %d, y value is %d" % (x, y))
​
    
two(5, 6)
​
# 输出
>>> input params is 5
>>> now you are in function one.
>>> now you are in function two.
>>> x value is 5, y value is 6

上面讲述的就是Python装饰器的一些常用方法。

 

 

 

 

 

Pysnooper

作者：Jackpop
链接：https://www.zhihu.com/question/27376156/answer/699936650
来源：知乎
著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。

import numpy as np
import pysnooper
​
​
@pysnooper.snoop()
def one(number):
    mat = []
    while number:
        mat.append(np.random.normal(0, 1))
        number -= 1
    return mat
​
​
one(3)
​
# 输出
​
Starting var:.. number = 3
22:17:10.634566 call         6 def one(number):
22:17:10.634566 line         7     mat = []
New var:....... mat = []
22:17:10.634566 line         8     while number:
22:17:10.634566 line         9         mat.append(np.random.normal(0, 1))
Modified var:.. mat = [-0.4142847169210746]
22:17:10.634566 line        10         number -= 1
Modified var:.. number = 2
22:17:10.634566 line         8     while number:
22:17:10.634566 line         9         mat.append(np.random.normal(0, 1))
Modified var:.. mat = [-0.4142847169210746, -0.479901983375219]
22:17:10.634566 line        10         number -= 1
Modified var:.. number = 1
22:17:10.634566 line         8     while number:
22:17:10.634566 line         9         mat.append(np.random.normal(0, 1))
Modified var:.. mat = [-0.4142847169210746, -0.479901983375219, 1.0491540468063252]
22:17:10.634566 line        10         number -= 1
Modified var:.. number = 0
22:17:10.634566 line         8     while number:
22:17:10.634566 line        11     return mat
22:17:10.634566 return      11     return mat
Return value:.. [-0.4142847169210746, -0.479901983375219, 1.0491540468063252]
小李子
作者：Jackpop
链接：https://www.zhihu.com/question/27376156/answer/699936650
来源：知乎
著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。

@pysnooper.snoop(prefix="funcTwo ")
def two(x, y):
    z = x + y
    return z
​
​
@pysnooper.snoop(prefix="funcOne ")
def one(number):
    k = 0
    while number:
        k = two(k, number)
        number -= 1
    return number
​
​
one(3)
​
# 输出
funcOne Starting var:.. number = 3
funcOne 22:28:14.259212 call        12 def one(number):
funcOne 22:28:14.260211 line        13     k = 0
funcOne New var:....... k = 0
funcOne 22:28:14.260211 line        14     while number:
funcOne 22:28:14.260211 line        15         k = two(k, number)
funcTwo     Starting var:.. x = 0
funcTwo     Starting var:.. y = 3
funcTwo     22:28:14.260211 call         6 def two(x, y):
funcTwo     22:28:14.260211 line         7     z = x + y
funcTwo     New var:....... z = 3
funcTwo     22:28:14.260211 line         8     return z
funcTwo     22:28:14.260211 return       8     return z
funcTwo     Return value:.. 3
funcOne Modified var:.. k = 3
funcOne 22:28:14.260211 line        16         number -= 1
funcOne Modified var:.. number = 2
funcOne 22:28:14.260211 line        14     while number:
funcOne 22:28:14.260211 line        15         k = two(k, number)
funcTwo     Starting var:.. x = 3
funcTwo     Starting var:.. y = 2
funcTwo     22:28:14.260211 call         6 def two(x, y):
funcTwo     22:28:14.260211 line         7     z = x + y
funcTwo     New var:....... z = 5
funcTwo     22:28:14.260211 line         8     return z
funcTwo     22:28:14.260211 return       8     return z
funcTwo     Return value:.. 5
funcOne Modified var:.. k = 5
funcOne 22:28:14.260211 line        16         number -= 1
funcOne Modified var:.. number = 1
funcOne 22:28:14.260211 line        14     while number:
funcOne 22:28:14.260211 line        15         k = two(k, number)
funcTwo     Starting var:.. x = 5
funcTwo     Starting var:.. y = 1
funcTwo     22:28:14.260211 call         6 def two(x, y):
funcTwo     22:28:14.260211 line         7     z = x + y
funcTwo     New var:....... z = 6
funcTwo     22:28:14.260211 line         8     return z
funcTwo     22:28:14.260211 return       8     return z
funcTwo     Return value:.. 6
funcOne Modified var:.. k = 6
funcOne 22:28:14.260211 line        16         number -= 1
funcOne Modified var:.. number = 0
funcOne 22:28:14.260211 line        14     while number:
funcOne 22:28:14.260211 line        17     return number
funcOne 22:28:14.260211 return      17     return number
funcOne Return value:.. 0
除了缩进之外，PySnooper还提供了参数prefix给debug信息添加前缀的方式便于识别，
class Test():
    t = 10
​
​
test = Test()
​
​
@pysnooper.snoop(watch=("test.t", "x"))
​
# 输出
Starting var:.. number = 3
Starting var:.. test.t = 10
Starting var:.. x = 10
参数watch可以用于查看一些非局部变量
#### watch_explode ####
d = {
    "one": 1,
    "two": 1
}
​
​
@pysnooper.snoop(watch_explode="d")
​
# 输出
Starting var:.. number = 3
Starting var:.. d = {'one': 1, 'two': 1}
Starting var:.. d['one'] = 1
Starting var:.. d['two'] = 1
​
#### watch ####
d = {
    "one": 1,
    "two": 1
}
​
​
@pysnooper.snoop(watch="d")
​
# 输出
Starting var:.. d = {'one': 1, 'two': 1}


# 以看出watch_explode能够展开字典的属性值。

# 另外还有参数depth显示函数中调用函数的snoop行，默认值为1，参数值需要大于或等于1。
参数watch_explode可以展开字典或者列表显示它的所有属性值，对比一下它和watch的区别，

 

PySnooper输出信息主要包括以下几个部分：

  • 局部变量值
  • 代码片段
  • 局部变量所在行号
  • 返回结果

上述结果输出到控制台，还可以把日志输出到文件，

@pysnooper.snoop("debug.log")




各种时间形式转换

 

 

 

 

 

or else值得说下。不break的话就执行else
for i in range(10):
    if i == 10:
        break
    print(i)
else:
    print('10不在里面！')

相当于：
flag = False
for i in range(10):
    if i == 10:
        flag = True
        break
    print(i)
if not flag:
    print('10不在里面！')


 
 


cached_property

它的作用是将一个方法的计算结果缓存到对象的 __dict__ 当中，熟悉 Flask 的人对这个应该不陌生，Django 应该也有类似的实现。这是 werkzeug 中的源码实现：

class cached_property(property):
​
  def __init__(self, func, name=None, doc=None):
        self.__name__ = name or func.__name__
        self.__module__ = func.__module__
        self.__doc__ = doc or func.__doc__
        self.func = func
​
    def __set__(self, obj, value):
        obj.__dict__[self.__name__] = value
​
    def __get__(self, obj, type=None):
        if obj is None:
            return self
        value = obj.__dict__.get(self.__name__, _missing)
        if value is _missing:
            value = self.func(obj)
            obj.__dict__[self.__name__] = value
        return value

 

Flask 自己实现了一个线程安全的版本，locked_cached_property：

class locked_cached_property(object):
      def __init__(self, func, name=None, doc=None):
            self.__name__ = name or func.__name__
            self.__module__ = func.__module__
            self.__doc__ = doc or func.__doc__
            self.func = func
            self.lock = RLock()
​
    def __get__(self, obj, type=None):
        if obj is None:
            return self
        with self.lock:
            value = obj.__dict__.get(self.__name__, _missing)
            if value is _missing:
                value = self.func(obj)
                obj.__dict__[self.__name__] = value
            return value

 

 

 

 

 

更多可以前往https://www.zhihu.com/question/27376156?sort=created

 

posted @ 2020-03-02 11:21  星涅爱别离  阅读(476)  评论(0)  编辑  收藏  举报