好记性不如烂笔头
带关键字的格式化
>>>
>>> print "Hello %(name)s !" % {'name':'James'}
Hello James !
>>>
>>> print "Hello {name} !".format(name="James")
Hello James !
>>>
使用dict.__missing__() 避免出现KeyError
If a subclass of dict defines a method __missing__() and key is not present, the d[key] operation calls that method with the key(key as argument). The d[key] operation then returns or raises whatever is returned or raised by the __missing__(key) call.
>>>
>>> class Counter(dict):
... def __missing__(self, key):
... return 0
...
>>> c = Counter()
>>> print c['num']
0
>>> c['num'] += 1
>>> print c['num']
1
>>>
>>> c
{'num': 1}
>>>
__getattr__ 调用默认方法
>>>
>>> class A(object):
... def __init__(self,num):
... self.num = num
... print 'init...'
... def mydefault(self, *args, **kwargs):
... print 'default func...'
... print args
... print kwargs
... def __getattr__(self,name):
... print 'No %(name)s found, goto default...' % {'name':name}
... return self.mydefault
...
>>> a1 = A(9)
init...
>>> a1.fn1()
No fn1 found, goto default...
default func...
()
{}
>>> a1.fn2(1,2)
No fn2 found, goto default...
default func...
(1, 2)
{}
>>> a1.fn3(name='standby',age=18)
No fn3 found, goto default...
default func...
()
{'age': 18, 'name': 'standby'}
>>>
>>>
obj.xxx = aaa 触发类的 __setattr__ obj.xxx 触发类的 __getattr__ obj['xxx'] = 'vvv' 触发类的 __setitem__ obj['xxx'] 触发类的 __getitem__ with app1.app_context(): 触发 __enter__ __exit__
__new__ 和 __init__ 的执行顺序
>>> >>> class B(object): ... def fn(self): ... print 'B fn' ... def __init__(self): ... print "B INIT" ... >>> class A(object): ... def fn(self): ... print 'A fn' ... def __new__(cls,a): ... print "NEW", a ... if a>10: ... return super(A, cls).__new__(cls) ... return B() ... def __init__(self,a): ... print "INIT", a ... >>> >>> a1 = A(5) NEW 5 B INIT >>> a1.fn() B fn >>> a2=A(20) NEW 20 INIT 20 >>> a2.fn() A fn >>>
类继承之 __class__
>>> >>> class A(object): ... def show(self): ... print 'base show' ... >>> class B(A): ... def show(self): ... print 'derived show' ... >>> obj = B() >>> obj.show() derived show >>> >>> obj.__class__ <class '__main__.B'> >>> >>> obj.__class__ = A >>> obj.__class__ <class '__main__.A'> >>> obj.show() base show >>>
对象方法 __call__
>>>
>>> class A(object):
... def obj_func(self, *args, **kwargs):
... print args
... print kwargs
... def __call__(self, *args, **kwargs):
... print 'Object method ...'
... return self.obj_func(*args, **kwargs)
...
>>> a1=A()
>>> a1(9,name='standby',city='beijing')
Object method ...
(9,)
{'city': 'beijing', 'name': 'standby'}
>>>
补充：
>>> >>> class test(object): ... def __init__(self, value): ... self.x = value ... def __call__(self, value): ... return self.x * value ... >>> a = test(4) >>> print a(5) 20 >>>
补充：
- 什么后面可以加括号？（只有4种表现形式） - 函数 执行函数 - 类 执行类的__init__方法 - 方法 obj.func - 对象 前提：类里有 __call__ 方法 obj() 直接执行类的 __call__方法
关于类的继承
>>> >>> class Parent(object): ... x = 1 ... >>> class Child1(Parent): ... pass ... >>> class Child2(Parent): ... pass ... >>> Child1.x = 2 >>> Parent.x = 3 >>> print Parent.x, Child1.x, Child2.x 3 2 3 >>>
类属性和对象属性
类属性 >>> >>> class Student: ... score = [] ... >>> stu1 = Student() >>> stu2 = Student() >>> stu1.score.append(99) >>> stu1.score.append(96) >>> stu2.score.append(98) >>> >>> >>> stu2.score [99, 96, 98] >>> >>> 对象属性 >>> >>> class Student: ... def __init__(self):; ... self.lst = [] ... >>> stu1 = Student() >>> stu2 = Student() >>> >>> >>> stu1.lst.append(1) >>> stu1.lst.append(2) >>> stu2.lst.append(9) >>> >>> stu1.lst [1, 2] >>> >>> stu2.lst [9] >>>
一行代码实现列表偶数位加3后求和
>>> a = [1,2,3,4,5,6] >>> [item+3 for item in a if a.index(item)%2==0] [4, 6, 8] >>> result = sum([item+3 for item in a if a.index(item)%2==0]) >>> result 18 >>>
字符串连接
>>> >>> name = 'hi ' 'standby' ' !' >>> name 'hi standby !' >>>
Python解释器中的 '_'
_ 即Python解释器上一次返回的值
>>> >>> range(5) [0, 1, 2, 3, 4] >>> _ [0, 1, 2, 3, 4] >>>
嵌套列表推导式
>>> >>> [(i, j) for i in range(3) for j in range(i)] [(1, 0), (2, 0), (2, 1)] >>>
Python3 中的unpack
>>> >>> first, second, *rest, last = range(10) >>> first 0 >>> second 1 >>> last 9 >>> rest [2, 3, 4, 5, 6, 7, 8] >>>
关于__setattr__ __getattr__ __getitem__ __setitem__ 参考：http://www.cnblogs.com/standby/p/7045718.html
Python把常用数字缓存在内存里 *****
>>> >>> a = 1 >>> b = 1 >>> a is b True >>> >>> >>> a = 256 >>> b = 256 >>> a is b True >>> >>> a = 257 >>> b = 257 >>> a is b False >>> >>> a = 300 >>> b = 300 >>> a is b False >>>
注意：在[-5,256]之间的数字用在内存中的id号是相同的；
Python为了提高运行效率而将这些常用数字缓存到内存里了，所以他们的id号是相同的；
另外，对a,b,c,....等的赋值也只是一种引用而已：
>>> >>> id(9) 10183288 >>> num = 9 >>> id(num) 10183288 >>>
Python对于短字符串会使用同一个空间，但是对于长字符串会重新开辟空间
>>> >>> a = 'I love PythonSomething!' >>> b = 'I love PythonSomething!' >>> c = [1, 2, 3] >>> d = [1, 2, 3] >>> >>> a is b False >>> c is d False >>> >>> id(a) 139848068316272 >>> id(b) 139848068316336 >>> id(c) 139848068310152 >>> id(d) 139848068309936 >>>
字符串 * 操作
>>>
>>> def func(a):
... a = a + '2'
... a = a*2
... return a
...
>>>
>>> func("hello")
'hello2hello2'
>>>
Python浮点数比较
>>> >>> 0.1 0.10000000000000001 >>> 0.2 0.20000000000000001 >>> 0.1 + 0.2 0.30000000000000004 >>> >>> 0.3 0.29999999999999999 >>> >>> 0.1 + 0.2 == 0.3 False >>>
Python里的 '~' 取反
>>> >>> 5 5 >>> ~5 -6 >>> ~~5 5 >>> ~~~5 -6 >>> ~~~~5 5 >>>
~5 即对5取反，得到的是 -6 ， 为什么？
参考：https://www.cnblogs.com/piperck/p/5829867.html 和 http://blog.csdn.net/u011080472/article/details/51280919
- 原码就是符号位加上真值的绝对值；
- 反码的表示方法是：正数的反码就是其本身；负数的反码是在其原码的基础上, 符号位不变，其余各个位取反；
- 补码的表示方式是：正数的补码就是其本身；负数的补码是在其原码的基础上, 符号位不变, 其余各位取反, 最后+1 (即在反码的基础上+1)；
| 真值 | 原码 | 反码 | 补码 | |
| 5 | +000 0101 | 0000 0101 | 0000 0101 | 0000 0101 |
| -5 | -000 0101 | 1000 0101 | 1111 1010 | 1111 1011 |
对5取反即对 0000 0101 取反， 得到 1111 1010，那这个值的十进制是多少呢？
因为 负数在计算机中是以补码形式表示的， 所以实际上就是求哪个值的补码是 1111 1010，
按照上面的规则反向计算：
1111 1010 减1 得到其反码表示：1111 1001
在保证符号位不变，其余各位取反：1000 0110 就是该值的原码，对应真值就是 -000 0110 ，对应十进制就是 -6 。
那么对 -6 取反，得到的是多少呢？
对-6取反即对 -6 的补码取反，就是对1111 1010取反，得到 0000 0101，很明显是一个正数。
而正数原码==正数反码==正数补码，所以该值的原码就是 0000 0101，真值就是 +000 0101，对应十进制就是 5。
bool()
>>>
>>> bool('True')
True
>>> bool('False')
True
>>> bool('')
False
>>> bool()
False
>>>
>>>
>>> bool(1)
True
>>> bool(0)
False
>>>
等价于
>>> >>> True==False==False False >>> >>> True==False and False==False False >>>
>>> >>> 1 in [0,1] True >>> 1 in [0,1] == True False >>> >>> (1 in [0,1]) == True True >>> >>> 1 in ([0,1] == True) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: argument of type 'bool' is not iterable >>> >>> >>> >>> (1 in [0,1]) and ([0,1] == True) False >>>
Note that comparisons, membership tests, and identity tests,
all have the same precedence and have a left-to-right chaining feature as described in the Comparisons section.
参考：https://stackoverflow.com/questions/31354429/why-is-true-is-false-false-false-in-python
while 结合 break
>>> >>> i = 0 >>> while i < 5: ... print(i) ... i += 1 ... if i == 3: ... break ... else: ... print(0) ... 0 1 2 >>>
set给list去重
>>> >>> nums = set([1,1,2,3,3,3,4]) >>> >>> nums set([1, 2, 3, 4]) >>> >>> type(nums) <type 'set'> >>> >>> len(nums) 4 >>> >>> >>> li = list(nums) >>> li [1, 2, 3, 4] >>> >>> type(li) <type 'list'> >>>
函数是第一类对象（First-Class Object）
在 Python 中万物皆为对象，函数也不例外，
函数作为对象可以赋值给一个变量、可以作为元素添加到集合对象中、
可作为参数值传递给其它函数，还可以当做函数的返回值，这些特性就是第一类对象所特有的。
函数可以嵌套，函数中里面嵌套的函数不能在函数外面访问，只能是在函数内部使用：
def get_length(text):
def clean(t):
return t[1:]
res = clean(text)
return len(res)
print(get_length('standby'))
Python里的高阶函数
函数接受一个或多个函数作为输入或者函数输出（返回）的值是函数时，我们称这样的函数为高阶函数。
Python内置函数中，典型的高阶函数是 map 函数，map 接受一个函数和一个迭代对象作为参数，
调用 map 时，依次迭代把迭代对象的元素作为参数调用该函数。
def foo(text):
return len(text)
li = map(foo, ["the","zen","of","python"])
print(li) # <map object at 0x0000000001119FD0>
li = list(li)
print(li) # [3, 3, 2, 6]
lambda应用场景
- 函数式编程
有一个列表： list1 = [3,5,-4,-1,0,-2,-6]，需要按照每个元素的绝对值升序排序，如何做？
# 使用lambda的方式 >>> >>> list1 [3, 5, -4, -1, 0, -2, -6] >>> >>> sorted(list1, key=lambda i : abs(i)) [0, -1, -2, 3, -4, 5, -6] >>> # 不使用lambda的方式 >>> >>> def foo(x): ... return abs(x) ... >>> sorted(list1, key=foo) [0, -1, -2, 3, -4, 5, -6] >>>
如何把一个字典按照value进行排序？
>>>
>>> dic = {'a': 9, 'c': 3, 'b': 1, 'd': 7, 'f': 12}
>>> dic
{'a': 9, 'f': 12, 'c': 3, 'd': 7, 'b': 1}
>>>
>>> from collections import Iterable
>>> isinstance(dic.items(),Iterable)
True
>>>
>>> dic.items()
dict_items([('a', 9), ('f', 12), ('c', 3), ('d', 7), ('b', 1)])
>>>
>>> sorted(dic.items(), key=lambda x:x[1])
[('b', 1), ('c', 3), ('d', 7), ('a', 9), ('f', 12)]
>>>
2019-08-20补充示例：列表排序
In [57]: ver
Out[57]:
['10.3',
'15.100',
'10.50',
'2.3',
'10.30',
'2.1',
'10.0',
'10.6',
'10.20',
'4.0',
'3.0',
'10.2',
'10.8',
'10.1',
'15.0',
'10.10',
'10.9']
In [58]: ver.sort(key=lambda x:tuple(int(v) for v in x.split(".")), reverse=True)
In [59]: ver
Out[59]:
['15.100',
'15.0',
'10.50',
'10.30',
'10.20',
'10.10',
'10.9',
'10.8',
'10.6',
'10.3',
'10.2',
'10.1',
'10.0',
'4.0',
'3.0',
'2.3',
'2.1']
In [60]:
- 闭包
# 不用lambda的方式 >>> >>> def my_add(n): ... def wrapper(x): ... return x+n ... return wrapper ... >>> add_3 = my_add(3) >>> add_3(7) 10 >>> # 使用lambda的方式 >>> >>> def my_add(n): ... return lambda x:x+n ... >>> add_3 = my_add(3) >>> add_3(7) 10 >>>
方法和函数的区别
#!/usr/bin/python3
from types import MethodType,FunctionType
class Foo(object):
def __init__(self):
pass
def func(self):
print('func...')
obj = Foo()
print(obj.func) # 自动传递 self
# <bound method Foo.func of <__main__.Foo object at 0x7f86121505f8>>
print(Foo.func)
# <function Foo.func at 0x7f861214e488>
print(isinstance(obj.func,MethodType)) # True
print(isinstance(obj.func,FunctionType)) # False
print(isinstance(Foo.func,MethodType)) # False
print(isinstance(Foo.func,FunctionType)) # True
时间戳转换成年月日时分秒
>>> from datetime import datetime
>>> ts=1531123200
>>> date_str = datetime.fromtimestamp(ts).strftime('%Y-%m-%d %H:%M:%S')
>>> date_str
'2018-07-09 16:00:00'
>>>
In [11]: import time
In [12]: ts = int(time.time())
In [13]: ts
Out[13]: 1559549982
In [14]: time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(ts))
Out[14]: '2019-06-03 16:19:42'
In [15]:
年月日时分秒转换成时间戳
>>> date_str '2018-07-09 16:00:00' >>> >>> date_struct=time.strptime(date_str,'%Y-%m-%d %H:%M:%S') >>> date_struct time.struct_time(tm_year=2018, tm_mon=7, tm_mday=9, tm_hour=16, tm_min=0, tm_sec=0, tm_wday=0, tm_yday=190, tm_isdst=-1) >>> >>> int(time.mktime(date_struct)) 1531123200 >>>
In [16]: d Out[16]: '2019-06-03 16:19:42' In [17]: time.mktime(time.strptime(d, "%Y-%m-%d %H:%M:%S")) Out[17]: 1559549982.0 In [18]:
日期是时间戳相互转换
def date2ts(date_str, layout="%Y-%m-%d %H:%M:%S"):
date_struct=time.strptime(date_str, layout)
return int(time.mktime(date_struct))
def ts2date(ts, layout="%Y-%m-%d %H:%M:%S"):
ts = int(ts)
return datetime.datetime.fromtimestamp(ts).strftime(layout)
dis = pd.date_range(start='2023-06-01 00:00:00', end='2023-07-01 00:00:00', freq="5min", closed="left").to_list()
tss = [ int(di.tz_localize("Asia/Shanghai").timestamp()) for di in dis ]
获取当前月初和月末时间戳
>>> import time >>> import datetime >>> >>> start_st = datetime.datetime.now() >>> start_st datetime.datetime(2018, 7, 17, 16, 45, 39, 95228) >>> startts = int(time.mktime((start_st.year, start_st.month-1, 1, 0, 0, 0, 0, 0, -1))) >>> startts # 当前月初时间戳 1527782400 >>> >>> stopts = int(time.mktime((start_st.year, start_st.month, 1, 0, 0, 0, 0, 0, -1))) - 1 >>> stopts 1530374399 # 当前月末时间戳 >>>
IP地址转换成数字，从而进行比较，适用于地址库
# 方法一：手动计算
In [62]: ip
Out[62]: '10.3.81.150'
In [63]: ip.split('.')[::-1]
Out[63]: ['150', '81', '3', '10']
In [64]: [ '{}-{}'.format(idx,num) for idx,num in enumerate(ip.split('.')[::-1]) ]
Out[64]: ['0-150', '1-81', '2-3', '3-10']
In [65]: [256**idx*int(num) for idx,num in enumerate(ip.split('.')[::-1])]
Out[65]: [150, 20736, 196608, 167772160]
In [66]: sum([256**idx*int(num) for idx,num in enumerate(ip.split('.')[::-1])])
Out[66]: 167989654
In [67]:
# 方法二：使用C扩展库来计算
In [71]: import socket,struct
In [72]: socket.inet_aton(ip)
Out[72]: b'\n\x03Q\x96'
In [73]: struct.unpack("!I", socket.inet_aton(ip)) # !表示使用网络字节顺序解析, 后面的I表示unsigned int, 对应Python里的integer or long
Out[73]: (167989654,)
In [74]: struct.unpack("!I", socket.inet_aton(ip))[0]
Out[74]: 167989654
In [75]: socket.inet_ntoa(struct.pack("!I", 167989654))
Out[75]: '10.3.81.150'
In [76]:
使用yield逐行读取多个文件并合并
#!/usr/bin/python2.7
def get_line_by_yield():
with open('total.txt','r') as rf_total, open('extra.txt','r') as rf_extra:
for line in rf_total:
extra = rf_extra.readline()
lst = line.strip().split() + extra.strip().split()
yield lst
with open('new_total.txt','w') as wf:
for lst in get_line_by_yield():
wf.write('%s\n' % ''.join(map(lambda i: str(i).rjust(20), lst)))
逐行读取
def read_line(path):
with open(path,'r') as rf:
for line in rf:
yield line
检查进程是否 running
In [16]: import signal In [17]: from os import kill In [18]: kill(17335, 0) # 17335 是进程ID，第二个参数传0/signal.SIG_DFL 返回值若是None则表示正在运行 In [19]: kill(17335, 15) # 给进程传递15/signal.SIGTERM，即终止该进程 In [20]: kill(17335, 0) # 再次检查发现该进程已经不再running，则raise一个OSError --------------------------------------------------------------------------- OSError Traceback (most recent call last) <ipython-input-20-cbb7c9624124> in <module>() ----> 1 kill(17335, 0) OSError: [Errno 3] No such process In [21]:
1 In [12]: import signal 2 3 In [13]: signal.SIGKILL 4 Out[13]: 9 5 6 In [14]: signal.SIGTERM 7 Out[14]: 15 8 9 In [15]: signal.__dict__.items() 10 Out[15]: 11 [('SIGHUP', 1), 12 ('SIG_DFL', 0), 13 ('SIGSYS', 31), 14 ('SIGQUIT', 3), 15 ('SIGUSR1', 10), 16 ('SIGFPE', 8), 17 ('SIGPWR', 30), 18 ('SIGTSTP', 20), 19 ('ITIMER_REAL', 0L), 20 ('SIGCHLD', 17), 21 ('SIGCONT', 18), 22 ('SIGIOT', 6), 23 ('SIGBUS', 7), 24 ('SIGXCPU', 24), 25 ('SIGPROF', 27), 26 ('SIGCLD', 17), 27 ('SIGUSR2', 12), 28 ('default_int_handler', <function signal.default_int_handler>), 29 ('pause', <function signal.pause>), 30 ('SIGKILL', 9), 31 ('NSIG', 65), 32 ('SIGTRAP', 5), 33 ('SIGINT', 2), 34 ('SIGIO', 29), 35 ('__package__', None), 36 ('getsignal', <function signal.getsignal>), 37 ('SIGILL', 4), 38 ('SIGPOLL', 29), 39 ('SIGABRT', 6), 40 ('SIGALRM', 14), 41 ('__doc__', 42 'This module provides mechanisms to use signal handlers in Python.\n\nFunctions:\n\nalarm() -- cause SIGALRM after a specified time [Unix only]\nsetitimer() -- cause a signal (described below) after a specified\n float time and the timer may restart then [Unix only]\ngetitimer() -- get current value of timer [Unix only]\nsignal() -- set the action for a given signal\ngetsignal() -- get the signal action for a given signal\npause() -- wait until a signal arrives [Unix only]\ndefault_int_handler() -- default SIGINT handler\n\nsignal constants:\nSIG_DFL -- used to refer to the system default handler\nSIG_IGN -- used to ignore the signal\nNSIG -- number of defined signals\nSIGINT, SIGTERM, etc. -- signal numbers\n\nitimer constants:\nITIMER_REAL -- decrements in real time, and delivers SIGALRM upon\n expiration\nITIMER_VIRTUAL -- decrements only when the process is executing,\n and delivers SIGVTALRM upon expiration\nITIMER_PROF -- decrements both when the process is executing and\n when the system is executing on behalf of the process.\n Coupled with ITIMER_VIRTUAL, this timer is usually\n used to profile the time spent by the application\n in user and kernel space. SIGPROF is delivered upon\n expiration.\n\n\n*** IMPORTANT NOTICE ***\nA signal handler function is called with two arguments:\nthe first is the signal number, the second is the interrupted stack frame.'), 43 ('SIG_IGN', 1), 44 ('getitimer', <function signal.getitimer>), 45 ('SIGURG', 23), 46 ('SIGPIPE', 13), 47 ('SIGWINCH', 28), 48 ('__name__', 'signal'), 49 ('SIGTERM', 15), 50 ('SIGVTALRM', 26), 51 ('ITIMER_PROF', 2L), 52 ('SIGRTMIN', 34), 53 ('SIGRTMAX', 64), 54 ('ITIMER_VIRTUAL', 1L), 55 ('set_wakeup_fd', <function signal.set_wakeup_fd>), 56 ('setitimer', <function signal.setitimer>), 57 ('signal', <function signal.signal>), 58 ('SIGSEGV', 11), 59 ('siginterrupt', <function signal.siginterrupt>), 60 ('SIGXFSZ', 25), 61 ('SIGTTIN', 21), 62 ('SIGSTOP', 19), 63 ('ItimerError', signal.ItimerError), 64 ('SIGTTOU', 22), 65 ('alarm', <function signal.alarm>)] 66 67 In [16]: dict((k, v) for v, k in reversed(sorted(signal.__dict__.items())) 68 ...: if v.startswith('SIG') and not v.startswith('SIG_')) 69 Out[16]: 70 {1: 'SIGHUP', 71 2: 'SIGINT', 72 3: 'SIGQUIT', 73 4: 'SIGILL', 74 5: 'SIGTRAP', 75 6: 'SIGABRT', 76 7: 'SIGBUS', 77 8: 'SIGFPE', 78 9: 'SIGKILL', 79 10: 'SIGUSR1', 80 11: 'SIGSEGV', 81 12: 'SIGUSR2', 82 13: 'SIGPIPE', 83 14: 'SIGALRM', 84 15: 'SIGTERM', 85 17: 'SIGCHLD', 86 18: 'SIGCONT', 87 19: 'SIGSTOP', 88 20: 'SIGTSTP', 89 21: 'SIGTTIN', 90 22: 'SIGTTOU', 91 23: 'SIGURG', 92 24: 'SIGXCPU', 93 25: 'SIGXFSZ', 94 26: 'SIGVTALRM', 95 27: 'SIGPROF', 96 28: 'SIGWINCH', 97 29: 'SIGIO', 98 30: 'SIGPWR', 99 31: 'SIGSYS', 100 34: 'SIGRTMIN', 101 64: 'SIGRTMAX'} 102 103 In [17]:
1 def check_if_process_is_alive(self): 2 try: 3 kill(self.current_pid, 0) 4 kill(self.parent_pid, 0) 5 except: 6 # do something... 7 exit(0)
多指标排序问题
In [26]: lst = [('john', 'A', 15), ('jane', 'B', 12), ('dave', 'B', 10)]
In [27]: import operator
In [28]: sorted(lst, key=operator.itemgetter(1))
Out[28]: [('john', 'A', 15), ('jane', 'B', 12), ('dave', 'B', 10)]
In [29]: sorted(lst, key=operator.itemgetter(1,2)) # 先根据第二个域排序，然后再根据第三个域排序
Out[29]: [('john', 'A', 15), ('dave', 'B', 10), ('jane', 'B', 12)]
In [30]:
两个纯数字列表元素个数相等，按序相加求和，得到一个新的列表
length = len(lst1) lst = reduce(lambda x,y:[x[i]+y[i] for i in range(length)], [lst1,lst2], [0]*length)
或者直接使用numpy.array
补充reduce+lambda合并多个列表
In [15]: lst = [[1,2,3],['a','c'],['hello','world'],[2,2,2,111]] In [16]: reduce(lambda x,y: x+y, lst) Out[16]: [1, 2, 3, 'a', 'c', 'hello', 'world', 2, 2, 2, 111] In [17]:
扩展示例1：
lst= [[{u'timestamp': 1545214320, u'value': 222842128},
{u'timestamp': 1545214380, u'value': 224080288},
{u'timestamp': 1545214440, u'value': 253812496},
{u'timestamp': 1545214500, u'value': 295170240},
{u'timestamp': 1545214560, u'value': 221196224},
{u'timestamp': 1545214620, u'value': 252992096}],
[{u'timestamp': 1545214320, u'value': 228121600},
{u'timestamp': 1545214380, u'value': 225682656},
{u'timestamp': 1545214440, u'value': 256428064},
{u'timestamp': 1545214500, u'value': 292691424},
{u'timestamp': 1545214560, u'value': 241462336},
{u'timestamp': 1545214620, u'value': 250864528}],
[{u'timestamp': 1545214320, u'value': 232334304},
{u'timestamp': 1545214380, u'value': 230452032},
{u'timestamp': 1545214440, u'value': 246094880},
{u'timestamp': 1545214500, u'value': 260281088},
{u'timestamp': 1545214560, u'value': 233277120},
{u'timestamp': 1545214620, u'value': 258726192}]]
# 要求：把上述列表合并
# 方法一：使用Python内置函数
In [83]: reduce(lambda x,y:[ { 'timestamp':x[i]['timestamp'], 'value':x[i]['value']+y[i]['value'] } for i in range(6) ], a)
Out[83]:
[{'timestamp': 1545214320, 'value': 683298032},
{'timestamp': 1545214380, 'value': 680214976},
{'timestamp': 1545214440, 'value': 756335440},
{'timestamp': 1545214500, 'value': 848142752},
{'timestamp': 1545214560, 'value': 695935680},
{'timestamp': 1545214620, 'value': 762582816}]
In [84]:
# 方法二：笨办法
In [87]: b = a.pop(0)
In [88]:
In [88]: for i in a:
...: for idx in range(len(i)):
...: b[idx]['value'] += i[idx]['value']
...:
In [89]: b
Out[89]:
[{u'timestamp': 1545214320, u'value': 683298032},
{u'timestamp': 1545214380, u'value': 680214976},
{u'timestamp': 1545214440, u'value': 756335440},
{u'timestamp': 1545214500, u'value': 848142752},
{u'timestamp': 1545214560, u'value': 695935680},
{u'timestamp': 1545214620, u'value': 762582816}]
In [90]:
扩展示例2：
In [48]: a
Out[48]:
[{'A078102C949EC2AB': [1, 2, 3, 4]},
{'457D37015E77700E': [2, 2, 2, 2]},
{'5095060C4552175D': [3, 3, 3, 3]}]
In [49]: reduce(lambda x,y: dict(x.items()+y.items()), a)
Out[49]:
{'457D37015E77700E': [2, 2, 2, 2],
'5095060C4552175D': [3, 3, 3, 3],
'A078102C949EC2AB': [1, 2, 3, 4]}
In [50]:
awk指定字段求和
awk -F '=' '{count+=$4} END{print count}' file.log
找出在列表1中但不在列表2中的元素
list(set(lst1).difference(set(lst2)))
解析url，把字段转换成字典
# 方法一
In [5]: url = 'index?name=standby&age=18&city=beijing'
In [6]: parameter = url.split('?')[1]
In [7]: parameter
Out[7]: 'name=standby&age=18&city=beijing'
In [8]: dict(map(lambda x:x.split('='),parameter.split('&')))
Out[8]: {'age': '18', 'city': 'beijing', 'name': 'standby'}
In [9]:
# 方法二
In [9]: import urlparse
In [10]: query = urlparse.urlparse(url).query
In [11]: query
Out[11]: 'name=standby&age=18&city=beijing'
In [12]: dict([(k, v[0]) for k, v in urlparse.parse_qs(query).items()])
Out[12]: {'age': '18', 'city': 'beijing', 'name': 'standby'}
In [13]:
fromkeys使用的陷阱
In [1]: a = dict.fromkeys(['k1','k2','k3'],{})
In [2]: a
Out[2]: {'k1': {}, 'k2': {}, 'k3': {}}
In [3]: a['k1']['2018-10-10'] = 'hi'
In [4]: a
Out[4]:
{'k1': {'2018-10-10': 'hi'},
'k2': {'2018-10-10': 'hi'},
'k3': {'2018-10-10': 'hi'}}
In [5]:
In [5]: a = dict.fromkeys(['k1','k2','k3'],[])
In [6]: a['k1'].append(999)
In [7]: a
Out[7]: {'k1': [999], 'k2': [999], 'k3': [999]}
In [8]:
In [8]: a = dict.fromkeys(['k1','k2','k3'],0)
In [9]: a['k1'] += 9
In [10]: a
Out[10]: {'k1': 9, 'k2': 0, 'k3': 0}
In [11]:
dateutil库解析时间对象
In [76]: import datetime
In [77]: datetime.datetime.strptime('2019-04-10','%Y-%m-%d')
Out[77]: datetime.datetime(2019, 4, 10, 0, 0)
In [78]: import dateutil
In [79]: dateutil.parser.parse('2019-04-10')
Out[79]: datetime.datetime(2019, 4, 10, 0, 0)
In [80]: dateutil.parser.parse('2019/04/10')
Out[80]: datetime.datetime(2019, 4, 10, 0, 0)
In [81]: dateutil.parser.parse('04/10/2019')
Out[81]: datetime.datetime(2019, 4, 10, 0, 0)
In [82]: dateutil.parser.parse('2019-Apr-10')
Out[82]: datetime.datetime(2019, 4, 10, 0, 0)
In [83]:
合并多个字典
# Python2.7
# 这种方式对资源的一种浪费
# 注意这种方式在Python3中会报错：TypeError: unsupported operand type(s) for +: 'dict_items' and 'dict_items'
In [7]: lst
Out[7]:
[{'k1': [1, 1, 1, 1, 1, 1]},
{'k3': [3, 3, 3, 4, 4, 4]},
{'k5': [5, 5, 5, 6, 6, 6]}]
In [8]: reduce(lambda x,y: dict(x.items()+y.items()), lst)
Out[8]: {'k1': [1, 1, 1, 1, 1, 1], 'k3': [3, 3, 3, 4, 4, 4], 'k5': [5, 5, 5, 6, 6, 6]}
In [9]:
# Python3.6
In [67]: lst
Out[67]:
[{'k1': [1, 1, 1, 1, 1, 1]},
{'k3': [3, 3, 3, 4, 4, 4]},
{'k5': [5, 5, 5, 6, 6, 6]}]
In [68]: reduce(lambda x,y: {**x,**y}, lst)
Out[68]: {'k1': [1, 1, 1, 1, 1, 1], 'k3': [3, 3, 3, 4, 4, 4], 'k5': [5, 5, 5, 6, 6, 6]}
In [69]:
# 另外补充两种兼容Py2和Py3的方法：
# 1. 使用字典的构造函数
reduce(lambda x,y: dict(x, **y), lst)
# 2. 笨办法
{k: v for d in lst for k, v in d.items()}
zip的反操作/unzip
In [2]: lst Out[2]: [[1560239100, 16], [1560239400, 11], [1560239700, 14], [1560240000, 18], [1560240300, 18], [1560240600, 12], [1560240900, 19], [1560241200, 13], [1560241500, 16], [1560241800, 16]] In [3]: tss,vals = [ list(tpe) for tpe in zip(*[ i for i in lst ]) ] In [4]: tss Out[4]: [1560239100, 1560239400, 1560239700, 1560240000, 1560240300, 1560240600, 1560240900, 1560241200, 1560241500, 1560241800] In [5]: vals Out[5]: [16, 11, 14, 18, 18, 12, 19, 13, 16, 16] In [6]:
获取前一天时间并格式化
In [17]: from datetime import timedelta, datetime
In [18]: yesterday = datetime.today() + timedelta(-1)
In [19]: yesterday.strftime('%Y%m%d')
Out[19]: '20190702'
In [20]:
时序数据列表转换位字典结构
In [87]: lst
Out[87]:
[(1562653200, 16408834),
(1562653500, 16180209),
(1562653800, 16178061),
(1562654100, 16147492),
(1562654400, 16103304),
(1562654700, 16182462),
(1562655000, 16334665),
(1562655300, 15440130),
(1562655600, 15433254),
(1562655900, 16607189)]
In [88]: { ts:value for ts,value in lst }
Out[88]:
{1562653200: 16408834,
1562653500: 16180209,
1562653800: 16178061,
1562654100: 16147492,
1562654400: 16103304,
1562654700: 16182462,
1562655000: 16334665,
1562655300: 15440130,
1562655600: 15433254,
1562655900: 16607189}
In [89]:
JavaScript实现table点击列头自动排序
1 function comparer(index) { 2 return function(a, b) { 3 var valA = getCellValue(a, index), 4 valB = getCellValue(b, index); 5 return $.isNumeric(valA) && $.isNumeric(valB) ? 6 valA - valB : valA.localeCompare(valB); 7 }; 8 } 9 function getCellValue(row, index) { 10 return $(row).children('td').eq(index).text(); 11 } 12 $(document).on('click', 'th', function() { 13 var table = $(this).parents('table').eq(0); 14 var rows = table.find('tr:gt(0)').toArray().sort(comparer($(this).index())); 15 this.asc = !this.asc; 16 if (!this.asc) { 17 rows = rows.reverse(); 18 } 19 table.children('tbody').empty().html(rows); 20 });
模拟登录
1 #1 curl 模拟登录并请求数据 2 curl -X GET -u username:password "http://...." 3 4 #2 python requests 模拟登录并请求数据 5 import requests 6 from requests.auth import HTTPBasicAuth 7 requests.get("http://....",auth=HTTPBasicAuth(username, password))
IP地址排序
In [37]: ips
Out[37]:
['10.0.9.11',
'10.13.8.1',
'10.8.1.12',
'10.0.120.99',
'10.8.1.110',
'10.0.13.12']
In [38]: sorted(ips)
Out[38]:
['10.0.120.99',
'10.0.13.12',
'10.0.9.11',
'10.13.8.1',
'10.8.1.110',
'10.8.1.12']
In [39]: sorted(ips, key=lambda x:''.join([ i.rjust(3,'0') for i in x.split('.') ]))
Out[39]:
['10.0.9.11',
'10.0.13.12',
'10.0.120.99',
'10.8.1.12',
'10.8.1.110',
'10.13.8.1']
In [40]:
生成随机字符串
In [7]: import random In [8]: import string In [9]: def id_generator(size): ...: chars = string.ascii_letters+string.digits ...: return ''.join(random.choice(chars) for _ in range(size)) ...: In [10]: id_generator(9) Out[10]: 'fL58Mqb7p' In [11]: id_generator(9) Out[11]: 'dijygQlZZ' In [13]: id_generator(12) Out[13]: 'k4Vki2BxREIy' In [14]: id_generator(7) Out[14]: 'N6sZmGK' In [15]:
生成指点范围时间戳列表
dis = pd.date_range(start='2022-12-22 00:00:00', end='2023-01-01 00:00:00', freq="min", closed="left").to_list()
tss = [ int(di.tz_localize("Asia/Shanghai").timestamp()) for di in dis ]
生成上个月日期列表（每天）
now_dt = datetime.datetime.now()
month_dt = now_dt.replace(day=1) - datetime.timedelta(days=1)
month = month_dt.strftime('%Y%m')
s_date_str = month_dt.strftime('%Y-%m-01 00:00:00')
e_date_str = now_dt.strftime('%Y-%m-01 00:00:00')
days = pd.date_range(start=s_date_str, end=e_date_str, freq="D", closed="left", tz="Asia/Shanghai").tolist()
day_str_lst = [day.strftime("%Y%m%d%H%M%S") for day in days]
Pandas生成Excel文件 reference
with pd.ExcelWriter('output.xlsx') as writer:
df1.to_excel(writer, sheet_name='Sheet_name_1')
df2.to_excel(writer, sheet_name='Sheet_name_2')
解析Excel文件不同的sheet
xls = pd.ExcelFile(f)
name2df ={}
for name in xls.sheet_names:
name2df[name] = pd.read_excel(f, sheet_name=name)
Pandas DataFrame tips
df = pd.DataFrame(series)
df = df[["username", "city", "online_time", "online_days"]] # 重新组合各个列
df = df[df["online_days"] > 1] # 按照指定列的值进行过滤
df = df.sort_values(['username', 'online_days'], ascending=True) # 按照value排序
df.to_excel("special_add_dcache_list.xlsx", index=None) # 输出excel
作者：Standby — 一生热爱名山大川、草原沙漠，还有我们小郭宝贝！
出处：http://www.cnblogs.com/standby/
本文版权归作者和博客园共有，欢迎转载，但未经作者同意必须保留此段声明，且在文章页面明显位置给出原文连接，否则保留追究法律责任的权利。
出处：http://www.cnblogs.com/standby/
本文版权归作者和博客园共有，欢迎转载，但未经作者同意必须保留此段声明，且在文章页面明显位置给出原文连接，否则保留追究法律责任的权利。
