python多线程学习笔记

一.多线程相关python包/模块

　　1.thread

　　　　该模块有问题，使用不便，在python3中被保留，并且改成了_thread

　　2.threading

　　　　python3中通用的多线程包

二.什么是多线程

　　案例1：

　　无多线程程序代码

import time

def loop1():
    print('loop1 begining:',time.ctime())
    time.sleep(4)
    print('loop1 ending:',time.ctime())

def loop2():
    print('loop2 begining:',time.ctime())
    time.sleep(2)
    print('loop2 ending:',time.ctime())

def main():
    print('main beginging',time.ctime())
    loop1()
    loop2()
    print('main ending',time.ctime())

if __name__ == '__main__':
    main()

main beginging Wed Aug 22 21:33:34 2018
loop1 begining: Wed Aug 22 21:33:34 2018
loop1 ending: Wed Aug 22 21:33:38 2018
loop2 begining: Wed Aug 22 21:33:38 2018
loop2 ending: Wed Aug 22 21:33:40 2018
main ending Wed Aug 22 21:33:40 2018

　　案例2：

　　含有多线程程序代码1

import time
import _thread as thread

def loop1():
    print('loop1 begining:',time.ctime())
    time.sleep(4)
    print('loop1 ending:',time.ctime())

def loop2():
    print('loop2 begining:',time.ctime())
    time.sleep(2)
    print('loop2 ending:',time.ctime())

def main():
    print('main beginging:',time.ctime())
    thread.start_new_thread(loop1,())
    thread.start_new_thread(loop2,())
    print('main ending:',time.ctime())

if __name__ == '__main__':
    main()

main beginging: Wed Aug 22 21:35:07 2018
main ending: Wed Aug 22 21:35:07 2018

启动多线程函数:start_new_thead
参数两个，一个是需要运行的函数名，第二是函数的参数作为元祖使用，为空则使用空元祖
注意：如果函数只有一个参数，需要参数后由一个逗号

 

案例中,main函数执行完，程序立刻退出.loop1,loop2并没有执行完

main函数相当于包工头

loop1,loop2相当于工人

main函数（包工头）分配loop1,loop2（工人）任务之后，立刻报告给boss,没有等loop1,loop2完成各自的任务

　　案例3：

　　含有多线程程序代码2-改进版

import time
import _thread as thread

def loop1():
    print('loop1 begining:',time.ctime())
    time.sleep(4)
    print('loop1 ending:',time.ctime())

def loop2():
    print('loop2 begining:',time.ctime())
    time.sleep(2)
    print('loop2 ending:',time.ctime())

def main():
    print('main beginging:',time.ctime())
    thread.start_new_thread(loop1,())
    thread.start_new_thread(loop2,())
    print('main ending:',time.ctime())

if __name__ == '__main__':
    main()
    while True:
        time.sleep(1)

main beginging: Wed Aug 22 21:45:10 2018
main ending: Wed Aug 22 21:45:10 2018
loop1 begining: Wed Aug 22 21:45:10 2018
loop2 begining: Wed Aug 22 21:45:10 2018
loop2 ending: Wed Aug 22 21:45:12 2018
loop1 ending: Wed Aug 22 21:45:14 2018

在改进案例中：

增加while True: // time.sleep()

while True:无限循环

这样就避免了程序在main函数执行完后立刻退出，使得loop1,loop2得以执行

　　案例4：

　　含有多线程程序代码3-带参数

from time import ctime,sleep
import _thread as thread

def loop1(x):
    print('loop1 begining:',ctime())
    sleep(4)
    print('LOOP1:I am {0}'.format(x))
    print('loop1 ending: ' ,ctime())

def loop2(x,y):
    print('loop2 begining:',ctime())
    sleep(2)
    print('LOOP2:I am {0}\nLOOP2:I am {1}'.format(x,y))
    print('loop2 ending: ' ,ctime())

def main():
    print('main begining:',ctime())
    thread.start_new_thread(loop1,('Jone',))
    thread.start_new_thread(loop2,('Tom','king'))
    print('main ending: ',ctime())

if __name__ == '__main__':
    main()
    while True:
        sleep(1)

 

三.threading模块的使用

　　 案例1：

import time
import threading

def loop1(x):
    print('loop1 start:',time.ctime())
    print('LOOP1:I am ',x)
    time.sleep(4)
    print('loop1 end:',time.ctime())

def loop2(x,y):
    print('loop2 start:',time.ctime())
    print('LOOP2:I am',x)
    print('LOOP2:I am',y)
    time.sleep(2)
    print('loop2 end:',time.ctime())

def main():
    print('main start:',time.ctime())
    t1 = threading.Thread(target = loop1,args = ('JONE',))
    t2 = threading.Thread(target = loop2,args = ('KIMI','SUKI'))
    t1.start()
    t2.start()
    print('main end:',time.ctime())
if __name__ == '__main__':
    main()

main start: Wed Aug 22 23:41:38 2018
loop1 start: Wed Aug 22 23:41:38 2018
LOOP1:I am  JONE
loop2 start: Wed Aug 22 23:41:38 2018
LOOP2:I am KIMI
LOOP2:I am SUKI
main end: Wed Aug 22 23:41:38 2018
loop2 end: Wed Aug 22 23:41:40 2018
loop1 end: Wed Aug 22 23:41:42 2018

　　案例2：

　　join案例

import time
import threading

def loop1(x):
    print('loop1 start:',time.ctime())
    print('LOOP1:I am ',x)
    time.sleep(4)
    print('loop1 end:',time.ctime())

def loop2(x,y):
    print('loop2 start:',time.ctime())
    print('LOOP2:I am',x)
    print('LOOP2:I am',y)
    time.sleep(2)
    print('loop2 end:',time.ctime())

def main():
    print('main start:',time.ctime())
    t1 = threading.Thread(target = loop1,args = ('JONE',))
    t2 = threading.Thread(target = loop2,args = ('KIMI','SUKI'))
    t1.start()
    t2.start()

    t1.join()
    t2.join()
    print('main end:',time.ctime())
if __name__ == '__main__':
    main()

main start: Wed Aug 22 23:44:50 2018
loop1 start: Wed Aug 22 23:44:50 2018
LOOP1:I am  JONE
loop2 start: Wed Aug 22 23:44:50 2018
LOOP2:I am KIMI
LOOP2:I am SUKI
loop2 end: Wed Aug 22 23:44:52 2018
loop1 end: Wed Aug 22 23:44:54 2018
main end: Wed Aug 22 23:44:54 2018

t.start()：开始启动多线程 

t.join()：等待多线性执行完成

threading.Thread(target = function,args = ())

可以发现，与_thread相比，没有while True 也能正常的执行

threading更为方便

四.守护线程

　　案例1：

　　无守护线程函数

import threading
import time

def fun():
    print('fun START')
    time.sleep(2)
    print('fun END')

t = threading.Thread(target = fun,args = ())
t.start()
time.sleep(1)
print('main END')

fun START
main END
fun END

　　案例2：

　　存在守护线程函数

import threading
import time

def fun():
    print('fun START')
    time.sleep(2)
    print('fun END')

t = threading.Thread(target = fun,args = ())
t.setDaemon(True)
t.start()
time.sleep(1)
print('main END')

fun START
main END

 

守护线程：

t.setDaemon(True)

或者

t.daemon = True

守护线程必须在start之前设置

在程序中把子线程设置成守护线程，那么在主线程结束的时候，该子线程也会跟着结束

通常，守护线程不可以离开主线程运行

守护线程，能否成功执行与编程环境有关

五.常用线程属性

1.threading.currentThread:返回当前线程变量

2.threadng.enumerate:返回一个包含正在运行的线程列表

3.threading.activeCount:返回正在运行线程数量，效果与len(threading.enumerate)相同

4.thr.setName:设置线程名字

5.thr.getName:获得线程名字

 

　　案例1：

import time
import threading

def loop1():
    print('loop1 start:',time.ctime())
    time.sleep(5)
    print('loop1 end:',time.ctime())

def loop2():
    print('loop2 start:',time.ctime())
    time.sleep(2)
    print('loop2 start:',time.ctime())
def loop3():
    print('loop3 start:',time.ctime())
    time.sleep(1)
    print('loop3 end:',time.ctime())

def main():
    print('MAIN START:',time.ctime())
    t1 = threading.Thread(target = loop1,args = ())
    t2 = threading.Thread(target = loop2,args = ())
    t3 = threading.Thread(target = loop3,args = ())
    t1.setName('Thr1')
    t2.setName('Thr2')
    t3.setName('Thr3')
    t1.start()
    t2.start()
    t3.start()
    time.sleep(3)
    for i in threading.enumerate():
        print('正在运行的子线程有：',i.getName())
    print('正在运行的子线程数量是：',threading.activeCount())
    t1.join()
    t2.join()
    t3.join()
    print('MAIN END:',time.ctime())

if __name__ == '__main__':
    main()

MAIN START: Fri Aug 24 23:27:08 2018
loop1 start: Fri Aug 24 23:27:08 2018
loop2 start: Fri Aug 24 23:27:08 2018
loop3 start: Fri Aug 24 23:27:08 2018
loop3 end: Fri Aug 24 23:27:09 2018
loop2 start: Fri Aug 24 23:27:10 2018
正在运行的子线程有： MainThread
正在运行的子线程有： Thr1
正在运行的子线程数量是： 2
loop1 end: Fri Aug 24 23:27:13 2018
MAIN END: Fri Aug 24 23:27:13 2018

六.线程继承

　　案例1：

import threading
import time

class Thread_My(threading.Thread):
    def __init__(self,args):
        self.args = args
    def run(self):
        time.sleep(2)
        print('The arg is {0}'.format(self.args))
        print('The time is {0}'.format(time.ctime()))

for i in range(5):
    t = Thread_My(i)
    t.start()
    t.join()

print('\nmian loop end',time.ctime())

 

Traceback (most recent call last):
  File "D:/pathon L/全栈/多线程/threading_inherit.py", line 15, in <module>
    t.start()
  File "C:\ProgramData\Anaconda3\lib\threading.py", line 839, in start
    raise RuntimeError("thread.__init__() not called")
RuntimeError: thread.__init__() not called

报错，父类初始化__init__函数没有被调用，不能正常使用父类函数模块

　　案例2：

　　更改后

import threading
import time

class Thread_My(threading.Thread):
    def __init__(self,args):
        super(Thread_My,self).__init__()
        self.args = args
    def run(self):
        time.sleep(2)
        print('The arg is {0}'.format(self.args))
        print('The time is {0}'.format(time.ctime()))

for i in range(5):
    t = Thread_My(i)
    t.start()
    t.join()

print('\nmian loop end',time.ctime())

The arg is 0
The time is Fri Aug 24 23:55:21 2018
The arg is 1
The time is Fri Aug 24 23:55:23 2018
The arg is 2
The time is Fri Aug 24 23:55:25 2018
The arg is 3
The time is Fri Aug 24 23:55:27 2018
The arg is 4
The time is Fri Aug 24 23:55:29 2018

mian loop end Fri Aug 24 23:55:29 2018

Thread_My继承threading.Thread的功能

run函数是原threading.Thread的内置函数

run函数代表真正执行的功能

将run函数改写成符合目的的函数

　　案例3：

　　继承——工业风

import threading
from time import sleep, ctime



class ThreadFunc:

    def __init__(self, name):
        self.name = name

    def loop(self, nloop, nsec):
        '''
        :param nloop: loop函数的名称
        :param nsec: 系统休眠时间
        :return:
        '''
        print('Start loop ', nloop, 'at ', ctime())
        sleep(nsec)
        print('Done loop ', nloop, ' at ', ctime())

def main():
    print("Starting at: ", ctime())

    # ThreadFunc("loop").loop 跟一下两个式子相等：
    # t = ThreadFunc("loop")
    # t.loop
    # 以下t1 和  t2的定义方式相等
    t = ThreadFunc("loop")
    t1 = threading.Thread( target = t.loop, args=("LOOP1", 4))
    # 下面这种写法更西方人，工业化一点
    t2 = threading.Thread( target = ThreadFunc('loop').loop, args=("LOOP2", 2))

    # 常见错误写法
    #t1 = threading.Thread(target=ThreadFunc('loop').loop(100,4))
    #t2 = threading.Thread(target=ThreadFunc('loop').loop(100,2))

    t1.start()
    t2.start()

    t1.join( )
    t2.join()


    print("ALL done at: ", ctime())


if __name__ == '__main__':
    main()

 七.共享变量问题

　　案例1：

import threading

sum = 0
sumd = 1000000

def my_Add():
    global sum,sumd
    for i in range(sumd):
        sum += 1

def my_Minu():
    global sum,sumd
    for i in range(sumd):
        sum -= 1

def main():
    print('main start:',sum)
    t1 = threading.Thread(target = my_Add,args = ())
    t2 = threading.Thread(target = my_Minu,args = ())
    t1.start()
    t2.start()
    t1.join()
    t2.join()
    print('main end:',sum)

if __name__ == '__main__':
    main()

main start: 0
main end: 462177

 共享变量在多线程中被同时访问会出现混乱，如案例1

解决方法：锁，信号灯

 

八.锁（lock）

　　案例1：

import threading

sum = 0
sumd = 1000000
lock = threading.Lock()

def my_Add():
    global sum,sumd
    for i in range(sumd):
        lock.acquire()
        sum += 1
        lock.release()

def my_Minu():
    global sum,sumd
    for i in range(sumd):
        lock.acquire()
        sum -= 1
        lock.release()

def main():
    print('main start:',sum)
    t1 = threading.Thread(target = my_Add,args = ())
    t2 = threading.Thread(target = my_Minu,args = ())
    t1.start()
    t2.start()
    t1.join()
    t2.join()
    print('main end:',sum)

if __name__ == '__main__':
    main()

 锁：标志一个线程在使用该资源

使用方法：

1.上锁：lock.acquire()

2.稳定使用共享资源

3.释放锁：lock.release()

 

九.线程安全问题

如果一个资源/变量在不加锁的情况下也不会引起任何问题，则为线程安全

线程安全变量类型：queue

线程不安全变量类型：list,set,dict...

 

十.生产消费者问题

　　 案例1：

import threading
import queue
import time

class Producer(threading.Thread):
    def run(self):
        global queue
        count = 0
        while True:
            if q.qsize() < 1000:
                for i in range(100):
                    count += 1
                    msg = self.name + '生产产品' + str(count)
                    q.put(msg)
                    print(msg)
            time.sleep(0.5)

class Customer(threading.Thread):
    def run(self):
        global queue
        while True:
            if q.qsize() > 100:
                for i in range(3):
                    msg = self.name + '消费' + q.get()
                    print(msg)
            time.sleep(1)


if __name__ == '__main__':
    q = queue.Queue()

    for i in range(500):
        q.put('初始产品'+str(i))
    for i in range(2):
        p = Producer()
        p.start()
    for i in range(5):
        c = Customer()
        c.start()

queue为用来存放变量的数据结构

特点：先进先出

可视为特殊的列表

q = queue.Queue()

q.qsize():返回q内容长度

q.put():在q中放入数据

q.get():在q中获得数据

 

十一.死锁问题

import threading
import time

l1 = threading.Lock()
l2 = threading.Lock()

def fun1():
    l1.acquire()
    print('fun1 申请了 l1......')
    time.sleep(2)
    print('fun1 等待l2......')
    l2.acquire()
    print('fun1 申请了 l2......')
    l1.release()
    print('fun1 释放了 ll......')
    l2.release()
    print('fun1 释放了 l2......')

def fun2():
    l2.acquire()
    print('fun2 申请了 l2......')
    time.sleep(4)
    print('fun2 等待l1......')
    l1.acquire()
    print('fun2 申请了 l1......')
    l2.release()
    print('fun2 释放了 l2......')
    l1.release()
    print('fun2 释放了 l1')

if __name__ == '__main__':
    t1 = threading.Thread(target = fun1,args = ())
    t2 = threading.Thread(target = fun2,args = ())
    t1.start()
    t2.start()
    t1.join()
    t2.join()

fun1 申请了 l1......
fun2 申请了 l2......
fun1 等待l2......
fun2 等待l1......

 

十二.锁的等待时间设置

 　　案例1：

import time
import threading

lock_1 = threading.Lock()
lock_2 = threading.Lock()

def fun1():
    print('start fun1......')
    lock_1.acquire()
    print('fun1 申请了 lock_1...')
    time.sleep(2)
    print('fun1 等待 lock_2...')
    st = lock_2.acquire(timeout = 10)
    if st:
        print('fun1 申请了 lock_2...')
        lock_1.release()
        print('fun1 释放了 lock_1...')
        lock_2.release()
        print('fun1 释放了 lock_2...')
    else:
        print('fun1 申请 lock_2 失败')
        lock_1.release()
        print('fun1 释放了 lock_1...')
    print('end fun1......')

def fun2():
    print('start fun2......')
    lock_2.acquire()
    print('fun2 申请了 lock_2...')
    time.sleep(4)
    print('fun2 等待 lock_1...')
    lock_1.acquire()
    print('fun2 申请了 lock_1...')
    lock_2.release()
    print('fun2 释放了 lock_2...')
    lock_1.release()
    print('fun2 释放了 lock_1...')
    print('end fun2......')

def main():
    print('main begining......')
    t1 = threading.Thread(target = fun1,args = ())
    t2 = threading.Thread(target = fun2,args = ())
    t1.start()
    t2.start()
    t1.join()
    t2.join()
    print('main end......')

if __name__ == '__main__':
    main()

main begining......
start fun1......
fun1 申请了 lock_1...
start fun2......
fun2 申请了 lock_2...
fun1 等待 lock_2...
fun2 等待 lock_1...
fun1 申请 lock_2 失败
fun1 释放了 lock_1...
end fun1......
fun2 申请了 lock_1...
fun2 释放了 lock_2...
fun2 释放了 lock_1...
end fun2......
main end......

 

使用timeout(t)，如果在t时间内没有申请到该锁，则放弃申请继续下一步

 

十三.线程处理方法

　　案例1:

import time
import threading

def fun():
    print('FUN START AT',time.ctime())
    time.sleep(3)
    print('FUN END AT',time.ctime())

if __name__ == '__main__':
    print('MAIN BEGIN AT',time.ctime())
    t = threading.Timer(6,fun)
    t.start()
    t.join()
    print('MAIB DONE AT',time.ctime())

MAIN BEGIN AT Sat Aug 25 20:31:14 2018
FUN START AT Sat Aug 25 20:31:20 2018
FUN END AT Sat Aug 25 20:31:23 2018
MAIB DONE AT Sat Aug 25 20:31:23 2018

 timer:利用多线程，在指定时间之后启动一个功能

　　案例2：

　　未使用可重入锁

import time
import threading

class My_Thread(threading.Thread):
    def run(self):
        global num
        time.sleep(1)
        if suo.acquire():
            num += 1
            msg = self.name + ' set to num ' + str(num)
            print(msg)
            suo.acquire()
            suo.release()
            suo.release()

def main():
    for i in range(1,6):
        print('START ',i)
        t = My_Thread()
        t.start()
        print('END ',i,'\n')

num = 0
suo = threading.Lock()

if __name__ == '__main__':
    main()　　

START  1
END  1 

START  2
END  2 

START  3
END  3 

START  4
END  4 

START  5
END  5 

Thread-3 set to num 1

　　

　　案例3：

　　可重入锁

import time
import threading

class My_Thread(threading.Thread):
    def run(self):
        global num
        time.sleep(1)
        if suo.acquire():
            num += 1
            msg = self.name + ' set to num ' + str(num)
            print(msg)
            suo.acquire()
            suo.release()
            suo.release()

def main():
    for i in range(1,6):
        print('START ',i)
        t = My_Thread()
        t.start()
        print('END ',i,'\n')

num = 0
suo = threading.RLock()

if __name__ == '__main__':
    main()

 

START  1
END  1 

START  2
END  2 

START  3
END  3 

START  4
END  4 

START  5
END  5 

Thread-3 set to num 1
Thread-5 set to num 2
Thread-1 set to num 3
Thread-4 set to num 4
Thread-2 set to num 5

 

将Lock改为RLock，即为可重入锁

使用可重入锁后，一个锁可以被一个线程多次申请　

　　案例4：

　　加入join()的可重入锁

import time
import threading

class My_Thread(threading.Thread):
    def run(self):
        global num
        time.sleep(1)
        if suo.acquire():
            num += 1
            msg = self.name + ' set to num ' + str(num)
            print(msg)
            suo.acquire()
            suo.release()
            suo.release()

def main():
    for i in range(1,6):
        print('START ',i)
        t = My_Thread()
        t.start()
        t.join()
        print('END ',i,'\n')

num = 0
suo = threading.RLock()

if __name__ == '__main__':
    main()

START  1
Thread-1 set to num 1
END  1 

START  2
Thread-2 set to num 2
END  2 

START  3
Thread-3 set to num 3
END  3 

START  4
Thread-4 set to num 4
END  4 

START  5
Thread-5 set to num 5
END  5 

案例二三中，线程的顺序是随机的

在加入join()之后，开始排序

　　案例5：

import time
import threading

sign = threading.Semaphore(3)
def fun():
    if sign.acquire():
        for i in range(5):
            print(threading.currentThread().getName() + 'get Semaphore')
        time.sleep(5)
        sign.release()
        print(threading.currentThread().getName() + 'release Semaphore')
12
def main():

    for i in range(9):
        t = threading.Thread(target = fun,args = ())
        t.start()

if __name__ == '__main__':
    main()

Thread-1get Semaphore
Thread-1get Semaphore
Thread-1get Semaphore
Thread-1get Semaphore
Thread-1get Semaphore
Thread-2get Semaphore
Thread-2get Semaphore
Thread-2get Semaphore
Thread-2get Semaphore
Thread-2get Semaphore
Thread-3get Semaphore
Thread-3get Semaphore
Thread-3get Semaphore
Thread-3get Semaphore
Thread-3get Semaphore
Thread-2release Semaphore
Thread-3release Semaphore
Thread-4get Semaphore
Thread-4get Semaphore
Thread-4get Semaphore
Thread-4get Semaphore
Thread-4get Semaphore
Thread-5get Semaphore
Thread-5get Semaphore
Thread-1release Semaphore
Thread-5get Semaphore
Thread-6get Semaphore
Thread-6get Semaphore
Thread-6get Semaphore
Thread-6get Semaphore
Thread-6get Semaphore
Thread-5get Semaphore
Thread-5get Semaphore
Thread-4release Semaphore
Thread-7get Semaphore
Thread-7get Semaphore
Thread-7get Semaphore
Thread-7get Semaphore
Thread-7get Semaphore
Thread-5release Semaphore
Thread-8get Semaphore
Thread-8get Semaphore
Thread-8get Semaphore
Thread-8get Semaphore
Thread-8get Semaphore
Thread-6release Semaphore
Thread-9get Semaphore
Thread-9get Semaphore
Thread-9get Semaphore
Thread-9get Semaphore
Thread-9get Semaphore
Thread-9release Semaphore
Thread-7release Semaphore
Thread-8release Semaphore

 Semaphore：允许一个资源最多有几个多线程同时使用

十四.线程替代方法

- subprocess
- 完全跳过线程，使用进程
- 是派生进程的主要替代方案
- python2.4后引入

- multiprocessiong
- 使用threadiing借口派生，使用子进程
- 允许为多核或者多cpu派生进程，接口跟threading非常相似
- python2.6

- concurrent.futures
- 新的异步执行模块
- 任务级别的操作
- python3.2后引入