Python-并发和线程

1、并发和并行的区别

　　并行：parallel

　　　　同一时刻上，有多件互不干扰的事要做。

　　并发：concurrency

　　　　同一时间内，多少事要做。

　　补充：

buffer(缓冲)是为了提高内存和硬盘或其他I/0设备之间的数据交换的速度而设计的。 
cache(缓存)是为了提高cpu和内存之间的数据交换速度而设计。

 

2、并发的解决

　　什么是高并发：同一个时间段，发起大量的数据请求

　　2.1、队列、缓冲区：

　　　　使用队列就是，其实就是一个缓冲地带，即缓冲区，当然根据优先级别，设置不同的队列，优先级高的的先处理。例如queue模块中的类Queue , LifoQueue, PriorityQueue

　　2.2、争抢型：网站一般不会选择抢占处理并发，否则有些用户总觉得反应慢。

　　　　资源争抢型，当争抢到资源后，就不会为其他请求提供服务，这就是：锁机制 其他请求只能等待。

　　　　但是这样并不好，因为有可能有的请求一直抢不到资源。

　　2.3、预处理：基于lru原则

　　　　一种提前加载用户所需数据的思路，即预处理思想，也就是缓存思想

　　　　常用的是字典，如果数据量大的时候，使用内存数据库，即Redis 或者memcached。

　　2.4、并行：

　　　　日常可以通过购买更多的服务器，或多开进程，线程实现并行处理，来解决并发问题。

　　　　注意这些都是水平扩展 思想

　　　　但是 这样成本上升。

　　　　注:

　　　　每个cpu称为一路，一个cpu有多核，如果 单cpu单核 是无法并行，看起来是像 并行，但并不是，只是轮询。

　　2.5、提速：

　　　　提高单个cpu的性能，或者单个服务器安装更多的cpu，或者提高程序的效率。

　　　　垂直扩展思想

　　2.6、消息中间件：

　　　　即系统之外的队列，常见的消息中间件：RabbitMQ， ActvieMQ（Apache），RocketMQ( 阿里Apache)， kafka(Apavhe)。

　　　　通过消息中间件，成为分布式系统，对生产者和消费者（程序之间）， 解耦同时做缓冲，将数据平滑起来。

　　2.7、多地区：多地服务器

 

3、进程和线程

　　在实现了线程的操作系统中(并不是所有的os都实现了 线程)，线程是操作系统能够进行运算调度的最小单位，它被包含在进程中，是进程中的实际运作单位，一个程序的执行实例就是一个进程。

　　进程（process）是计算机中的程序关于某数据集合上的一次运行活动，是系统进行资源分配和调度的基本单位，是操作系统结构的基础。

　　进程和程序的区别：

　　　　程序是源代码编译后的文件，而这些文件存放在磁盘上，当程序被操作系统加载到内存中，就是进程，进程中存放着指令和数据（资源），它也是线程的容器。

　　Linux是分父进程和子进程，Windows的进程时平等关系。

　　线程，有时 被称为轻量级进程（ Lightweight Process ,  LWP）,是程序执行流的最小单元。

　　一个标准的线程由线程ID ，当前指令指针(PC），寄存器集合和堆栈组成。

　　在许多系统中，创建一个线程比创建一个进程块 10 --- 100 倍，但这并不是说线程越多越好。

　　进程和线程的理解：

　　　　现代操作系统提出进程的概念，没一个进程都认为自己独自占有所有的计算机硬件资源，

　　　　进程就是独立的王国，进程间不可以随便共享数据。

　　　　线程可以共享进程的资源，但是线程之间也不是可以随便共享数据。

　　　　每一个线程拥有自己独立的堆栈。不一定都实现堆，但是一定实现栈。

4、线程的状态：

状态 含义
就绪（ready） 线程能够运行，但在等待被调度，可能线程刚刚创建启动，或刚刚从阻塞中恢复，或被其他线程抢占。
运行（running） 线程正在运行
阻塞（blocked） 线程等待外部事件发生而无法运行，如I/O操作
终止（terminal） 线程完成，或者退出，或被取消

 

　　　　

　　　　注：针对单cpu单核：cpu一次只能调度一个线程，要不抢占，要不优先，要不排队。

　　　　　　单cpu单核，轮询对线程调度，每个线程调度时间片由于优先级也有不同。

5、Python中的进程和线程。

　　进程会启动一个解释器进程，线程共享一个解释器进程。　

6、Python的线程开发：

　　Python的线程开发使用标准库threading　

7、Thread类　

Thread原码：
def __init__(self, group=None, target=None, name=None,
              args=(), kwargs=None, *, daemon=None):

 

　　target：线程调用的对象，就是目标函数

　　name：为线程起一个名字

　　args：为目标函数传递实参，元组（注意一个元素的时候，逗号）

　　kwarg：为目标函数关键字传参，字典

8、线程启动：

 1 import  threading
 2 
 3 # 最简单的线程程序
 4 def work():
 5     print('working')
 6     print('finished')
 7 
 8 t = threading.Thread(target=work, name='work1')
 9 
10 t.start()

 

　　线程之所以执行函数是因为线程中就是执行代码的，而最简单的封装就是函数，所以还是函数调用。

　　函数执行完，线程也就退出了。

　　如果不让线程退出，或者让线程一直工作。通过死循环。　

 1 import  threading
 2 import time
 3 
 4 def work():
 5     while True:
 6         time.sleep(1)
 7         print('working')
 8     print('finished')
 9 
10 t = threading.Thread(target=work, name='work1')
11 
12 t.start()

 

9、线程退出：

　　Python没有提供线程退出的方法，线程在下面的情况下退出：

　　1、线程函数内语句执行完毕

　　2、线程函数中抛出未处理的异常。　

 1 import  threading
 2 import time
 3 
 4 def work():
 5     count = 0
 6     while True:
 7         if count > 5:
 8             break
 9             # raise Exception('count')
10             # return 也是可以的
11         time.sleep(1)
12         print('working')
13         count += 1
14 
15 t = threading.Thread(target=work, name='work1')
16 t.start()
17 print('--------------------')
18 print('======= end  =======')
1 --------------------
2 ======= end  =======
3 working
4 working
5 working
6 working
7 working
8 working
结果

 

　　注：Python的线程没有优先级，没有线程组的概念，也不能销毁、停止、挂起，那也就没有恢复、中断了。

10、线程的传参

 1 import  threading
 2 import time
 3 
 4 def add(x, y):
 5     print('{} + {} = {}'.format(x, y, x + y), threading.current_thread().ident)
 6 
 7 t1 = threading.Thread(target=add, name='add', args=(4, 5))
 8 t1.start()
 9 
10 time.sleep(2)
11 
12 t2 = threading.Thread(target=add, name='add', args=(5,), kwargs={'y':4})
13 t2.start()
14 
15 time.sleep(2)
16 
17 t3 = threading.Thread(target=add, name='add', kwargs={'x':4, 'y':5})
18 t3.start()
1 4 + 5 = 9 8604
2 5 + 4 = 9 9836
3 4 + 5 = 9 10932
结果

 

　　注：线程传参和函数传参没有什么区别，本质上就是函数传参。

　　threading的属性和方法：

名称 含义
current_thread() 返回当前线程对象
main_thread() 返回主线程对象
active_count() 当前处于alive状态的线程个数
enumerate() 返回所有活着的线程列表，不包括已经终止的线程和未开始的线程
get_ident() 返回当前线程的ID ，非0 的整数

　　　　active_cont, enumerate方法返回的值还包括主线程。

　　　　测试：

 1 import  threading
 2 import time
 3 
 4 def showthreadinfo():
 5     print('currenttherad = {}'.format(threading.current_thread()))
 6     print('main thread = {}'.format(threading.main_thread()))
 7     print('active count = {}'.format(threading.active_count()))
 8     print('enmuerate = {}'.format(threading.enumerate()))
 9 
10 def work():
11     count = 0
12     showthreadinfo()
13     while True:
14         if count > 5:
15             break
16         time.sleep(1)
17         count += 1
18         print('working')
19 
20 t = threading.Thread(target=work, name='work1')
21 t.start()
22 
23 print('============ end ==========')

　　　　结果：可以看到有并行的影子，虽然这是假并行！！！

　　

11、Thread实例的属性和方法

　　name：只是一个名字，只是个标记，名称可以重名。getName(), setName() 获取，设置这个名词

　　ident：线程ID， 非0整数，线程启动后才会有ID，否则为None，线程退出，此 ID 依旧可以访问，此ID 可以重复使用。

　　is_alive()： 返回线程是否活着。

　　注：　线程的name这是一个名称，可以重复，ID 必须唯一，但是在线程退出后，可以再利用。

　　测试：

 1 import  threading
 2 import time
 3 
 4 def work():
 5     count = 0
 6     while True:
 7         if count > 5:
 8             break
 9         time.sleep(1)
10         count += 1
11         print(threading.current_thread().name)
12 
13 
14 t = threading.Thread(target=work, name='work1')
15 t.start()
16 
17 while True:
18     time.sleep(1)
19     if t.is_alive():
20         print('{}{} alive'.format(t.name, t.ident))
21     else:
22         print('{}{} dead'.format(t.name, t.ident))
23         t.start()

 

　　结果：结果很明确， 线程对象，只能执行一次start，尽管对象还存在。

　　

 

　　start()： 启动线程，每一个线程必须且只能执行该方法 一次

　　run()； 运行线程函数

　　测试  start():

 1 import  threading
 2 import time
 3 
 4 def work():
 5     count = 0
 6     while True:
 7         if count > 5:
 8             break
 9         time.sleep(1)
10         count += 1
11         print('working------------')
12 
13 class MyThread(threading.Thread):
14     def start(self):
15         print('start -----------------')
16         super().start()
17 
18     def run(self):
19         print('run -------------------')
20         super().run()
21 
22 
23 t = MyThread(target=work, name='work1')
24 t.start()
start()

　　结果：　　

1 start -----------------
2 run -------------------
3 working------------
4 working------------
5 working------------
6 working------------
7 working------------
8 working------------
结果

　　测试 run():

 1 import  threading
 2 import time
 3 
 4 def work():
 5     count = 0
 6     while True:
 7         if count > 5:
 8             break
 9         time.sleep(1)
10         count += 1
11         print('working------------')
12 
13 class MyThread(threading.Thread):
14     def start(self):
15         print('start -----------------')
16         super().start()
17 
18     def run(self):
19         print('run -------------------')
20         super().run()
21 
22 
23 t = MyThread(target=work, name='work1')
24 t.run()
run()

　　结果：

1 run -------------------
2 working------------
3 working------------
4 working------------
5 working------------
6 working------------
7 working------------
结果

　　总结：start() 方法会调用run() 方法，而run() 方法可以运行函数。

 1 ## run()
 2 import  threading
 3 import time
 4 
 5 def work():
 6     count = 0
 7     while True:
 8         if count > 5:
 9             break
10         time.sleep(1)
11         count += 1
12         print('working------------')
13         print(threading.current_thread().name)
14 
15 class MyThread(threading.Thread):
16     def start(self):
17         print('start -----------------')
18         super().start()
19 
20     def run(self):
21         print('run -------------------')
22         super().run()
23 
24 
25 t = MyThread(target=work, name='work1')
26 t.run()
27 
28 ## start()
29 import  threading
30 import time
31 
32 def work():
33     count = 0
34     while True:
35         if count > 5:
36             break
37         time.sleep(1)
38         count += 1
39         print('working------------')
40         print(threading.current_thread().name)
41 
42 class MyThread(threading.Thread):
43     def start(self):
44         print('start -----------------')
45         super().start()
46 
47     def run(self):
48         print('run -------------------')
49         super().run()
50 
51 
52 t = MyThread(target=work, name='work1')
53 t.start()
比较start和run

　　结果：

 1 ## start
 2 start -----------------
 3 run -------------------
 4 working------------
 5 work1
 6 working------------
 7 work1
 8 working------------
 9 work1
10 working------------
11 work1
12 working------------
13 work1
14 working------------
15 work1
16 
17 ## run
18 run -------------------
19 working------------
20 MainThread
21 working------------
22 MainThread
23 working------------
24 MainThread
25 working------------
26 MainThread
27 working------------
28 MainThread
29 working------------
30 MainThread
结果

 

　　使用start方法启动线程，启动了一个新的线程，名字叫work1 运行，但是使用run方法的，并没有启动新的线程，就是在主线程中调用了一个普通的函数而已。

　　因此，启动线程使用start方法，才能启动多个线程。

　　run就是单线程的，start是多线程的。

　　线程适可而止，不能太多，否则时间都浪费到切换上了。而且线程是共享资源的，100个线程和100000个线程对资源的获取是不同的。

    start只能一次，其次ID 在线程消亡后 还会留给之前的线程

    run 只是运行线程函数，不会创建线程，依旧还是主线程，在主线程中运行线程函数，
        单线程 是 顺序执行，从上往下执行。

    start 创建子线程，并通过run执行线程函数。

 12、多线程：

　　顾明思议，多线程， 一个进程中如果有多个线程，就是多线程，实现一种并发。

　　测试： start

 1 import threading
 2 import time
 3 
 4 def work():
 5     count = 0
 6     while True:
 7         if count > 5:
 8             break
 9         time.sleep(0.5)
10         count += 1
11         print('worker running')
12         print(threading.current_thread().name, threading.current_thread().ident)
13 class MyThread(threading.Thread):
14     def start(self):
15         print('start -----')
16         super().start()
17 
18     def run(self):
19         print('run -------')
20         super().run()
21 
22 t1 = threading.Thread(name='worker1', target=work)
23 t2 = threading.Thread(name='worker2', target=work)
24 
25 t1.start()
26 t2.start()
start

 

　　结果：

 1 worker runningworker running
 2 worker2
 3  632
 4 worker1 7456
 5 worker running
 6 worker1 7456
 7 worker running
 8 worker2 632
 9 worker running
10 worker1 7456
11 worker running
12 worker2 632
13 worker runningworker running
14 worker2 632
15 
16 worker1 7456
17 worker running
18 worker2 632
19 worker running
20 worker1 7456
21 worker running
22 worker2 632
23 worker running
24 worker1 7456
View Code

 

　　测试： run

 1 import threading
 2 import time
 3 
 4 def work():
 5     count = 0
 6     while True:
 7         if count > 5:
 8             break
 9         time.sleep(0.5)
10         count += 1
11         print('worker running')
12         print(threading.current_thread().name, threading.current_thread().ident)
13 class MyThread(threading.Thread):
14     def start(self):
15         print('start -----')
16         super().start()
17 
18     def run(self):
19         print('run -------')
20         super().run()
21 
22 t1 = MyThread(name='worker1', target=work)
23 t2 = MyThread(name='worker2', target=work)
24 
25 # t1.start()
26 # t2.start()
27 t1.run()
28 t2.run()
run

 　　结果：

 1 run -------
 2 worker running
 3 MainThread 8868
 4 worker running
 5 MainThread 8868
 6 worker running
 7 MainThread 8868
 8 worker running
 9 MainThread 8868
10 worker running
11 MainThread 8868
12 worker running
13 MainThread 8868
14 run -------
15 worker running
16 MainThread 8868
17 worker running
18 MainThread 8868
19 worker running
20 MainThread 8868
21 worker running
22 MainThread 8868
23 worker running
24 MainThread 8868
25 worker running
26 MainThread 8868
View Code

　　可以看出：没有开新的线程， 这就是普通的函数调用，所以执行完t1.run() 然后执行t2.run() ，这里就不是多线程。

　　当使用start方法启动线程后，进程内有多个活动的线程 并行的工作，就是多线程。

　　一个进程中至少有一个线程，并作为程序的入口，这个线程就是主线程，一个进程至少有一个主线程。其他线程称为工作线程。

　　主线程做协调管理，工作线程具体工作，线程在各自的占中调用函数，完成压栈，弹出，互相独立。

 

13、线程安全：

　　IPython中演示，python命令行，pycharm一般很难演示出效果：

　　

　　总结：

　　　　看代码，应该是一行一行打印，但是很多字符串打印在一起，说明print函数被打断了，被线程切换打断了。

　　　　print函数分为两步，第一步打印字符串，第二部打印换行，就在这之间发生了线程切换。

　　　　这说明，pirnt函数是线程不安全的

　　　　线程安全：线程执行一段代码，不会产生不确定的结果，那这段代码就是线程安全的。

　　　　上例中，本以为print 应该是打印文本之后紧跟着一个换行的，但并不是，所以print函数是线程不安全的

　　　　但是发现，print打印的内容是内有打断的，所以可以不让print换行实现线程安全。

　　　　如下测试：

 1 import threading
 2 import time
 3 
 4 def work():
 5     for x in range(100):
 6         print('{} is running\n'.format(threading.current_thread().name), end='')
 7 
 8 for x in range(1, 5):
 9     name = 'worker{}'.format(x)
10     t = threading.Thread(name=name, target=work)
11     t.start()
将换行符作为内容打印

 

　　　　结果：　

  1 D:\python3.7\python.exe E:/code_pycharm/code_test_daye/t1.py
  2 worker1 is running
  3 worker1 is running
  4 worker2 is running
  5 worker1 is running
  6 worker2 is running
  7 worker2 is running
  8 worker2 is running
  9 worker2 is running
 10 worker2 is running
 11 worker2 is running
 12 worker2 is running
 13 worker2 is running
 14 worker2 is running
 15 worker2 is running
 16 worker2 is running
 17 worker2 is running
 18 worker2 is running
 19 worker2 is running
 20 worker2 is running
 21 worker1 is running
 22 worker2 is running
 23 worker1 is running
 24 worker2 is running
 25 worker2 is running
 26 worker2 is running
 27 worker2 is running
 28 worker2 is running
 29 worker2 is running
 30 worker2 is running
 31 worker2 is running
 32 worker2 is running
 33 worker2 is running
 34 worker2 is running
 35 worker2 is running
 36 worker2 is running
 37 worker1 is running
 38 worker1 is running
 39 worker1 is running
 40 worker1 is running
 41 worker1 is running
 42 worker1 is running
 43 worker1 is running
 44 worker1 is running
 45 worker1 is running
 46 worker1 is running
 47 worker1 is running
 48 worker1 is running
 49 worker1 is running
 50 worker1 is running
 51 worker1 is running
 52 worker1 is running
 53 worker1 is running
 54 worker1 is running
 55 worker1 is running
 56 worker1 is running
 57 worker1 is running
 58 worker1 is running
 59 worker1 is running
 60 worker2 is running
 61 worker1 is running
 62 worker1 is running
 63 worker1 is running
 64 worker1 is running
 65 worker1 is running
 66 worker1 is running
 67 worker2 is running
 68 worker1 is running
 69 worker1 is running
 70 worker1 is running
 71 worker3 is running
 72 worker1 is running
 73 worker1 is running
 74 worker1 is running
 75 worker1 is running
 76 worker1 is running
 77 worker1 is running
 78 worker1 is running
 79 worker1 is running
 80 worker1 is running
 81 worker1 is running
 82 worker1 is running
 83 worker1 is running
 84 worker1 is running
 85 worker1 is running
 86 worker1 is running
 87 worker1 is running
 88 worker1 is running
 89 worker1 is running
 90 worker1 is running
 91 worker1 is running
 92 worker1 is running
 93 worker1 is running
 94 worker1 is running
 95 worker1 is running
 96 worker1 is running
 97 worker1 is running
 98 worker1 is running
 99 worker1 is running
100 worker1 is running
101 worker1 is running
102 worker1 is running
103 worker1 is running
104 worker1 is running
105 worker1 is running
106 worker1 is running
107 worker1 is running
108 worker1 is running
109 worker2 is running
110 worker3 is running
111 worker4 is running
112 worker2 is running
113 worker2 is running
114 worker1 is running
115 worker4 is running
116 worker4 is running
117 worker2 is running
118 worker4 is running
119 worker2 is running
120 worker2 is running
121 worker2 is running
122 worker2 is running
123 worker2 is running
124 worker2 is running
125 worker2 is running
126 worker2 is running
127 worker2 is running
128 worker2 is running
129 worker2 is running
130 worker2 is running
131 worker2 is running
132 worker2 is running
133 worker2 is running
134 worker2 is running
135 worker2 is running
136 worker2 is running
137 worker2 is running
138 worker2 is running
139 worker2 is running
140 worker2 is running
141 worker4 is running
142 worker1 is running
143 worker1 is running
144 worker1 is running
145 worker1 is running
146 worker3 is running
147 worker3 is running
148 worker3 is running
149 worker3 is running
150 worker3 is running
151 worker3 is running
152 worker3 is running
153 worker2 is running
154 worker4 is running
155 worker4 is running
156 worker4 is running
157 worker4 is running
158 worker4 is running
159 worker4 is running
160 worker4 is running
161 worker4 is running
162 worker4 is running
163 worker4 is running
164 worker4 is running
165 worker4 is running
166 worker4 is running
167 worker4 is running
168 worker4 is running
169 worker4 is running
170 worker4 is running
171 worker4 is running
172 worker4 is running
173 worker4 is running
174 worker4 is running
175 worker4 is running
176 worker4 is running
177 worker4 is running
178 worker4 is running
179 worker4 is running
180 worker4 is running
181 worker4 is running
182 worker4 is running
183 worker4 is running
184 worker4 is running
185 worker4 is running
186 worker4 is running
187 worker4 is running
188 worker4 is running
189 worker4 is running
190 worker4 is running
191 worker4 is running
192 worker4 is running
193 worker4 is running
194 worker4 is running
195 worker4 is running
196 worker4 is running
197 worker4 is running
198 worker4 is running
199 worker4 is running
200 worker4 is running
201 worker4 is running
202 worker4 is running
203 worker4 is running
204 worker4 is running
205 worker4 is running
206 worker4 is running
207 worker4 is running
208 worker4 is running
209 worker4 is running
210 worker1 is running
211 worker3 is running
212 worker3 is running
213 worker3 is running
214 worker2 is running
215 worker2 is running
216 worker2 is running
217 worker2 is running
218 worker4 is running
219 worker4 is running
220 worker4 is running
221 worker4 is running
222 worker2 is running
223 worker4 is running
224 worker3 is running
225 worker4 is running
226 worker4 is running
227 worker4 is running
228 worker4 is running
229 worker4 is running
230 worker4 is running
231 worker3 is running
232 worker4 is running
233 worker4 is running
234 worker4 is running
235 worker1 is running
236 worker3 is running
237 worker3 is running
238 worker3 is running
239 worker1 is running
240 worker1 is running
241 worker1 is running
242 worker3 is running
243 worker3 is running
244 worker3 is running
245 worker3 is running
246 worker3 is running
247 worker3 is running
248 worker3 is running
249 worker2 is running
250 worker3 is running
251 worker3 is running
252 worker3 is running
253 worker3 is running
254 worker1 is running
255 worker1 is running
256 worker1 is running
257 worker3 is running
258 worker3 is running
259 worker3 is running
260 worker1 is running
261 worker1 is running
262 worker1 is running
263 worker1 is running
264 worker1 is running
265 worker1 is running
266 worker3 is running
267 worker3 is running
268 worker3 is running
269 worker1 is running
270 worker1 is running
271 worker1 is running
272 worker3 is running
273 worker3 is running
274 worker3 is running
275 worker3 is running
276 worker3 is running
277 worker3 is running
278 worker1 is running
279 worker1 is running
280 worker1 is running
281 worker1 is running
282 worker4 is running
283 worker2 is running
284 worker2 is running
285 worker3 is running
286 worker3 is running
287 worker3 is running
288 worker3 is running
289 worker3 is running
290 worker3 is running
291 worker3 is running
292 worker4 is running
293 worker4 is running
294 worker4 is running
295 worker4 is running
296 worker4 is running
297 worker4 is running
298 worker4 is running
299 worker4 is running
300 worker2 is running
301 worker3 is running
302 worker3 is running
303 worker2 is running
304 worker3 is running
305 worker3 is running
306 worker3 is running
307 worker3 is running
308 worker2 is running
309 worker3 is running
310 worker2 is running
311 worker3 is running
312 worker3 is running
313 worker3 is running
314 worker2 is running
315 worker2 is running
316 worker2 is running
317 worker2 is running
318 worker2 is running
319 worker2 is running
320 worker2 is running
321 worker2 is running
322 worker2 is running
323 worker2 is running
324 worker3 is running
325 worker2 is running
326 worker2 is running
327 worker2 is running
328 worker2 is running
329 worker2 is running
330 worker2 is running
331 worker2 is running
332 worker2 is running
333 worker2 is running
334 worker2 is running
335 worker2 is running
336 worker2 is running
337 worker2 is running
338 worker2 is running
339 worker2 is running
340 worker2 is running
341 worker2 is running
342 worker2 is running
343 worker2 is running
344 worker4 is running
345 worker4 is running
346 worker4 is running
347 worker4 is running
348 worker4 is running
349 worker4 is running
350 worker4 is running
351 worker4 is running
352 worker4 is running
353 worker4 is running
354 worker4 is running
355 worker4 is running
356 worker4 is running
357 worker4 is running
358 worker4 is running
359 worker4 is running
360 worker3 is running
361 worker3 is running
362 worker3 is running
363 worker3 is running
364 worker3 is running
365 worker3 is running
366 worker3 is running
367 worker3 is running
368 worker3 is running
369 worker3 is running
370 worker3 is running
371 worker3 is running
372 worker3 is running
373 worker3 is running
374 worker3 is running
375 worker3 is running
376 worker3 is running
377 worker3 is running
378 worker3 is running
379 worker3 is running
380 worker3 is running
381 worker3 is running
382 worker3 is running
383 worker3 is running
384 worker3 is running
385 worker3 is running
386 worker3 is running
387 worker3 is running
388 worker3 is running
389 worker3 is running
390 worker3 is running
391 worker3 is running
392 worker3 is running
393 worker3 is running
394 worker3 is running
395 worker3 is running
396 worker3 is running
397 worker3 is running
398 worker3 is running
399 worker3 is running
400 worker3 is running
401 worker3 is running
402 
403 Process finished with exit code 0
View Code

 

　　　　测试：使用logging 日志输出实现线程安全：

 1 import threading
 2 import logging
 3 import time
 4 
 5 def work():
 6     for x in range(100):
 7         logging.warning('{} is running'.format(threading.current_thread().name))
 8 
 9 for x in range(1, 5):
10     name = 'worker{}'.format(x)
11     t = threading.Thread(name=name, target=work)
12     t.start()
使用logging模块

　　　　结果：可以看到没有出现不确定结果，但是是stderr ，所以 日志处理模块是线程安全的。

　　

14、daemon 线程 和non-daemon线程

　　注意：这里的daemon 不是Linux中的守护进程。

　　进程靠线程执行代码，至少有一个主线程，其他线程是工作线程。

　　主线程是第一个启动的线程，作为程序的入口

　　父线程：如果线程A 中启动了一个线程B，A 就是B 的父线程

　　子线程：B就是A 的子线程。

　　Python中，构造线程的时候，可以设置daemon属性，这个属性必须在start方法前设置好。　

 1  def start(self):
 2         """Start the thread's activity.
 3 
 4         It must be called at most once per thread object. It arranges for the
 5         object's run() method to be invoked in a separate thread of control.
 6 
 7         This method will raise a RuntimeError if called more than once on the
 8         same thread object.
 9 
10         """
11         if not self._initialized:
12             raise RuntimeError("thread.__init__() not called")
13 
14         if self._started.is_set():
15             raise RuntimeError("threads can only be started once")
16         with _active_limbo_lock:
17             _limbo[self] = self
18         try:
19             _start_new_thread(self._bootstrap, ())
20         except Exception:
21             with _active_limbo_lock:
22                 del _limbo[self]
23             raise
24         self._started.wait()
25 
26     def run(self):
27         """Method representing the thread's activity.
28 
29         You may override this method in a subclass. The standard run() method
30         invokes the callable object passed to the object's constructor as the
31         target argument, if any, with sequential and keyword arguments taken
32         from the args and kwargs arguments, respectively.
33 
34         """
35         try:
36             if self._target:
37                 self._target(*self._args, **self._kwargs)
38         finally:
39             # Avoid a refcycle if the thread is running a function with
40             # an argument that has a member that points to the thread.
41             del self._target, self._args, self._kwargs
start和run原码

 

　　Thread的部分原码：

　　

　　从原码看出 daemon属性 是keyword-only ，其次 如果daemon是None 使用当前父线程的daemon，因为这个虽然说current_thread(),但是此时线程没有start，所以这的当前线程就是父线程。

　　如果设定了，就是用户的设置。

　　主线程是non-daemon线程，即daemon = False 也就是None，也就是 默认值。

　　测试：non-daemon:

 1 import threading
 2 import time
 3 
 4 def foo():
 5     time.sleep(5)
 6     for i in range(20):
 7         print(i)
 8 # 主线程是non-daemon线程
 9 # t = threading.Thread(target=foo, daemon=False)
10 # t = threading.Thread(target=foo, daemon=None)
11 t = threading.Thread(target=foo)
12 t.start()
13 
14 print('main thread exit')
non-daemon

 

　　结果：三者结果一样：但是注意None是看情况的，看Thread 原码

 1 main thread exit
 2 0
 3 1
 4 2
 5 3
 6 4
 7 5
 8 6
 9 7
10 8
11 9
12 10
13 11
14 12
15 13
16 14
17 15
18 16
19 17
20 18
21 19
View Code

 　　测试：daemon = True

 1 import threading
 2 import time
 3 
 4 def foo():
 5     time.sleep(5)
 6     for i in range(20):
 7         print(i)
 8 # 主线程是non-daemon线程
 9 t = threading.Thread(target=foo, daemon=True)
10 t.start()
11 
12 print('main thread exit')
daemon=True

 

　　结果：

1 main thread exit
2 
3 Process finished with exit code 0
View Code

 

　　总结：

　　　　线程 t 依然执行，主线程 已经执行完，但一直等着线程 t。

　　　　如果daemon=True，主线程结束，程序立即结束，根本没有等线程 t

名称            含义
daemon属性 表示线程是否是daemon线程，这个值必须在start()之前设置，否则引发RuntimeError异常
isDaaemon() 是否是daemon线程
setDaemon 设置为daemon线程，必须在start方法之前设置

　　

　　总结：

　　　　线程具有一个daemon属性，可以显示这只为True或False，也可以不设置，则取默认值None

　　　　如果不设置daemon， 就取当前线程的daemon来设置它

　　　　主线程是non-daemon 线程，即daemon=False

　　　　从主线程创建的所有线程不设置daemon属性，则默认都是daemon= False，也就是non-daemon线程。

　　　　Python程序在没有活着的non-daemon线程运行时退出，也就是剩下的只能是daemon线程，主线程才能退出，否则主线程只能等待。

　　测试1:

　　

　　测试2 ：

　　

　　测试 3：

　　

　　通过实验说明:

　　　　　　如果有non-daemon线程的时候，主线程退出时，也不会杀掉daemon线程，知道所有的non-daemon线程全部结束，如果还有daemon线程，主线程需要退出，会结束所有的daemon线程，退出。也就是说，主线程退出时，有non-daemon就等，没有就退出。看下面的测试。

　　测试：

　　

　　测试：

 1 import time
 2 import threading
 3 
 4 def work():
 5     time.sleep(10)
 6     print('worK')
 7 
 8 def bar():
 9     threading.Thread(target=work, daemon=False).start()
10     time.sleep(5)
11     print('bar')
12 
13 def foo():
14     time.sleep(2)
15     print('foo')
16 
17 t = threading.Thread(target=foo, daemon=False, name='outer1')
18 t.start()
19 
20 for _ in range(10):
21     t1 = threading.Thread(name='work', target=bar, daemon=True)
22     t1.start()
23 
24 print('main thread exit')
比较复杂的

 

　　结果：　

 1 main thread exit
 2 foo
 3 bar
 4 barbar
 5 bar
 6 bar
 7 bar
 8 
 9 barbar
10 
11 bar
12 bar
13 worKworK
14 worKworK
15 worK
16 worK
17 
18 
19 worKworKworK
20 
21 worK
View Code

 

　　反正记住一条：主线程结束的时候，没有non-daemon 就结束程序。

 

15、join方法：

　　测试 1 加入join 和没有join比较

 1 import time
 2 import threading
 3 
 4 def foo(n):
 5     for i in range(n):
 6         print(i)
 7         time.sleep(1)
 8 
 9 t1 = threading.Thread(target=foo, args=(10,), daemon=True)
10 t1.start()
11 t1.join()
12 
13 print('main thread exit')
View Code

　　结果：　　

1 没有 join
2 
3 主线程结束，直接程序结束
4 
5 有join
6 
7 主线程等工作线程结束，在退出
View Code

 

 　 测试 2：

　　

　　测试 3：

　　

　　测试 4：

　　

　　测试 5：

　　

　　总结：

　　　　使用了join 方法后，daemon 线程执行完了，主线程才退出了。

　　　　join(timeout=None)， 是线程的标准方法之一

　　　　一个线程中调用另一个线程的join方法，调用者将被阻塞，知道被调用线程终止。

　　　　一个线程可以被join多次。

　　　　timeout 参数指定调用者等待多久，没有设置超时，就一直等待到被调用线程结束。

　　　　调用谁的join 方法，就是join 谁，就要等谁。

 

　　daemon线程应用场景：

　　　　简单来说就是，本来并没有daemon thread，为了简化程序员的工作，让他们不用去记录和管理那些后台线程，创造了一个daemon thread的概念，这个概念唯一的作用就是，当你把一个线程设置为daemon，它会随主线程的退出而退出，

　　　　主要应用场景有：

    1. 后台任务，如发送心跳包，监控，这种场景最多。
    2. 主线程工作才有用的线程，如 主线程中维护这公共的资源，主线程已经清理了，准备退出，而工作线程使用这些资源工作也没有意义了，一起退出最合适
    3. 随时可以被终止的线程。

　　　　如果主线程退出，想所有其他工作线程一起退出，就使用daemon=True来创建工作线程。

　　　　

　　　　

 　　　　注：

　　　　　　主线程M退出，进程就退出，子线程A创建的子线程B不受A影响

 　　　　　   主线程是non-daemon的

 

 

16、threading.local类

　　测试1：局部变量x

　　

 　　测试 2：全局变量 x

　　

 　　测试 3：全局变量 x 

 　　

 　　测试 全局变量 类实例对象的属性

 

　　　测试：python提供了一个threading.local类，将这个类实例化得到一个全局对象，但是不同的线程使用这个对象存储的数据其他线程看不见。

　　

　　　　和使用局部变量样的结果！

　　测试：　

　　　　测试：报错的时候：

　　　

　　总结：

　　　　从运行结果来看，另起一个线程打印ctx.x出错了，但是ctx 打印 没有出错，说明看到ctx，但是ctx中的x看不到，这个x不能夸线程。　　

　　　　threading.local 类构建了一个大字典，存放所有线程相关的字典，定义如下：（源码）　　　

{ id(Thread) -> (ref(Thread), thread-local dict) }

 

　　　　每一个线程实例的id 为key，元组为vlaue

　　　　value中2 部分，线程对象引用，每个线程自己的字典。

　　　　本质：

　　　　　　运行时，threading.local实例处在不同的线程中，就从大字典中找到了当前线程相关键值对中的字典，覆盖threading，.local实例的__dict__

　　　　　　这样 就可以在不同的线程中，安全的使用线程独有的数据，做到了线程间数据隔离，如同本地变量一样安全。

 

 17、定时器 Timer / 延迟执行

 1 class Timer(Thread):
 2     """Call a function after a specified number of seconds:
 3 
 4             t = Timer(30.0, f, args=None, kwargs=None)
 5             t.start()
 6             t.cancel()     # stop the timer's action if it's still waiting
 7 
 8     """
 9 
10     def __init__(self, interval, function, args=None, kwargs=None):
11         Thread.__init__(self)
12         self.interval = interval
13         self.function = function
14         self.args = args if args is not None else []
15         self.kwargs = kwargs if kwargs is not None else {}
16         self.finished = Event()
17 
18     def cancel(self):
19         """Stop the timer if it hasn't finished yet."""
20         self.finished.set()
21 
22     def run(self):
23         self.finished.wait(self.interval)
24         if not self.finished.is_set():
25             self.function(*self.args, **self.kwargs)
26         self.finished.set()
threading.Timer原码

　　threading.Timer 继承Thread， 这个类用来定义延迟多久后执行一个函数。

　　

　　start方法执行之后，Timer对象会处于等待状态，等待interval秒后，开始执行function函数

　　测试:

　　

 　　测试 cancel 1：

 　　

 　　测试 cancel 2：

 　　

 　　Timer提供了 cancel方法，用来取消 一个未执行的函数，如果上面的例子中worker函数已经开始执行，cancel就没有任何效果了。

 　　如：等待时间为0，线程刚开始就执行函数了，但是在执行cancle就没有取消效果了。

　　

 

 　　总结:

　　　　Timer是线程Thread的子类，就是线程类，具有线程的能力和特征。

　　　　它的实例是能够延时执行目标函数的线程，在真正执行目标函数之前，都是可以cancel他的

　　　　cancel方法本质使用Event 类 实现，并不是线程提供了取消方法。

 

 

 

 

{ id(Thread) -> (ref(Thread), thread-local dict) }
posted @ 2018-10-09 22:25  JerryZao  阅读(532)  评论(0)  编辑  收藏  举报