多线程修改同一个数据
个人理解:
GIL:存在于Cpython中,称为全局解释器锁,在同一时间只能一个python线程在跑,但这并不是说是串行运行的,他还是“并行”的,CPU在不断的分配cpu时间给每个线程去运行,只是同一时间片刻或同一个cpu时间片刻只有一个线程在跑。
线程锁:只让一个线程运行加锁的那段代码。
示例:
1 print(n) 2 n += 1 3 print(n) 4 # 这里三条语句需要三次获取n的值,第一次print(n)和 n += 1时拿到的n 不一定相同,第二次print(n)也不一定是 n += 1的结果。 但 n += 1的操作是原子性的,执行n += 1过程中,从拿到n 再 n + 1再重新赋值给n的这个过程中,其他线程不能修改,且也不能访问n的值。
简单示例1:
import threading import time def f(i): global n time.sleep(1) print("begin += 1: i=%s,n=%s"%(i,n)) n += 1 print("end +=1: i=%s,n=%s"%(i,n)) if __name__ == "__main__": t_list = [] n = 0 for i in range(100): t = threading.Thread(target=f,args=(i,)) t.start() t_list.append(t) for i in t_list: i.join()
执行结果:
begin += 1: i=2,n=0 end +=1: i=2,n=1 begin += 1: i=1,n=1 end +=1: i=1,n=2 begin += 1: i=0,n=2 end +=1: i=0,n=3 begin += 1: i=4,n=3 end +=1: i=4,n=4 begin += 1: i=3,n=4 end +=1: i=3,n=5 begin += 1: i=6,n=5 end +=1: i=6,n=6 begin += 1: i=5,n=6 end +=1: i=5,n=7 begin += 1: i=9,n=7 begin += 1: i=8,n=7 begin += 1: i=7,n=7 end +=1: i=7,n=8 end +=1: i=8,n=9 end +=1: i=9,n=10 begin += 1: i=13,n=10 end +=1: i=13,n=11 begin += 1: i=15,n=11 begin += 1: i=12,n=11 begin += 1: i=11,n=11 end +=1: i=12,n=12 begin += 1: i=14,n=12 end +=1: i=14,n=13 begin += 1: i=10,n=13 end +=1: i=11,n=14 begin += 1: i=16,n=14 end +=1: i=16,n=15 end +=1: i=10,n=16 end +=1: i=15,n=17 begin += 1: i=19,n=17 end +=1: i=19,n=18 begin += 1: i=18,n=18 begin += 1: i=17,n=18 end +=1: i=18,n=19 end +=1: i=17,n=20 begin += 1: i=21,n=20 end +=1: i=21,n=21 begin += 1: i=22,n=20 end +=1: i=22,n=22 begin += 1: i=20,n=22 end +=1: i=20,n=23 begin += 1: i=24,n=23 end +=1: i=24,n=24 begin += 1: i=23,n=24 end +=1: i=23,n=25 begin += 1: i=25,n=24 end +=1: i=25,n=26 begin += 1: i=28,n=26 end +=1: i=28,n=27 begin += 1: i=27,n=27 begin += 1: i=26,n=27 end +=1: i=26,n=28 end +=1: i=27,n=29 begin += 1: i=29,n=29 end +=1: i=29,n=30 begin += 1: i=37,n=30 begin += 1: i=36,n=30 end +=1: i=36,n=31 begin += 1: i=34,n=31 end +=1: i=34,n=32 begin += 1: i=33,n=32 end +=1: i=33,n=33 begin += 1: i=31,n=33 end +=1: i=31,n=34 begin += 1: i=30,n=34 begin += 1: i=35,n=34 end +=1: i=35,n=35 end +=1: i=30,n=36 end +=1: i=37,n=37 begin += 1: i=40,n=37 end +=1: i=40,n=38 begin += 1: i=32,n=38 end +=1: i=32,n=39 begin += 1: i=39,n=39 end +=1: i=39,n=40 begin += 1: i=38,n=40 end +=1: i=38,n=41 begin += 1: i=42,n=41 begin += 1: i=41,n=42 end +=1: i=41,n=43 end +=1: i=42,n=42 begin += 1: i=43,n=43 end +=1: i=43,n=44 begin += 1: i=46,n=44 end +=1: i=46,n=45 begin += 1: i=45,n=45 end +=1: i=45,n=46 begin += 1: i=44,n=46 end +=1: i=44,n=47 begin += 1: i=48,n=47 end +=1: i=48,n=48 begin += 1: i=47,n=48 end +=1: i=47,n=49 begin += 1: i=50,n=49 end +=1: i=50,n=50 begin += 1: i=51,n=50 end +=1: i=51,n=51 begin += 1: i=49,n=51 end +=1: i=49,n=52 begin += 1: i=53,n=52 begin += 1: i=52,n=52 end +=1: i=53,n=53 end +=1: i=52,n=54 begin += 1: i=54,n=54 end +=1: i=54,n=55 begin += 1: i=55,n=55 end +=1: i=55,n=56 begin += 1: i=56,n=56 end +=1: i=56,n=57 begin += 1: i=57,n=57 end +=1: i=57,n=58 begin += 1: i=59,n=58 end +=1: i=59,n=59 begin += 1: i=58,n=59 end +=1: i=58,n=60 begin += 1: i=60,n=60 end +=1: i=60,n=61 begin += 1: i=62,n=61 end +=1: i=62,n=62 begin += 1: i=61,n=62 end +=1: i=61,n=63 begin += 1: i=65,n=63 begin += 1: i=64,n=63 end +=1: i=64,n=64 end +=1: i=65,n=65 begin += 1: i=63,n=65 end +=1: i=63,n=66 begin += 1: i=67,n=66 end +=1: i=67,n=67 begin += 1: i=68,n=67 end +=1: i=68,n=68 begin += 1: i=66,n=68 end +=1: i=66,n=69 begin += 1: i=70,n=69 end +=1: i=70,n=70 begin += 1: i=69,n=70 end +=1: i=69,n=71 begin += 1: i=73,n=71 begin += 1: i=72,n=71 end +=1: i=72,n=72 end +=1: i=73,n=73 begin += 1: i=71,n=72 end +=1: i=71,n=74 begin += 1: i=74,n=74 end +=1: i=74,n=75 begin += 1: i=75,n=75 end +=1: i=75,n=76 begin += 1: i=76,n=76 end +=1: i=76,n=77 begin += 1: i=77,n=77 end +=1: i=77,n=78 begin += 1: i=78,n=78 end +=1: i=78,n=79 begin += 1: i=79,n=79 begin += 1: i=80,n=79 end +=1: i=80,n=80 end +=1: i=79,n=81 begin += 1: i=82,n=81 end +=1: i=82,n=82 begin += 1: i=81,n=82 end +=1: i=81,n=83 begin += 1: i=84,n=83 begin += 1: i=85,n=83 end +=1: i=85,n=84 end +=1: i=84,n=85 begin += 1: i=83,n=85 end +=1: i=83,n=86 begin += 1: i=86,n=86 end +=1: i=86,n=87 begin += 1: i=87,n=87 begin += 1: i=88,n=87 end +=1: i=87,n=88 end +=1: i=88,n=89 begin += 1: i=90,n=89 end +=1: i=90,n=90 begin += 1: i=89,n=90 end +=1: i=89,n=91 begin += 1: i=93,n=91 end +=1: i=93,n=92 begin += 1: i=91,n=92 end +=1: i=91,n=93 begin += 1: i=92,n=93 end +=1: i=92,n=94 begin += 1: i=94,n=94 end +=1: i=94,n=95 begin += 1: i=95,n=95 end +=1: i=95,n=96 begin += 1: i=96,n=96 begin += 1: i=97,n=96 end +=1: i=97,n=97 end +=1: i=96,n=98 begin += 1: i=99,n=98 begin += 1: i=98,n=98 end +=1: i=98,n=99 end +=1: i=99,n=100
这里可以看到第99次中第一次print(n)获取的值为98,第二次print(n)的值为100。说明执行n += 1时(即n = n + 1)右边的n的值不一定是98。
简单示例2:
import threading import time def f(i): global n time.sleep(1) print("begin += 1: i=%s,n=%s,d=%s"%(i,n,d)) d[n] ,n = n+1,n+1 print("end +=1: i=%s,n=%s,d=%s"%(i,n,d)) if __name__ == "__main__": t_list = [] n = 0 d = {} for i in range(10): t = threading.Thread(target=f,args=(i,)) t.start() t_list.append(t) for i in t_list: i.join()
运行结果:
begin += 1: i=0,n=0,d={} begin += 1: i=1,n=0,d={} end +=1: i=1,n=1,d={0: 1} end +=1: i=0,n=2,d={0: 1, 1: 2} begin += 1: i=5,n=2,d={0: 1, 1: 2} begin += 1: i=4,n=2,d={0: 1, 1: 2} end +=1: i=4,n=3,d={0: 1, 1: 2, 2: 3} end +=1: i=5,n=4,d={0: 1, 1: 2, 2: 3, 3: 4} begin += 1: i=2,n=4,d={0: 1, 1: 2, 2: 3, 3: 4} end +=1: i=2,n=5,d={0: 1, 1: 2, 2: 3, 3: 4, 4: 5} begin += 1: i=3,n=5,d={0: 1, 1: 2, 2: 3, 3: 4, 4: 5} end +=1: i=3,n=6,d={0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6} begin += 1: i=6,n=6,d={0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6} end +=1: i=6,n=7,d={0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7} begin += 1: i=9,n=7,d={0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7} begin += 1: i=8,n=7,d={0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7} end +=1: i=8,n=8,d={0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7, 7: 8} begin += 1: i=7,n=8,d={0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7, 7: 8} end +=1: i=7,n=9,d={0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7, 7: 8, 8: 9} end +=1: i=9,n=10,d={0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7, 7: 8, 8: 9, 9: 10}
这里可以看到d[n], n = n+1, n+1是原子性的,这里先是d[n]和右边的两个n获取值,然后分别赋值给d[n]和n,这里不可能出现d[n]和右边的两个n获取的值是8,而在赋值d[n]和n时赋的值是10.
python2.7:
不加线程锁时,n += 1操作时,多个线程能同时访问n获取n的值,这样就会导致多个线程运行n += 1后n的值并不是 n + 1的次数。
import threading import time def f(i): global n time.sleep(1) # print("begin += 1: i=%s,n=%s"%(i,n)) d[i] , n = "%s:%s"%(n,n +1),n+1 # print("end +=1: i=%s,n=%s"%(i,n)) if __name__ == "__main__": t_list = [] n = 0 d= {} for i in range(100): t = threading.Thread(target=f,args=(i,)) t.start() t_list.append(t) for i in t_list: i.join() time.sleep(2) for i,v in d.items(): print(i,v) print("n=",n)
运行结果:
(0, '1:2') (1, '0:1') (2, '3:4') (3, '2:3') (4, '5:6') (5, '4:5') (6, '7:8') (7, '6:7') (8, '6:7') (9, '8:9') (10, '7:8') (11, '10:11') (12, '9:10') (13, '11:12') (14, '12:13') (15, '13:14') (16, '15:16') (17, '14:15') (18, '17:18') (19, '16:17') (20, '19:20') (21, '18:19') (22, '21:22') (23, '20:21') (24, '22:23') (25, '23:24') (26, '26:27') (27, '25:26') (28, '24:25') (29, '27:28') (30, '27:28') (31, '31:32') (32, '30:31') (33, '29:30') (34, '32:33') (35, '34:35') (36, '33:34') (37, '36:37') (38, '35:36') (39, '38:39') (40, '37:38') (41, '40:41') (42, '40:41') (43, '39:40') (44, '43:44') (45, '42:43') (46, '41:42') (47, '44:45') (48, '46:47') (49, '45:46') (50, '47:48') (51, '49:50') (52, '48:49') (53, '50:51') (54, '51:52') (55, '52:53') (56, '54:55') (57, '53:54') (58, '56:57') (59, '55:56') (60, '57:58') (61, '57:58') (62, '60:61') (63, '59:60') (64, '61:62') (65, '62:63') (66, '64:65') (67, '65:66') (68, '63:64') (69, '72:73') (70, '71:72') (71, '73:74') (72, '70:71') (73, '69:70') (74, '66:67') (75, '68:69') (76, '67:68') (77, '75:76') (78, '74:75') (79, '76:77') (80, '78:79') (81, '77:78') (82, '80:81') (83, '79:80') (84, '82:83') (85, '81:82') (86, '85:86') (87, '83:84') (88, '84:85') (89, '87:88') (90, '86:87') (91, '89:90') (92, '88:89') (93, '92:93') (94, '91:92') (95, '90:91') (96, '94:95') (97, '93:94') (98, '96:97') (99, '95:96') ('n=', 97)
这里可以看到线程7和8,29和30,60和61分别获取了同一份n的值。
加锁代码示例,加锁代码段可以理解为串行执行:
import threading import time def f(i): global n time.sleep(1) # print("begin += 1: i=%s,n=%s"%(i,n)) lock.acquire() d[i] , n = "%s:%s"%(n,n +1),n+1 lock.release() # print("end +=1: i=%s,n=%s"%(i,n)) if __name__ == "__main__": t_list = [] n = 0 d= {} lock= threading.Lock() for i in range(100): t = threading.Thread(target=f,args=(i,)) t.start() t_list.append(t) for i in t_list: i.join() time.sleep(2) for i,v in d.items(): print(i,v) print("n=",n)
如果time.sleep(1)被加锁,由于加锁代码段串行执行,那么这个sleep(x)也是串行的。
lock.acquire() time.sleep(0.1) d[i] , n = "%s:%s"%(n,n +1),n+1 lock.release()
递归锁,存在函数之间调用,而函数中又都有锁时用递归锁:
RLock
import threading import time def a(i): global data lock.acquire() data += 2 print("a,%s,%s"%(i,data)) lock.release() def b(i): global data lock.acquire() data *= 2 print("b,%s,%s\n"%(i,data)) lock.release() def c(i): time.sleep(1) lock.acquire() print("c,%s,%s"%(i,data)) a(i) print("----------a,b---------------") b(i) lock.release() if __name__ == "__main__": lock = threading.RLock() data = 100 t_list = [] for i in range(5): t = threading.Thread(target=c,args=(i,)) t.start() t_list.append(t) for t in t_list: t.join() print("main:",data)
运行结果:
c,1,100 a,1,102 ----------a,b--------------- b,1,204 c,0,204 a,0,206 ----------a,b--------------- b,0,412 c,3,412 a,3,414 ----------a,b--------------- b,3,828 c,4,828 a,4,830 ----------a,b--------------- b,4,1660 c,2,1660 a,2,1662 ----------a,b--------------- b,2,3324 main: 3324
可以看到被锁住的代码串行运行。