Python-Basis-17th
周日,晴,记录生活分享点滴
参考博客:https://www.cnblogs.com/yuanchenqi/articles/5692716.html
Python版本:3.5
网络通讯要素
IP地址
(1) 用来标识网络上一台独立的主机
(2) IP地址 = 网络地址 + 主机地址(网络号:用于识别主机所在的网络/网段。主机号:用于识别该网络中的主机)
(3) 特殊的IP地址:127.0.0.1(本地回环地址、保留地址,点分十进制)可用于简单的测试网卡是否故障。表示本机。
端口号
(1) 用于标识进程的逻辑地址。不同的进程都有不同的端口标识。
(2) 端口:要将数据发送到对方指定的应用程序上,为了标识这些应用程序,所以给这些网络应用程序都用数字进行标识。为了方便称呼这些数字,则将这些数字称为端口。(此端口是一个逻辑端口
传输协议
通讯的规则。例如:TCP、UDP协议(好比两个人得用同一种语言进行交流)
UDP
User Datagram Protocol用户数据报协议
特点:
- 面向无连接:传输数据之前源端和目的端不需要建立连接。
- 每个数据报的大小都限制在64K(8个字节)以内。
- 面向报文的不可靠协议。(即:发送出去的数据不一定会接收得到)
- 传输速率快,效率高。
- 现实生活实例:邮局寄件、实时在线聊天、视频会议…等。
TCP
Transmission Control Protocol传输控制协议
特点:
- 面向连接:传输数据之前需要建立连接。
- 在连接过程中进行大量数据传输。
- 通过“三次握手”的方式完成连接,是安全可靠协议。
- 传输速度慢,效率低。
注意:在TCP/IP协议中,TCP协议通过三次握手建立一个可靠的连接
网络通讯步骤
socket 编程
应用程序两端通过“套接字”向网络发出请求或者应答网络请求。(应用程序两端值服务端与客户端;套接字socket指客户端的套接字socket)
参数
family
family = AF_INET 服务器之间的通信(默认)
family = AF_INET6 服务器之间的通信
family = AF_UNIX Unix不同进程间通信
type
SOCK_STREAM 通过TCP传输控制协议(默认)
SOCK_DGRAM 通过UDP用户数据报协议
相关方法及参数介绍
sk.bind(address) #s.bind(address) 将套接字绑定到地址。address地址的格式取决于地址族。在AF_INET下,以元组(host,port)的形式表示地址。 sk.listen(backlog) #开始监听传入连接。backlog指定在拒绝连接之前,可以挂起的最大连接数量。 #backlog等于5,表示内核已经接到了连接请求,但服务器还没有调用accept进行处理的连接个数最大为5 #这个值不能无限大,因为要在内核中维护连接队列 sk.setblocking(bool) #是否阻塞(默认True),如果设置False,那么accept和recv时一旦无数据,则报错。 sk.accept() #接受连接并返回(conn,address),其中conn是新的套接字对象,可以用来接收和发送数据。address是连接客户端的地址。 #接收TCP 客户的连接(阻塞式)等待连接的到来 sk.connect(address) #连接到address处的套接字。一般,address的格式为元组(hostname,port),如果连接出错,返回socket.error错误。 sk.connect_ex(address) #同上,只不过会有返回值,连接成功时返回 0 ,连接失败时候返回编码,例如:10061 sk.close() #关闭套接字 sk.recv(bufsize[,flag]) #接受套接字的数据。数据以字符串形式返回,bufsize指定最多可以接收的数量。flag提供有关消息的其他信息,通常可以忽略。 sk.recvfrom(bufsize[.flag]) #与recv()类似,但返回值是(data,address)。其中data是包含接收数据的字符串,address是发送数据的套接字地址。 sk.send(string[,flag]) #将string中的数据发送到连接的套接字。返回值是要发送的字节数量,该数量可能小于string的字节大小。即:可能未将指定内容全部发送。 sk.sendall(string[,flag]) #将string中的数据发送到连接的套接字,但在返回之前会尝试发送所有数据。成功返回None,失败则抛出异常。 #内部通过递归调用send,将所有内容发送出去。 sk.sendto(string[,flag],address) #将数据发送到套接字,address是形式为(ipaddr,port)的元组,指定远程地址。返回值是发送的字节数。该函数主要用于UDP协议。 sk.settimeout(timeout) #设置套接字操作的超时期,timeout是一个浮点数,单位是秒。值为None表示没有超时期。一般,超时期应该在刚创建套接字时设置,因为它们可能用于连接的操作(如 client 连接最多等待5s ) sk.getpeername() #返回连接套接字的远程地址。返回值通常是元组(ipaddr,port)。 sk.getsockname() #返回套接字自己的地址。通常是一个元组(ipaddr,port) sk.fileno() #套接字的文件描述符
流程
流程图
流程描述
# 流程描述: # # 1 服务器根据地址类型(ipv4,ipv6)、socket类型、协议创建socket # # 2 服务器为socket绑定ip地址和端口号 # # 3 服务器socket监听端口号请求,随时准备接收客户端发来的连接,这时候服务器的socket并没有被打开 # # 4 客户端创建socket # # 5 客户端打开socket,根据服务器ip地址和端口号试图连接服务器socket # # 6 服务器socket接收到客户端socket请求,被动打开,开始接收客户端请求,直到客户端返回连接信息。这时候socket进入阻塞状态,所谓阻塞即accept()方法一直等到客户端返回连接信息后才返回,开始接收下一个客户端连接请求 # # 7 客户端连接成功,向服务器发送连接状态信息 # # 8 服务器accept方法返回,连接成功 # # 9 客户端向socket写入信息(或服务端向socket写入信息) # # 10 服务器读取信息(客户端读取信息) # # 11 客户端关闭 # # 12 服务器端关闭
应用
一次传输
#################server import socket ip_port = ('127.0.0.1',9997) sk = socket.socket() sk.bind(ip_port) sk.listen(5) print ('server waiting...') conn,addr = sk.accept() client_data = conn.recv(1024) print (str(client_data,"utf8")) conn.sendall(bytes('不在!',encoding="utf-8")) sk.close() ################client import socket ip_port = ('127.0.0.1',9997) sk = socket.socket() sk.connect(ip_port) sk.sendall(bytes('在吗?',encoding="utf8")) server_reply = sk.recv(1024) print (str(server_reply,"utf8"))
#-------------------------------------------------server.py #------------------------------------------------- import socket ip_port = ('127.0.0.1',8888) sk = socket.socket() sk.bind(ip_port) sk.listen(2) print ("服务端启动...") conn,address = sk.accept() while True: client_data=conn.recv(1024) # 1.1接收 if str(client_data,"utf8")=='exit': break print (str(client_data,"utf8")) # 1.2打印 server_response=input(">>>") # 2.1输入 conn.sendall(bytes(server_response,"utf8")) # 2.2发送 conn.close() #-------------------------------------------------client.py #------------------------------------------------- import socket ip_port = ('127.0.0.1',8888) sk = socket.socket() sk.connect(ip_port) print ("客户端启动:") while True: inp = input('>>>') # 1.1打印 sk.sendall(bytes(inp,"utf8")) # 1.2发送 if inp == 'exit': break server_response=sk.recv(1024) # 2.1接收 print (str(server_response,"utf8")) # 2.2打印 sk.close()
不间断传输和退出处理
服务端 server【Windows】
# server.py 服务端 【Windows】 import socket ip_port = ('127.0.0.1',8870) sk = socket.socket() sk.bind(ip_port) sk.listen(2) # 排队数 print ("服务端启动...") while True: conn,address = sk.accept() print(address) while True: try: client_data=conn.recv(1024) except: print("意外中断") break print (str(client_data,"utf8")) server_response=input(">>>") conn.sendall(bytes(server_response,"utf8")) conn.close()
服务端 server【Linux】
# serverf.py 服务端 【Linux】 import socket ip_port = ('127.0.0.1',8870) sk = socket.socket() sk.bind(ip_port) sk.listen(2) # 排队数 print ("服务端启动...") while True: conn,address = sk.accept() print(address) while True: client_data=conn.recv(1024) print (str(client_data,"utf8")) if len(client_data)==0: # Windows中进行强行报错;Linux认为没有发送,默认为空,然后捕获空进行break print("意外中断") break server_response=input(">>>") conn.sendall(bytes(server_response,"utf8")) conn.close()
# client.py 客户端 import socket ip_port = ('127.0.0.1',8870) sk = socket.socket() sk.connect(ip_port) print ("客户端启动:") while True: inp = input('>>>') if inp == 'exit': break sk.sendall(bytes(inp,"utf8")) server_response=sk.recv(1024) print (str(server_response,"utf8")) sk.close()
server并发聊天
# server.py ----------------------------------------------------- import socketserver class MyServer(socketserver.BaseRequestHandler): # 1.1 自定义一个类MyServer,继承socketserver.BaseRequestHandler固定的类 def handle(self): # 1.2 重写handle方法。因为handle是父类的方法,目前需要重写父类,所以不能改名字 print ("服务端启动...") while True: conn = self.request # 2.1 通过self.request调用conn,同conn = sk.accept print (self.client_address) while True: client_data=conn.recv(1024) print (str(client_data,"utf8")) print ("waiting...") conn.sendall(client_data) conn.close() if __name__ == '__main__': server = socketserver.ThreadingTCPServer(('127.0.0.1',8091),MyServer) # 1.3 调用ThreadingTCPServer,创建server对象,目的是实现并发 server.serve_forever() # 1.4 启动,通过对象调用serve_forever方法来激活,执行的是handle方法 # client.py ----------------------------------------------------- import socket ip_port = ('127.0.0.1',8091) sk = socket.socket() sk.connect(ip_port) print ("客户端启动:") while True: inp = input('>>>') sk.sendall(bytes(inp,"utf8")) if inp == 'exit': break server_response=sk.recv(1024) print (str(server_response,"utf8")) sk.close()
# server.py ----------------------------------------------------- import socketserver class MyServer(socketserver.BaseRequestHandler): def handle(self): print ("服务端启动...") while True: conn = self.request print (self.client_address) while True: client_data=conn.recv(1024) print (str(client_data,"utf8")) print ("waiting...") server_response=input(">>>") # 与简单并发相比,只多了一个input conn.sendall(bytes(server_response,"utf8")) # conn.sendall(client_data) conn.close() # print self.request,self.client_address,self.server if __name__ == '__main__': server = socketserver.ThreadingTCPServer(('127.0.0.1',8098),MyServer) server.serve_forever() # client.py ----------------------------------------------------- import socket ip_port = ('127.0.0.1',8098) sk = socket.socket() sk.connect(ip_port) print ("客户端启动:") while True: inp = input('>>>') sk.sendall(bytes(inp,"utf8")) server_response=sk.recv(1024) print (str(server_response,"utf8")) if inp == 'exit': break sk.close()
# server ------------------------------------------------ import socket import subprocess ip_port = ('127.0.0.1',8879) sk = socket.socket() sk.bind(ip_port) sk.listen(5) print ("服务端启动...") while True: conn,address = sk.accept() while True: try: client_data=conn.recv(1024) except Exception: break print (str(client_data,"utf8")) print ("waiting...") # server_response=input(">>>") # conn.sendall(bytes(server_response,"utf8")) cmd=str(client_data,"utf8").strip() cmd_call=subprocess.Popen(cmd,shell=True,stdout=subprocess.PIPE) cmd_result=cmd_call.stdout.read() if len(cmd_result)==0: cmd_result=b"no output!" conn.sendall(cmd_result) print('send data size',len(cmd_result)) print('******************') print('******************') print('******************') conn.close() # client ------------------------------------------------ import socket ip_port = ('127.0.0.1',8879) sk = socket.socket() sk.connect(ip_port) print ("客户端启动:") while True: inp = input('cdm:>>>').strip( ) if len(inp)==0: continue if inp=="q": break sk.sendall(bytes(inp,"utf8")) server_response=sk.recv(1024) print (str(server_response,"gbk")) print('receive data size',len(server_response)) if inp == 'exit': break sk.close()
小结:
- sendall会把数据直接全部发送到客户端,
- 所以,并不能一次recv()无限大数据,所以这里应该通过循环去接收。 sk.recv(4096)
# server ------------------------------------------------ import socketserver import subprocess class Myserver(socketserver.BaseRequestHandler): def handle(self): while True: conn=self.request conn.sendall(bytes("欢迎登录","utf8")) while True: client_bytes=conn.recv(1024) if not client_bytes:break client_str=str(client_bytes,"utf8") print(client_str) command=client_str result_str=subprocess.getoutput(command) result_bytes = bytes(result_str,encoding='utf8') info_str="info|%d"%len(result_bytes) conn.sendall(bytes(info_str,"utf8")) # conn.recv(1024) # 当同时出现两个sendall时,可能会出现粘包现象,会报错,解决办法是在server添加conn.recv(1024),配合client中的sk.send(bytes('ok','utf8')) conn.sendall(result_bytes) conn.close() if __name__=="__main__": server=socketserver.ThreadingTCPServer(("127.0.0.1",9998),Myserver) server.serve_forever() # client ------------------------------------------------ import socket ip_port=("127.0.0.1",9998) sk=socket.socket() sk.connect(ip_port) print("客户端启动...") print(str(sk.recv(1024),"utf8")) while True: inp=input("please input:").strip() sk.sendall(bytes(inp,"utf8")) basic_info_bytes=sk.recv(1024) print(str(basic_info_bytes,"utf8")) # sk.send(bytes('ok','utf8')) # 配合server中的conn.recv(1024)解决粘包现象 result_length=int(str(basic_info_bytes,"utf8").split("|")[1]) print(result_length) has_received=0 content_bytes=bytes() while has_received != result_length: fetch_bytes=sk.recv(1024) has_received+=len(fetch_bytes) content_bytes+=fetch_bytes cmd_result=str(content_bytes,"utf8") print(cmd_result) sk.close()
编码拾遗(回顾)
数据类型
Python3中,只有两种数据类型:str、bytes
- str:unicode的形式叫str类型
- bytes:十六进制
示例
s = 'hello你好' # 注意:“hello你好”的数据类型是str,储存的是unicode的编码 print(type(s)) # <class 'str'>
# 编码第一种方式:通过内置函数bytes进行转换 b = bytes(s, 'utf8') # utf8是世界公认的规则 print(b) # b'hello\xe4\xbd\xa0\xe5\xa5\xbd' # utf8规则下的bytes类型,一个汉字占三个字节 # 编码第二种方式:通过内置函数s进行转换 b2 = s.encode('utf8') # 默认utf8,gbk同样可以 print(b2) # b'hello\xe4\xbd\xa0\xe5\xa5\xbd' # utf8规则下的bytes类型
# 解码第一种方式 s = str(b2, 'utf8') print(s) # hello你好 # str类型 # 解码第二种方式 b2.decode('utf8') print(s2) # hello你好
# server ------------------------------------------------ import socket,os ip_port=("127.0.0.1",8898) sk=socket.socket() sk.bind(ip_port) sk.listen(5) BASE_DIR=os.path.dirname(os.path.abspath(__file__)) while True: print("waiting connect") conn,addr=sk.accept() flag = True while flag: client_bytes=conn.recv(1024) ## 接收来自client的数据包 client_str=str(client_bytes,"utf8") ## 解析 func,file_byte_size,filename=client_str.split("|",2) ## 解析后的区分 path=os.path.join(BASE_DIR,'chung',filename) ## 路径 has_received=0 file_byte_size=int(file_byte_size) ## 转成int类型 f=open(path,"ab") #### 接收前需要打开文件 ab指以bytes类型写 while has_received<file_byte_size: #### client循环发送,所以server循环接收 data=conn.recv(1024) f.write(data) has_received+=len(data) print("ending") f.close() # client ------------------------------------------------ import socket import re,os,sys ip_port=("127.0.0.1",8898) sk=socket.socket() sk.connect(ip_port) BASE_DIR=os.path.dirname(os.path.abspath(__file__)) print("客户端启动....") while True: inp=input("please input:") if inp.startswith("post"): method,local_path=inp.split("|",1) # 解析 local_path=os.path.join(BASE_DIR,local_path) # 拼接 file_byte_size=os.stat(local_path).st_size # 通知server文件大小 file_name=os.path.basename(local_path) # 通过参数路径取到文件的名字 post_info="post|%s|%s"%(file_byte_size,file_name) # 通过字符串拼接成一个数据包 sk.sendall(bytes(post_info,"utf8")) # 将数据包发送到server has_sent=0 ### 指int数据类型 file_obj=open(local_path,"rb") ### 打开文件 rb指以bytes类型读,不加b指以字符串读或写 while has_sent != file_byte_size: ### 一段一段的循环发 data=file_obj.read(1024) sk.sendall(data) has_sent+=len(data) file_obj.close() print("上传成功")
小结:
- 纸条就是conn
- 一收一发
- client_data=conn.recv(1024)
SocketServer
socketserver模块可以简化网络服务器的编写
Python把网络服务抽象成两个主要的类:
- 一个是Server类,用于处理连接相关的网络操作,
- 另外一个则是RequestHandler类,用于处理数据相关的操作。
- 并且提供两个MixIn 类,用于扩展 Server,实现多进程或多线程。
Server类
五种server类
1. BaseServer(不直接对外服务)
Base class for server classes.
class BaseServer
2. TCPServer使用TCP协议
This uses the Internet TCP protocol, which provides for continuous streams of data between the client and server.
class socketserver.TCPServer(server_address, RequestHandlerClass, bind_and_activate=True)
3. UDPServer使用UDP协议
This uses datagrams, which are discrete packets of information that may arrive out of order or be lost while in transit. The parameters are the same as for TCPServer.
class socketserver.UDPServer(server_address, RequestHandlerClass, bind_and_activate=True)
4.(不常用)UnixStreamServer和UnixDatagramServer,这两个类只在unix环境下有用(AF_unix)
These more infrequently used classes are similar to the TCP and UDP classes, but use Unix domain sockets; they’re not available on non-Unix platforms. The parameters are the same as for TCPServer.
class socketserver.UnixStreamServer(server_address, RequestHandlerClass, bind_and_activate=True) class socketserver.UnixDatagramServer(server_address, RequestHandlerClass,bind_and_activate=True)<br><br>
class UnixStreamServer(TCPServer): address_family = socket.AF_UNIX class UnixDatagramServer(UDPServer): address_family = socket.AF_UNIX
BaseServer的源码:
class BaseServer: """Base class for server classes. Methods for the caller: - __init__(server_address, RequestHandlerClass) - serve_forever(poll_interval=0.5) - shutdown() - handle_request() # if you do not use serve_forever() - fileno() -> int # for select() Methods that may be overridden: - server_bind() - server_activate() - get_request() -> request, client_address - handle_timeout() - verify_request(request, client_address) - server_close() - process_request(request, client_address) - shutdown_request(request) - close_request(request) - service_actions() - handle_error() Methods for derived classes: - finish_request(request, client_address) Class variables that may be overridden by derived classes or instances: - timeout - address_family - socket_type - allow_reuse_address Instance variables: - RequestHandlerClass - socket """ timeout = None def __init__(self, server_address, RequestHandlerClass): """Constructor. May be extended, do not override.""" self.server_address = server_address self.RequestHandlerClass = RequestHandlerClass self.__is_shut_down = threading.Event() self.__shutdown_request = False def server_activate(self): """Called by constructor to activate the server. May be overridden. """ pass def serve_forever(self, poll_interval=0.5): """Handle one request at a time until shutdown. Polls for shutdown every poll_interval seconds. Ignores self.timeout. If you need to do periodic tasks, do them in another thread. """ self.__is_shut_down.clear() try: while not self.__shutdown_request: # XXX: Consider using another file descriptor or # connecting to the socket to wake this up instead of # polling. Polling reduces our responsiveness to a # shutdown request and wastes cpu at all other times. r, w, e = _eintr_retry(select.select, [self], [], [], poll_interval) if self in r: self._handle_request_noblock() self.service_actions() finally: self.__shutdown_request = False self.__is_shut_down.set() def shutdown(self): """Stops the serve_forever loop. Blocks until the loop has finished. This must be called while serve_forever() is running in another thread, or it will deadlock. """ self.__shutdown_request = True self.__is_shut_down.wait() def service_actions(self): """Called by the serve_forever() loop. May be overridden by a subclass / Mixin to implement any code that needs to be run during the loop. """ pass # The distinction between handling, getting, processing and # finishing a request is fairly arbitrary. Remember: # # - handle_request() is the top-level call. It calls # select, get_request(), verify_request() and process_request() # - get_request() is different for stream or datagram sockets # - process_request() is the place that may fork a new process # or create a new thread to finish the request # - finish_request() instantiates the request handler class; # this constructor will handle the request all by itself def handle_request(self): """Handle one request, possibly blocking. Respects self.timeout. """ # Support people who used socket.settimeout() to escape # handle_request before self.timeout was available. timeout = self.socket.gettimeout() if timeout is None: timeout = self.timeout elif self.timeout is not None: timeout = min(timeout, self.timeout) fd_sets = _eintr_retry(select.select, [self], [], [], timeout) if not fd_sets[0]: self.handle_timeout() return self._handle_request_noblock() def _handle_request_noblock(self): """Handle one request, without blocking. I assume that select.select has returned that the socket is readable before this function was called, so there should be no risk of blocking in get_request(). """ try: request, client_address = self.get_request() except OSError: return if self.verify_request(request, client_address): try: self.process_request(request, client_address) except: self.handle_error(request, client_address) self.shutdown_request(request) def handle_timeout(self): """Called if no new request arrives within self.timeout. Overridden by ForkingMixIn. """ pass def verify_request(self, request, client_address): """Verify the request. May be overridden. Return True if we should proceed with this request. """ return True def process_request(self, request, client_address): """Call finish_request. Overridden by ForkingMixIn and ThreadingMixIn. """ self.finish_request(request, client_address) self.shutdown_request(request) def server_close(self): """Called to clean-up the server. May be overridden. """ pass def finish_request(self, request, client_address): """Finish one request by instantiating RequestHandlerClass.""" self.RequestHandlerClass(request, client_address, self) def shutdown_request(self, request): """Called to shutdown and close an individual request.""" self.close_request(request) def close_request(self, request): """Called to clean up an individual request.""" pass def handle_error(self, request, client_address): """Handle an error gracefully. May be overridden. The default is to print a traceback and continue. """ print('-'*40) print('Exception happened during processing of request from', end=' ') print(client_address) import traceback traceback.print_exc() # XXX But this goes to stderr! print('-'*40)
There are five classes in an inheritance diagram, four of which represent synchronous servers of four types:
RequestHandler类
所有requestHandler都继承BaseRequestHandler基类。
class BaseRequestHandler: """Base class for request handler classes. This class is instantiated for each request to be handled. The constructor sets the instance variables request, client_address and server, and then calls the handle() method. To implement a specific service, all you need to do is to derive a class which defines a handle() method. The handle() method can find the request as self.request, the client address as self.client_address, and the server (in case it needs access to per-server information) as self.server. Since a separate instance is created for each request, the handle() method can define arbitrary other instance variariables. """ def __init__(self, request, client_address, server): self.request = request self.client_address = client_address self.server = server self.setup() try: self.handle() finally: self.finish() def setup(self): pass def handle(self): pass def finish(self): pass
创建socketserver步骤
- First, you must create a request handler class by subclassing the
BaseRequestHandler
class and overriding itshandle()
method; this method will process incoming requests. - Second, you must instantiate one of the server classes, passing it the server’s address and the request handler class.
- Then call the
handle_request()
orserve_forever()
method of the server object to process one or many requests. - Finally, call
server_close()
to close the socket.