python之路[8] - Socket网络编程 - 弃
【转】
正文
1. Socket介绍
概念
socket本质上就是在2台网络互通的电脑之间,架设一个通道,两台电脑通过这个通道来实现数据的互相传递。 我们知道网络 通信 都 是基于 ip+port 方能定位到目标的具体机器上的具体服务,操作系统有0-65535个端口,每个端口都可以独立对外提供服务,如果 把一个公司比做一台电脑 ,那公司的总机号码就相当于ip地址, 每个员工的分机号就相当于端口, 你想找公司某个人,必须 先打电话到总机,然后再转分机 。
建立一个socket必须至少有2端, 一个服务端,一个客户端, 服务端被动等待并接收请求,客户端主动发起请求, 连接建立之后,双方可以互发数据。
伪代码
Socket socket = getSocket(type = "TCP") #设定好协议类型 connect(socket, address = "1.2.3.4", port = "80") #连接远程机器 send(socket, "Hello, world!") #发送消息 close(socket) #关闭连接
A socket API is an application programming interface (API), usually provided by the operating system, that allows application programs to control and use network sockets. Internet socket APIs are usually based on the Berkeley sockets standard. In the Berkeley sockets standard, sockets are a form of file descriptor (a file handle), due to the Unix philosophy that "everything is a file", and the analogies between sockets and files: you can read, write, open, and close both.
A socket address is the combination of an IP address and a port number, much like one end of a telephone connection is the combination of a phone number and a particular extension. Sockets need not have an address (for example for only sending data), but if a program binds a socket to an address, the socket can be used to receive data sent to that address. Based on this address, internet sockets deliver incoming data packets to the appropriate application process or thread.
Socket Families(地址簇)
socket.
AF_UNIX unix本机进程间通信
socket.
AF_INET IPV4
socket.
AF_INET6 IPV6
These constants represent the address (and protocol) families, used for the first argument to
socket()
. If the AF_UNIX
constant is not defined then this protocol is unsupported. More constants may be available depending on the system.
Socket Types
socket.
SOCK_STREAM #for tcp
socket.
SOCK_DGRAM #for udp
socket.
SOCK_RAW #原始套接字,普通的套接字无法处理ICMP、IGMP等网络报文,而SOCK_RAW可以;其次,SOCK_RAW也可以处理特殊的IPv4报文;此外,利用原始套接字,可以通过IP_HDRINCL套接字选项由用户构造IP头。
socket.
SOCK_RDM #是一种可靠的UDP形式,即保证交付数据报但不保证顺序。SOCK_RAM用来提供对原始协议的低级访问,在需要执行某些特殊操作时使用,如发送ICMP报文。SOCK_RAM通常仅限于高级用户或管理员运行的程序使用。
socket.
SOCK_SEQPACKET #废弃了
These constants represent the socket types, used for the second argument to socket()
. More constants may be available depending on the system. (Only SOCK_STREAM
and SOCK_DGRAM
appear to be generally useful.)
2. Socket 参数介绍
socket.
socket
(family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None) 必会
Create a new socket using the given address family, socket type and protocol number. The address family should be AF_INET
(the default), AF_INET6
, AF_UNIX
, AF_CAN
or AF_RDS
. The socket type should beSOCK_STREAM
(the default), SOCK_DGRAM
, SOCK_RAW
or perhaps one of the other SOCK_
constants. The protocol number is usually zero and may be omitted or in the case where the address family is AF_CAN
the protocol should be one of CAN_RAW
or CAN_BCM
. If fileno is specified, the other arguments are ignored, causing the socket with the specified file descriptor to return. Unlike socket.fromfd()
, fileno will return the same socket and not a duplicate. This may help close a detached socket using socket.close()
.
socket.
socketpair
([family[, type[, proto]]])
Build a pair of connected socket objects using the given address family, socket type, and protocol number. Address family, socket type, and protocol number are as for the socket()
function above. The default family is AF_UNIX
if defined on the platform; otherwise, the default is AF_INET
.
socket.
create_connection
(address[, timeout[, source_address]])
Connect to a TCP service listening on the Internet address (a 2-tuple (host, port)
), and return the socket object. This is a higher-level function than socket.connect()
: if host is a non-numeric hostname, it will try to resolve it for both AF_INET
and AF_INET6
, and then try to connect to all possible addresses in turn until a connection succeeds. This makes it easy to write clients that are compatible to both IPv4 and IPv6.
Passing the optional timeout parameter will set the timeout on the socket instance before attempting to connect. If no timeout is supplied, the global default timeout setting returned by getdefaulttimeout()
is used.
If supplied, source_address must be a 2-tuple (host, port)
for the socket to bind to as its source address before connecting. If host or port are ‘’ or 0 respectively the OS default behavior will be used.
socket.
getaddrinfo
(host, port, family=0, type=0, proto=0, flags=0) #获取要连接的对端主机地址 必会
sk.bind(address) 必会
s.bind(address) 将套接字绑定到地址。address地址的格式取决于地址族。在AF_INET下,以元组(host,port)的形式表示地址。
sk.listen(backlog) 必会
开始监听传入连接。backlog指定在拒绝连接之前,可以挂起的最大连接数量。
backlog等于5,表示内核已经接到了连接请求,但服务器还没有调用accept进行处理的连接个数最大为5
这个值不能无限大,因为要在内核中维护连接队列
sk.setblocking(bool) 必会
是否阻塞(默认True),如果设置False,那么accept和recv时一旦无数据,则报错。
sk.accept() 必会
接受连接并返回(conn,address),其中conn是新的套接字对象,可以用来接收和发送数据。address是连接客户端的地址。
接收TCP 客户的连接(阻塞式)等待连接的到来
sk.connect(address) 必会
连接到address处的套接字。一般,address的格式为元组(hostname,port),如果连接出错,返回socket.error错误。
sk.connect_ex(address)
同上,只不过会有返回值,连接成功时返回 0 ,连接失败时候返回编码,例如:10061
sk.close() 必会
关闭套接字
sk.recv(bufsize[,flag]) 必会
接受套接字的数据。数据以字符串形式返回,bufsize指定最多可以接收的数量。flag提供有关消息的其他信息,通常可以忽略。
sk.recvfrom(bufsize[.flag])
与recv()类似,但返回值是(data,address)。其中data是包含接收数据的字符串,address是发送数据的套接字地址。
sk.send(string[,flag]) 必会
将string中的数据发送到连接的套接字。返回值是要发送的字节数量,该数量可能小于string的字节大小。即:可能未将指定内容全部发送。
sk.sendall(string[,flag]) 必会
将string中的数据发送到连接的套接字,但在返回之前会尝试发送所有数据。成功返回None,失败则抛出异常。
内部通过递归调用send,将所有内容发送出去。
sk.sendto(string[,flag],address)
将数据发送到套接字,address是形式为(ipaddr,port)的元组,指定远程地址。返回值是发送的字节数。该函数主要用于UDP协议。
sk.settimeout(timeout) 必会
设置套接字操作的超时期,timeout是一个浮点数,单位是秒。值为None表示没有超时期。一般,超时期应该在刚创建套接字时设置,因为它们可能用于连接的操作(如 client 连接最多等待5s )
sk.getpeername() 必会
返回连接套接字的远程地址。返回值通常是元组(ipaddr,port)。
sk.getsockname()
返回套接字自己的地址。通常是一个元组(ipaddr,port)
sk.fileno()
套接字的文件描述符
socket.
sendfile
(file, offset=0, count=None)
发送文件 ,但目前多数情况下并无什么卵用。
3. 基本Socket实例
前面讲了这么多,到底咋么用呢?
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
import socket server = socket.socket() #获得socket实例 server.bind(("localhost",9998)) #绑定ip port server.listen() #开始监听 print("等待客户端的连接...") conn,addr = server.accept() #接受并建立与客户端的连接,程序在此处开始阻塞,只到有客户端连接进来... print("新连接:",addr ) data = conn.recv(1024) print("收到消息:",data) server.close()
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
import socket client = socket.socket() client.connect(("localhost",9998)) client.send(b"hey") client.close()
上面的代码的有一个问题, 就是SocketServer.py运行起来后, 接收了一次客户端的data就退出了。。。, 但实际场景中,一个连接建立起来后,可能要进行多次往返的通信。
多次的数据交互怎么实现呢?
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
import socket server = socket.socket() #获得socket实例 server.bind(("localhost",9998)) #绑定ip port server.listen() #开始监听 print("等待客户端的连接...") conn,addr = server.accept() #接受并建立与客户端的连接,程序在此处开始阻塞,只到有客户端连接进来... print("新连接:",addr ) while True: data = conn.recv(1024) print("收到消息:",data) conn.send(data.upper()) server.close()
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
import socket client = socket.socket() client.connect(("localhost",9998)) while True: msg = input(">>:").strip() if len(msg) == 0:continue client.send( msg.encode("utf-8") ) data = client.recv(1024) print("来自服务器:",data) client.close()
看客户端断开时服务器端的输出
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
import socket server = socket.socket() #获得socket实例 server.bind(("localhost",9998)) #绑定ip port server.listen() #开始监听 print("等待客户端的连接...") conn,addr = server.accept() #接受并建立与客户端的连接,程序在此处开始阻塞,只到有客户端连接进来... print("新连接:",addr ) while True: data = conn.recv(1024) if not data: print("客户端断开了...") break print("收到消息:",data) conn.send(data.upper()) server.close()
4.Socket实现多连接处理
上面的代码虽然实现了服务端与客户端的多次交互,但是你会发现,如果客户端断开了, 服务器端也会跟着立刻断开,因为服务器只有一个while 循环,客户端一断开,服务端收不到数据 ,就会直接break跳出循环,然后程序就退出了,这显然不是我们想要的结果 ,我们想要的是,客户端如果断开了,我们这个服务端还可以为下一个客户端服务,它不能断,她接完一个客,擦完嘴角的遗留物,就要接下来勇敢的去接待下一个客人。 在这里如何实现呢?
conn,addr = server.accept() #接受并建立与客户端的连接,程序在此处开始阻塞,只到有客户端连接进来...
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
import socket server = socket.socket() #获得socket实例 server.bind(("localhost",9998)) #绑定ip port server.listen() #开始监听 while True: #第一层loop print("等待客户端的连接...") conn,addr = server.accept() #接受并建立与客户端的连接,程序在此处开始阻塞,只到有客户端连接进来... print("新连接:",addr ) while True: data = conn.recv(1024) if not data: print("客户端断开了...") break #这里断开就会再次回到第一次外层的loop print("收到消息:",data) conn.send(data.upper()) server.close()
5.通过socket实现简单的ssh
光只是简单的发消息、收消息没意思,干点正事,可以做一个极简版的ssh,就是客户端连接上服务器后,让服务器执行命令,并返回结果给客户端。
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
import socket import os server = socket.socket() #获得socket实例 #server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) server.bind(("localhost",9998)) #绑定ip port server.listen() #开始监听 while True: #第一层loop print("等待客户端的连接...") conn,addr = server.accept() #接受并建立与客户端的连接,程序在此处开始阻塞,只到有客户端连接进来... print("新连接:",addr ) while True: data = conn.recv(1024) if not data: print("客户端断开了...") break #这里断开就会再次回到第一次外层的loop print("收到命令:",data) res = os.popen(data.decode()).read() #py3 里socket发送的只有bytes,os.popen又只能接受str,所以要decode一下 print(len(res)) conn.send(res.encode("utf-8")) server.close() socket ssh server
客户端代码
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
import socket client = socket.socket() client.connect(("localhost",9998)) while True: msg = input(">>:").strip() if len(msg) == 0:continue client.send( msg.encode("utf-8") ) data = client.recv(1024) print(data.decode()) #命令执行结果 client.close() socket ssh client
|
由于socket粘包的缘故,需要插入一个send行为和一个total_data_size来循环接受数据
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
#_*_coding:utf-8_*_ __author__ = 'Alex Li' import socket import os,subprocess server = socket.socket() #获得socket实例 server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) server.bind(("localhost",9999)) #绑定ip port server.listen() #开始监听 while True: #第一层loop print("等待客户端的连接...") conn,addr = server.accept() #接受并建立与客户端的连接,程序在此处开始阻塞,只到有客户端连接进来... print("新连接:",addr ) while True: data = conn.recv(1024) if not data: print("客户端断开了...") break #这里断开就会再次回到第一次外层的loop print("收到命令:",data) #res = os.popen(data.decode()).read() #py3 里socket发送的只有bytes,os.popen又只能接受str,所以要decode一下 res = subprocess.Popen(data,shell=True,stdout=subprocess.PIPE).stdout.read() #跟上面那条命令的效果是一样的 if len(res) == 0: res = "cmd exec success,has not output!".encode("utf-8") conn.send(str(len(res)).encode("utf-8")) #发送数据之前,先告诉客户端要发多少数据给它 print("等待客户ack应答...") client_final_ack = conn.recv(1024) #等待客户端响应 print("客户应答:",client_final_ack.decode()) print(type(res)) conn.sendall(res) #发送端也有最大数据量限制,所以这里用sendall,相当于重复循环调用conn.send,直至数据发送完毕 server.close()
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
#_*_coding:utf-8_*_ __author__ = 'Alex Li' import socket import sys client = socket.socket() client.connect(("localhost",9999)) while True: msg = input(">>:").strip() if len(msg) == 0:continue client.send( msg.encode("utf-8") ) res_return_size = client.recv(1024) #接收这条命令执行结果的大小 print("getting cmd result , ", res_return_size) total_rece_size = int(res_return_size) print("total size:",res_return_size) client.send("准备好接收了,发吧loser".encode("utf-8")) received_size = 0 #已接收到的数据 cmd_res = b'' f = open("test_copy.html","wb")#把接收到的结果存下来,一会看看收到的数据 对不对 while received_size != total_rece_size: #代表还没收完 data = client.recv(1024) received_size += len(data) #为什么不是直接1024,还判断len干嘛,注意,实际收到的data有可能比1024少 cmd_res += data else: print("数据收完了",received_size) #print(cmd_res.decode()) f.write(cmd_res) #把接收到的结果存下来,一会看看收到的数据 对不对 #print(data.decode()) #命令执行结果 client.close()
6. SocketServer
The socketserver
module simplifies the task of writing network servers.
socketserver一共有这么几种类型
#TCP
class socketserver.TCPServer(server_address, RequestHandlerClass, bind_and_activate=True) #UDP class socketserver.UDPServer(server_address, RequestHandlerClass, bind_and_activate=True) # Unix服务器 class socketserver.UnixStreamServer(server_address, RequestHandlerClass, bind_and_activate=True) class socketserver.UnixDatagramServer(server_address, RequestHandlerClass,bind_and_activate=True)
There are five classes in an inheritance diagram, four of which represent synchronous servers of four types:
+------------+
| BaseServer |
+------------+
|
v
+-----------+ +------------------+
| TCPServer |------->| UnixStreamServer |
+-----------+ +------------------+
|
v
+-----------+ +--------------------+
| UDPServer |------->| UnixDatagramServer |
+-----------+ +--------------------+
创建一个socketserver 至少分以下几步:
- First, you must create a request handler class by subclassing the
BaseRequestHandler
class and overriding itshandle()
method; this method will process incoming requests. - Second, you must instantiate one of the server classes, passing it the server’s address and the request handler class.
- Then call the
handle_request()
orserve_forever()
method of the server object to process one or many requests. - Finally, call
server_close()
to close the socket.
基本的socketserver代码
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
import socketserver class MyTCPHandler(socketserver.BaseRequestHandler): """ The request handler class for our server. It is instantiated once per connection to the server, and must override the handle() method to implement communication to the client. """ def handle(self): # self.request is the TCP socket connected to the client self.data = self.request.recv(1024).strip() print("{} wrote:".format(self.client_address[0])) print(self.data) # just send back the same data, but upper-cased self.request.sendall(self.data.upper()) if __name__ == "__main__": HOST, PORT = "localhost", 9999 # Create the server, binding to localhost on port 9999 server = socketserver.TCPServer((HOST, PORT), MyTCPHandler) # Activate the server; this will keep running until you # interrupt the program with Ctrl-C server.serve_forever()
但你发现,上面的代码,依然不能同时处理多个连接,擦,那我搞这个干嘛?别急,不是不能处理多并发,如果你想,你还要启用多线程,多线程我们现在还没讲,但你大体知道,有了多线程,就能同时让cpu干多件事了就行先。
让你的socketserver并发起来, 必须选择使用以下一个多并发的类
class socketserver.
ForkingTCPServer
class socketserver.
ForkingUDPServer
class socketserver.
ThreadingTCPServer
class socketserver.
ThreadingUDPServer
so 只需要把下面这句
server = socketserver.TCPServer((HOST, PORT), MyTCPHandler)
换成下面这个,就可以多并发了,这样,客户端每连进一个来,服务器端就会分配一个新的线程来处理这个客户端的请求
server = socketserver.ThreadingTCPServer((HOST, PORT), MyTCPHandler)
class socketserver.
BaseServer
(server_address, RequestHandlerClass) 主要有以下方法
![](https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif)
class socketserver.BaseServer(server_address, RequestHandlerClass) This is the superclass of all Server objects in the module. It defines the interface, given below, but does not implement most of the methods, which is done in subclasses. The two parameters are stored in the respective server_address and RequestHandlerClass attributes. fileno() Return an integer file descriptor for the socket on which the server is listening. This function is most commonly passed to selectors, to allow monitoring multiple servers in the same process. handle_request() Process a single request. This function calls the following methods in order: get_request(), verify_request(), and process_request(). If the user-provided handle() method of the handler class raises an exception, the server’s handle_error() method will be called. If no request is received within timeout seconds, handle_timeout() will be called and handle_request() will return. serve_forever(poll_interval=0.5) Handle requests until an explicit shutdown() request. Poll for shutdown every poll_interval seconds. Ignores the timeout attribute. It also calls service_actions(), which may be used by a subclass or mixin to provide actions specific to a given service. For example, the ForkingMixIn class uses service_actions() to clean up zombie child processes. Changed in version 3.3: Added service_actions call to the serve_forever method. service_actions() This is called in the serve_forever() loop. This method can be overridden by subclasses or mixin classes to perform actions specific to a given service, such as cleanup actions. New in version 3.3. shutdown() Tell the serve_forever() loop to stop and wait until it does. server_close() Clean up the server. May be overridden. address_family The family of protocols to which the server’s socket belongs. Common examples are socket.AF_INET and socket.AF_UNIX. RequestHandlerClass The user-provided request handler class; an instance of this class is created for each request. server_address The address on which the server is listening. The format of addresses varies depending on the protocol family; see the documentation for the socket module for details. For Internet protocols, this is a tuple containing a string giving the address, and an integer port number: ('127.0.0.1', 80), for example. socket The socket object on which the server will listen for incoming requests. The server classes support the following class variables: allow_reuse_address Whether the server will allow the reuse of an address. This defaults to False, and can be set in subclasses to change the policy. request_queue_size The size of the request queue. If it takes a long time to process a single request, any requests that arrive while the server is busy are placed into a queue, up to request_queue_size requests. Once the queue is full, further requests from clients will get a “Connection denied” error. The default value is usually 5, but this can be overridden by subclasses. socket_type The type of socket used by the server; socket.SOCK_STREAM and socket.SOCK_DGRAM are two common values. timeout Timeout duration, measured in seconds, or None if no timeout is desired. If handle_request() receives no incoming requests within the timeout period, the handle_timeout() method is called. There are various server methods that can be overridden by subclasses of base server classes like TCPServer; these methods aren’t useful to external users of the server object. finish_request() Actually processes the request by instantiating RequestHandlerClass and calling its handle() method. get_request() Must accept a request from the socket, and return a 2-tuple containing the new socket object to be used to communicate with the client, and the client’s address. handle_error(request, client_address) This function is called if the handle() method of a RequestHandlerClass instance raises an exception. The default action is to print the traceback to standard output and continue handling further requests. handle_timeout() This function is called when the timeout attribute has been set to a value other than None and the timeout period has passed with no requests being received. The default action for forking servers is to collect the status of any child processes that have exited, while in threading servers this method does nothing. process_request(request, client_address) Calls finish_request() to create an instance of the RequestHandlerClass. If desired, this function can create a new process or thread to handle the request; the ForkingMixIn and ThreadingMixIn classes do this. server_activate() Called by the server’s constructor to activate the server. The default behavior for a TCP server just invokes listen() on the server’s socket. May be overridden. server_bind() Called by the server’s constructor to bind the socket to the desired address. May be overridden. verify_request(request, client_address) Must return a Boolean value; if the value is True, the request will be processed, and if it’s False, the request will be denied. This function can be overridden to implement access controls for a server. The default implementation always returns True.
利用socket缓存dns信息
import socket _dnscache={} def _setDNSCache(): """ Makes a cached version of socket._getaddrinfo to avoid subsequent DNS requests. """ def _getaddrinfo(*args, **kwargs): global _dnscache if args in _dnscache: print str(args)+" in cache" return _dnscache[args] else: print str(args)+" not in cache" _dnscache[args] = socket._getaddrinfo(*args, **kwargs) return _dnscache[args] if not hasattr(socket, '_getaddrinfo'): socket._getaddrinfo = socket.getaddrinfo socket.getaddrinfo = _getaddrinfo def test(): _setDNSCache() import urllib urllib.urlopen('http://www.baidu.com') urllib.urlopen('http://www.baidu.com') test()
不过这个方案虽好,但也有缺陷,罗列如下:
1.相当于只对socket.getaddrinfo打了一个patch,但socket.gethostbyname,socket.gethostbyname_ex还是走之前的策略
2.只对本程序有效,而修改/etc/hosts将对所有程序有效,包括ping
7层网络模型
作业:开发一个支持多用户在线的FTP程序
要求:
- 用户加密认证
- 允许同时多用户登录
- 每个用户有自己的家目录 ,且只能访问自己的家目录
- 对用户进行磁盘配额,每个用户的可用空间不同
- 允许用户在ftp server上随意切换目录
- 允许用户查看当前目录下文件
- 允许上传和下载文件,保证文件一致性
- 文件传输过程中显示进度条
- 附加功能:支持文件的断点续传