3.5.3 Select()

Select()版Socket

Python的select()方法直接调用操作系统的IO接口，它监控sockets,open files, and pipes(所有带fileno()方法的文件句柄)何时变成readable 和writeable, 或者通信错误，select()使得同时监控多个连接变的简单，并且这比写一个长循环来等待和监控多客户端连接要高效，因为select直接通过操作系统提供的C的网络接口进行操作，而不是通过Python的解释器。

注意：Using Python’s file objects with select() works for Unix, but is not supported under Windows.

注意: 对Python的文件对象使用select()只适用于Unix，但在windows下则不支持。

接下来通过echo server例子要以了解select 是如何通过单进程实现同时处理多个非阻塞的socket连接的

服务器端

import select
import socket
import queue

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setblocking(False)
#生成socket实例，创建server，设为非阻塞
server_address = ('localhost', 9999)
print('Server is starting up on address %s, port %s' % (server_address[0], server_address[1]))
server.bind(server_address)
#绑定地址，端口
server.listen(5)
#开始监听

select()方法接收并监控3个通信列表， 第一个是所有的输入的data,就是指外部发过来的数据，第2个是监控和接收所有要发出去的data(outgoing data),第3个监控错误信息，接下来我们需要创建2个列表来包含输入和输出信息来传给select().

inputs = [server]
#我们希望读取数据的socket
outputs = []
#我们希望写入数据的socket

所有客户端的进来的连接和数据将会被server的主循环程序放在上面的list中处理，我们现在的server端需要等待连接可写（writable)之后才能过来，然后接收数据并返回(因此不是在接收到数据之后就立刻返回)，因为每个连接要把输入或输出的数据先缓存到queue里，然后再由select取出来再发出去。

Connections are added to and removed from these lists by the server main loop. Since this version of the server is going to

服务器主循环会把链接（们，connections，有s）从这些列表中加入或移除。因为这个版本的Server将在发送任何数据前会一直等待socket

wait for a socket to become writable before sending any data (instead of immediately sending the reply), each output

变为可写状态（取代之前的立即发送回复），所以每个输出连接都需要一个队列作为数据的缓冲区

connection needs a queue to act as a buffer for the data to be sent through it.

message_queues = {}
#将要发送出去的消息队列

The main portion of the server program loops, calling select() to block and wait for network activity.

下面是此程序的主循环，调用select()时会阻塞和等待直到新的连接和数据进来

while inputs:
    print('Waiting for next event...')
    readable, writeable, exceptional = select.select(inputs, outputs, inputs)
    #等待至少一个链接准备好进行处理

当你把inputs,outputs,exceptional(这里跟inputs共用)传给select()后，它返回3个新的list，我们上面将他们分别赋值为readable,writable,exceptional, 所有在readable list中的socket连接代表有数据可接收(recv),所有在writable list中的存放着你可以对其进行发送(send)操作的socket连接，当连接通信出现error时会把error写到exceptional列表中。

select() returns three new lists, containing subsets of the contents of the lists passed in. All of the sockets in the readable list have incoming data buffered and available to be read. All of the sockets in the writable list have free space in their buffer and can be written to. The sockets returned in exceptional have had an error (the actual definition of “exceptional condition” depends on the platform).

 

Readable list 中的socket 可以有3种可能状态，第一种是如果这个socket是main "server" socket,它负责监听客户端的连接，如果这个main server socket出现在readable里，那代表这是server端已经ready来接收一个新的连接进来了，为了让这个main server能同时处理多个连接，在下面的代码里，我们把这个main server的socket设置为非阻塞模式。

The “readable” sockets represent three possible cases. If the socket is the main “server” socket, the one being used to listen for connections, then the “readable” condition means it is ready to accept another incoming connection. In addition to adding the new connection to the list of inputs to monitor, this section sets the client socket to not block.

    for c in readable:
    #处理inputs
        if c is server:
        #如果这个readable socket是server，说明有新链接进来了，准备接收
            connection, client_address = c.accept()
            #新链接接入
            print('New connection from: ', client_address)
            connection.setblocking(False)
            #将链接设为非阻塞
            inputs.append(connection)
            #inputs，就是readable，将新链接添加进去
            message_queues[connection] = queue.Queue()
            #为将要发送的消息准备一个消息队列，字典格式，键是connection

第二种情况是这个socket是已经建立了的连接，它把数据发了过来，这个时候你就可以通过recv()来接收它发过来的数据，然后把接收到的数据放到queue里，这样你就可以把接收到的数据再传回给客户端了。

The next case is an established connection with a client that has sent data. The data is read with recv(), then placed on the queue so it can be sent through the socket and back to the client.

        else:
        #如果这个readable socket不是server，说明它就是client，那么就准备接收数据
            data = c.recv(1024)
            #接收数据
            if data:
            #如果有数据
                print('Received data from ', c.getpeername())
                print('Data: ', data.decode())
                message_queues[c].put(data)
                #将数据放在要发送的队列里，字典格式，connection的值
                if c not in outputs:
                #同时如果这个client链接没有在outputs列表（writeable）里
                    outputs.append(c)
                    #就将这个链接添加进outputs

第三种情况就是这个客户端已经断开了，所以你再通过recv()接收到的数据就为空了，所以这个时候你就可以把这个跟客户端的连接关闭了。

A readable socket without data available is from a client that has disconnected, and the stream is ready to be closed.

            else:
            #如果没有数据，说明链接已断开
                print('Client connection has lost.')
                #断开信息
                if c in outputs:
                #如果此链接socket在outputs列表（writeable）里
                    outputs.remove(c)
                    #将此链接socket移除
                inputs.remove(c)
                #inputs里也要移除
                c.close()
                #关闭此socket链接

                del message_queues[c]
                #删除掉消息发送字典里该socket存储的消息

对于writable list中的socket，也有几种状态，如果这个客户端连接在跟它对应的queue里有数据，就把这个数据取出来再发回给这个客户端，否则就把这个连接从output list中移除，这样下一次循环select()调用时检测到outputs list中没有这个连接，那就会认为这个连接还处于非活动状态

There are fewer cases for the writable connections. If there is data in the queue for a connection, the next message is sent. Otherwise, the connection is removed from the list of output connections so that the next time through the loop select() does not indicate that the socket is ready to send data.

    for c in writeable:
    #处理outputs
        try:
            next_msg = message_queues[c].get_nowait()
            #尝试从消息队列中取消息
        except queue.Empty:
            print('Output queue for ', c.getpeername(), ' is empty.')
            #如果消息队列为空，抓取Empty报错，然后输出报错信息
            outputs.remove(c)
            #没有消息，说明链接断开，于是从outputs列表中移除引起异常的该socket
        else:
            print('Send %s to %s.' % (next_msg, c.getpeername()))
            #否则就是取到消息
            c.send(next_msg)
            #于是发送

最后，如果在跟某个socket连接通信过程中出了错误，就把这个连接对象在inputs\outputs\message_queue中都删除，再把连接关闭掉

    for c in exceptional:
    #处理异常
        print('Handle exceptional condition for ', c.getpeername())
        inputs.remove(c)
        #将引发异常的socket链接移除
        if c in outputs:
            outputs.remove(c)
        c.close()
        #如果该socket链接在outputs里，移除，同时关闭该socket链接

        del message_queues[c]
        #同时删除消息队列中以该socket为键储存的消息队列

 

客户端

下面的这个是客户端程序展示了如何通过select()对socket进行管理并与多个连接同时进行交互，

The example client program uses two sockets to demonstrate how the server with select() manages multiple connections at the same time. The client starts by connecting each TCP/IP socket to the server.

import socket

messages = ['Saying that I love you',
            'Is not the words I want to hear from you',
            "It's not that I want you",
            'Not to say, but if you only knew',
            'How easy it would be to show me how you feel',
            'More than words is all you have to do to make it real',
            "Then you wouldn't have to say that you love me",
            "Cause I'd already know"]
server_address = ('localhost', 9999)

socks = [socket.socket(socket.AF_INET, socket.SOCK_STREAM),
              ]
#客户端发起的socket链接
print('Connecting to %s port %s' % server_address)
for s in socks:
    s.connect(server_address)
    #开始socks中的每条链接

接下来通过循环通过每个socket连接给server发送和接收数据。

Then it sends one pieces of the message at a time via each socket, and reads all responses available after writing new data.

for m in messages:
#每读取到一条数据
    for s in socks:
    #就让一条socks中的链接
        print('%s: sending %s.' % (s.getsockname(), m))
        s.send(m.encode('utf-8'))
        #发送这条数据

    for s in socks:
    #同时让每条socks中的链接
        data = s.recv(1024)
        #接收服务器返回的数据
        print('%s received data %s.' % (s.getsockname(), data))
        if not data:
            print('Connection %s has lost.' % s.getsockname())
            #没有数据说明链接已断开

完整代码，服务器端：

import select
import socket
import queue

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setblocking(False)
#生成socket实例，创建server，设为非阻塞
server_address = ('localhost', 9999)
print('Server is starting up on address %s, port %s' % (server_address[0], server_address[1]))
server.bind(server_address)
#绑定地址，端口
server.listen(5)
#开始监听
inputs = [server]
#我们希望读取数据的socket
outputs = []
#我们希望写入数据的socket
message_queues = {}
#将要发送出去的消息队列
while inputs:
    print('Waiting for next event...')
    readable, writeable, exceptional = select.select(inputs, outputs, inputs)
    #等待至少一个链接准备好进行处理

    for c in readable:
    #处理inputs
        if c is server:
        #如果这个readable socket是server，说明有新链接进来了，准备接收
            connection, client_address = c.accept()
            #新链接接入
            print('New connection from: ', client_address)
            connection.setblocking(False)
            #将链接设为非阻塞
            inputs.append(connection)
            #inputs，就是readable，将新链接添加进去
            message_queues[connection] = queue.Queue()
            #为将要发送的消息准备一个消息队列，字典格式，键是connection
        else:
        #如果这个readable socket不是server，说明它就是client，那么就准备接收数据
            data = c.recv(1024)
            #接收数据
            if data:
            #如果有数据
                print('Received data from ', c.getpeername())
                print('Data: ', data.decode())
                message_queues[c].put(data)
                #将数据放在要发送的队列里，字典格式，connection的值
                if c not in outputs:
                #同时如果这个client链接没有在outputs列表（writeable）里
                    outputs.append(c)
                    #就将这个链接添加进outputs
            else:
            #如果没有数据，说明链接已断开
                print('Client connection has lost.')
                #断开信息
                if c in outputs:
                #如果此链接socket在outputs列表（writeable）里
                    outputs.remove(c)
                    #将此链接socket移除
                inputs.remove(c)
                #inputs里也要移除
                c.close()
                #关闭此socket链接

                del message_queues[c]
                #删除掉消息发送字典里该socket存储的消息

    for c in writeable:
    #处理outputs
        try:
            next_msg = message_queues[c].get_nowait()
            #尝试从消息队列中取消息
        except queue.Empty:
            print('Output queue for ', c.getpeername(), ' is empty.')
            #如果消息队列为空，抓取Empty报错，然后输出报错信息
            outputs.remove(c)
            #没有消息，说明链接断开，于是从outputs列表中移除引起异常的该socket
        else:
            print('Send %s to %s.' % (next_msg, c.getpeername()))
            #否则就是取到消息
            c.send(next_msg)
            #于是发送

    for c in exceptional:
    #处理异常
        print('Handle exceptional condition for ', c.getpeername())
        inputs.remove(c)
        #将引发异常的socket链接移除
        if c in outputs:
            outputs.remove(c)
        c.close()
        #如果该socket链接在outputs里，移除，同时关闭该socket链接

        del message_queues[c]
        #同时删除消息队列中以该socket为键储存的消息队列

客户端

import socket

messages = ['Saying that I love you',
            'Is not the words I want to hear from you',
            "It's not that I want you",
            'Not to say, but if you only knew',
            'How easy it would be to show me how you feel',
            'More than words is all you have to do to make it real',
            "Then you wouldn't have to say that you love me",
            "Cause I'd already know"]
server_address = ('localhost', 9999)

socks = [socket.socket(socket.AF_INET, socket.SOCK_STREAM),
              ]
#客户端发起的socket链接
print('Connecting to %s port %s' % server_address)
for s in socks:
    s.connect(server_address)
    #开始socks中的每条链接

for m in messages:
#每读取到一条数据
    for s in socks:
    #就让一条socks中的链接
        print('%s: sending %s.' % (s.getsockname(), m))
        s.send(m.encode('utf-8'))
        #发送这条数据

    for s in socks:
    #同时让每条socks中的链接
        data = s.recv(1024)
        #接收服务器返回的数据
        print('%s received data %s.' % (s.getsockname(), data))
        if not data:
            print('Connection %s has lost.' % s.getsockname())
            #没有数据说明链接已断开

输出，服务器端，先启动服务器

Server is starting up on address localhost, port 9999
Waiting for next event...

再启动客户端

Connecting to localhost port 9999
('127.0.0.1', 61267): sending Saying that I love you.
('127.0.0.1', 61267) received data b'Saying that I love you'.
('127.0.0.1', 61267): sending Is not the words I want to hear from you.
('127.0.0.1', 61267) received data b'Is not the words I want to hear from you'.
('127.0.0.1', 61267): sending It's not that I want you.
('127.0.0.1', 61267) received data b"It's not that I want you".
('127.0.0.1', 61267): sending Not to say, but if you only knew.
('127.0.0.1', 61267) received data b'Not to say, but if you only knew'.
('127.0.0.1', 61267): sending How easy it would be to show me how you feel.
('127.0.0.1', 61267) received data b'How easy it would be to show me how you feel'.
('127.0.0.1', 61267): sending More than words is all you have to do to make it real.
('127.0.0.1', 61267) received data b'More than words is all you have to do to make it real'.
('127.0.0.1', 61267): sending Then you wouldn't have to say that you love me.
('127.0.0.1', 61267) received data b"Then you wouldn't have to say that you love me".
('127.0.0.1', 61267): sending Cause I'd already know.
('127.0.0.1', 61267) received data b"Cause I'd already know".

最后服务器

Server is starting up on address localhost, port 9999
Waiting for next event...
New connection from:  ('127.0.0.1', 61267)
Waiting for next event...
Received data from  ('127.0.0.1', 61267)
Data:  Saying that I love you
Waiting for next event...
Send b'Saying that I love you' to ('127.0.0.1', 61267).
Waiting for next event...
Output queue for  ('127.0.0.1', 61267)  is empty.
Waiting for next event...
Received data from  ('127.0.0.1', 61267)
Data:  Is not the words I want to hear from you
Waiting for next event...
Send b'Is not the words I want to hear from you' to ('127.0.0.1', 61267).
Waiting for next event...
Output queue for  ('127.0.0.1', 61267)  is empty.
Waiting for next event...
Received data from  ('127.0.0.1', 61267)
Data:  It's not that I want you
Waiting for next event...
Send b"It's not that I want you" to ('127.0.0.1', 61267).
Waiting for next event...
Output queue for  ('127.0.0.1', 61267)  is empty.
Waiting for next event...
Received data from  ('127.0.0.1', 61267)
Data:  Not to say, but if you only knew
Waiting for next event...
Send b'Not to say, but if you only knew' to ('127.0.0.1', 61267).
Waiting for next event...
Output queue for  ('127.0.0.1', 61267)  is empty.
Waiting for next event...
Received data from  ('127.0.0.1', 61267)
Data:  How easy it would be to show me how you feel
Waiting for next event...
Send b'How easy it would be to show me how you feel' to ('127.0.0.1', 61267).
Waiting for next event...
Output queue for  ('127.0.0.1', 61267)  is empty.
Waiting for next event...
Received data from  ('127.0.0.1', 61267)
Data:  More than words is all you have to do to make it real
Waiting for next event...
Send b'More than words is all you have to do to make it real' to ('127.0.0.1', 61267).
Waiting for next event...
Output queue for  ('127.0.0.1', 61267)  is empty.
Waiting for next event...
Received data from  ('127.0.0.1', 61267)
Data:  Then you wouldn't have to say that you love me
Waiting for next event...
Send b"Then you wouldn't have to say that you love me" to ('127.0.0.1', 61267).
Waiting for next event...
Output queue for  ('127.0.0.1', 61267)  is empty.
Waiting for next event...
Received data from  ('127.0.0.1', 61267)
Data:  Cause I'd already know
Waiting for next event...
Send b"Cause I'd already know" to ('127.0.0.1', 61267).
Waiting for next event...
Output queue for  ('127.0.0.1', 61267)  is empty.
Waiting for next event...
Client connection has lost.
Waiting for next event...