WebSocket协议学习
websocket协议规定了客户端和服务端socket连接和通信时的规则，一是连接握手时的认证，二是通信时的数据报文解析。其整个流程的简单分析如下：
（websocket简介参见：https://www.zhihu.com/question/20215561/answer/40316953）
1.websocket服务器和客户端连接
socket服务端
#coding: utf-8 import socket soc = socket.socket(socket.AF_INET,socket.SOCK_STREAM) soc.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1) soc.bind(('127.0.0.1',8080)) soc.listen(5) client,address = soc.accept() msg = client.recv(8096) print msg
websocket客户端
<!DOCTYPE html> <html lang="zh-CN"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Title</title> </head> <body> <script> var web = new WebSocket("ws://127.0.0.1:8080") </script> </body> </html>
执行后可以看到客户端发过来的请求信息如下，比普通的http请求头多了一个Sec-WebSocket-Key，用来进行握手认证
GET / HTTP/1.1 Host: 127.0.0.1:8080 User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:63.0) Gecko/20100101 Firefox/63.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2 Accept-Encoding: gzip, deflate Sec-WebSocket-Version: 13 Origin: http://localhost:63342 Sec-WebSocket-Extensions: permessage-deflate Sec-WebSocket-Key: lOfBaOFgUccUfIKUDD5Bxw== Connection: keep-alive, Upgrade Pragma: no-cache Cache-Control: no-cache Upgrade: websocket
服务端接受websocket客户端的请求消息后，若要与客户端进行握手认证，要遵循的规则如下：
- 从上述客户端请求信息中提取 Sec-WebSocket-Key
- 利用magic_string 和 Sec-WebSocket-Key 进行hmac1加密，再进行base64加密 （magic string为：258EAFA5-E914-47DA-95CA-C5AB0DC85B11 固定不变）
- 将加密结果响应给客户端
返回的请求头如下：
HTTP/1.1 101 Switching Protocols Upgrade:websocket Connection: Upgrade Sec-WebSocket-Accept: Ip8Lp7v3m6xnPYlNIQ83SgGwrwA= WebSocket-Location: ws://127.0.0.1:8080/
Sec-WebSocket-Accept为最重要的验证字段，其计算过程如下：
-
将 Sec-WebSocket-Key 跟 258EAFA5-E914-47DA-95CA-C5AB0DC85B11 拼接；
-
通过 SHA1 计算出摘要，并转成 base64 字符串。
代码实现如下：
#coding: utf-8 import socket import base64 import hashlib #处理请求头消息 def get_header(data): data = str(data) header_dict={} if data: header,body = data.split('\r\n\r\n',1) header_list = header.split('\r\n') #print header_list for i in range(0,len(header_list)): if i==0: lenth = len(header_list[i].split(' ')) if lenth==3: header_dict['Method'],header_dict['Url'],header_dict['Protocol']=header_list[i].split(' ') else: k,v=header_list[i].split(':',1) header_dict[k]=v.strip() # 此处注意要去除空格，否则后面的Sec-WebSocket-Key的加密验证会失败 return header_dict soc = socket.socket(socket.AF_INET,socket.SOCK_STREAM) soc.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1) soc.bind(('127.0.0.1',8080)) soc.listen(5) client,address = soc.accept() data = client.recv(8096) header = get_header(data) response_tpl = "HTTP/1.1 101 Switching Protocols\r\n" \ "Upgrade:websocket\r\n" \ "Connection: Upgrade\r\n" \ "Sec-WebSocket-Accept: %s\r\n" \ "WebSocket-Location: ws://%s%s\r\n\r\n" magic_string = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11' msg = header['Sec-WebSocket-Key'].strip()+magic_string #注意header['Sec-WebSocket-Key']前后是否有多余的空格 print msg encrypt_msg = base64.b64encode(hashlib.sha1(msg).digest()) #加密得到Sec-WebSocket-Accept response_str=response_tpl%(encrypt_msg,header['Host'],header['Url']) print response_str client.send(response_str)
2.websocket服务端和客户端通信
websocket客户端发送过来的数据报文格式如下，服务端需要对报文进行解析，然后再将回复内容进行封包，发送给客户端。
（websocket protocol: https://tools.ietf.org/html/rfc6455#section-5.1）
相关含义如下：
The MASK bit simply tells whether the message is encoded. Messages from the client must be masked, so your server should expect this to be 1. (In fact, section 5.1 of the spec says that your server must disconnect from a client if that client sends an unmasked message.) When sending a frame back to the client, do not mask it and do not set the mask bit. We'll explain masking later. Note: You have to mask messages even when using a secure socket.RSV1-3 can be ignored, they are for extensions.
The opcode field defines how to interpret the payload data: 0x0 for continuation,
0x1for text (which is always encoded in UTF-8),0x2for binary, and other so-called "control codes" that will be discussed later. In this version of WebSockets,0x3to0x7and0xBto0xFhave no meaning.The FIN bit tells whether this is the last message in a series. If it's 0, then the server will keep listening for more parts of the message; otherwise, the server should consider the message delivered. More on this later.
Decoding Payload Length
To read the payload data, you must know when to stop reading. That's why the payload length is important to know. Unfortunately, this is somewhat complicated. To read it, follow these steps:
- Read bits 9-15 (inclusive) and interpret that as an unsigned integer. If it's 125 or less, then that's the length; you're done. If it's 126, go to step 2. If it's 127, go to step 3.
- Read the next 16 bits and interpret those as an unsigned integer. You're done.
- Read the next 64 bits and interpret those as an unsigned integer (The most significant bit MUST be 0). You're done.
Reading and Unmasking the Data
If the MASK bit was set (and it should be, for client-to-server messages), read the next 4 octets (32 bits); this is the masking key. Once the payload length and masking key is decoded, you can go ahead and read that number of bytes from the socket. Let's call the data ENCODED, and the key MASK. To get DECODED, loop through the octets (bytes a.k.a. characters for text data) of ENCODED and XOR the octet with the (i modulo 4)th octet of MASK. In pseudo-code (that happens to be valid JavaScript):
var DECODED = "";
for (var i = 0; i < ENCODED.length; i++) {
DECODED[i] = ENCODED[i] ^ MASK[i % 4];
}Now you can figure out what DECODED means depending on your application.
第一步：对客户端数据报文解析
解包流程：
1，根据payload len的值（字节序号1的后七位）来确定payload占几个字节
2， 确定payload占的字节数后，其后四个字节即为Masking-key（MASK bit 设置为1时，Masking-key才存在），Masking-key后面的所有字节为payload data
3，利用Masking-key对payload data进行异或运算进行解码，拿到客户端发送的数据
代码实现解包流程如下：
python 2.7
def get_data(msg): length = ord(msg[1])&127 #127的二进制为01111111，和127进行与运算，能拿到msg[1]的后七位 if length==126: #不加ord时，msg[1]为字符窜，不支持与运算 mask = msg[4:8] pay_data = msg[8:] elif length==127: mask = msg[10:14] pay_data = msg[14:] else: mask = msg[2:6] pay_data = msg[6:] decode='' for i in range(len(pay_data)): decode+=chr(ord(pay_data[i]) ^ ord(mask[i%4])) return decode #python3环境下代码 # def get_data(msg): # length = msg[1]&127 # if length==126: # mask = msg[4:8] # pay_data = msg[8:] # elif length==127: # mask = msg[10:14] # pay_data = msg[14:] # else: # mask = msg[2:6] # pay_data = msg[6:] # bytes_list = bytearray() # for i in range(len(pay_data)): # chunk=pay_data[i] ^ mask[i%4] # decode=str(bytes_list.append(chunk),encoding='utf-8') # return decode
第二步：将数据封包，发送给客户端
返回数据报文的MASK bit为0，因此没有Masking-key，数据报文组成：token（字节序号0）+payload lenth +payload data
实现代码如下：
def response_data(msg): token = struct.pack('B',129) #写入第一个字节 10000001 payload_len = len(msg) if payload_len <=125: token += struct.pack('B',payload_len) elif payload_len<=126: token += struct.pack('BH',126,payload_len) else: token += struct.pack('BH', 127, payload_len) data = token+msg return data
3. 基于websocket的聊天简单测试
客户端：以js中的websocket做为客户端
<!DOCTYPE html> <html lang="zh-CN"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Title</title> </head> <body> <div id="content" style="border:solid gray 1px; width:400px; height:400px;margin:100px 0px 0px 100px"></div> <div style="margin-left:100px"> <input type="text" id="msg"/> <button onclick="sendMsg();">发送</button> <button onclick="closeCon();">断开连接</button> </div> <script> var web = new WebSocket("ws://127.0.0.1:8080/"); web.onopen=function () { var newTag = document.createElement('div'); newTag.innerHTML='[连接成功]'; document.getElementById('content').appendChild(newTag); } web.onerror=function (error) { console.log('Error:'+error); } web.onmessage=function (event) { var newTag = document.createElement('div'); newTag.innerHTML=event.data; document.getElementById('content').appendChild(newTag); }; web.onclose=function () { var newTag = document.createElement('div'); newTag.innerHTML='[断开连接]'; document.getElementById('content').appendChild(newTag); }; function sendMsg() { var mstag = document.getElementById('msg'); web.send(mstag.value); mstag.value=''; }; function closeCon() { web.close(); var newTag = document.createElement('div'); newTag.innerHTML='[断开连接]'; document.getElementById('content').appendChild(newTag); }; </script> </body> </html>
服务器：基于上面的握手和通信过程，对于客户端发过来的消息，回复其消息
#coding:utf-8 import socket import base64 import hashlib import struct #处理请求头消息 def get_header(data): data = str(data) header_dict={} if data: header,body = data.split('\r\n\r\n',1) header_list = header.split('\r\n') #print header_list for i in range(0,len(header_list)): if i==0: lenth = len(header_list[i].split(' ')) if lenth==3: header_dict['Method'],header_dict['Url'],header_dict['Protocol']=header_list[i].split(' ') else: k,v=header_list[i].split(':',1) header_dict[k]=v.strip() # 此处注意要去除空格，否则后面的Sec-WebSocket-Key的加密验证会失败 return header_dict def get_data(msg): length = ord(msg[1])&127 #127的二进制为01111111，和127进行与运算，能拿到msg[1]的后七位 if length==126: #不加ord时，msg[1]为字符窜，不支持与运算 mask = msg[4:8] pay_data = msg[8:] elif length==127: mask = msg[10:14] pay_data = msg[14:] else: mask = msg[2:6] pay_data = msg[6:] decode='' for i in range(len(pay_data)): decode+=chr(ord(pay_data[i]) ^ ord(mask[i%4])) return decode def response_data(msg): token = struct.pack('B',129) #写入第一个字节 10000001 payload_len = len(msg) if payload_len <=125: token += struct.pack('B',payload_len) elif payload_len<=126: token += struct.pack('BH',126,payload_len) else: token += struct.pack('BH', 127, payload_len) data = token+msg return data def run(): soc = socket.socket(socket.AF_INET,socket.SOCK_STREAM) soc.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1) soc.bind(('127.0.0.1',8080)) soc.listen(5) client,address = soc.accept() data = client.recv(8096) header = get_header(data) response_tpl = "HTTP/1.1 101 Switching Protocols\r\n" \ "Upgrade:websocket\r\n" \ "Connection: Upgrade\r\n" \ "Sec-WebSocket-Accept: %s\r\n" \ "WebSocket-Location: ws://%s%s\r\n\r\n" magic_string = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11' hand_str = header['Sec-WebSocket-Key'].strip()+magic_string #注意header['Sec-WebSocket-Key']前后是否有多余的空格 encrypt_str = base64.b64encode(hashlib.sha1(hand_str).digest()) response_str=response_tpl%(encrypt_str,header['Host'],header['Url']) print response_str client.send(response_str) while True: try: msg = client.recv(8096) decoded_msg = get_data(msg) print decoded_msg send_msg = response_data('回复：'+decoded_msg) print send_msg client.send(send_msg) #client.send('%c%c%s' % (0x81, 4, 'zack')) except Exception as e: print e if __name__ == '__main__': run()
4.tonardo框架中websocket的使用
https://www.tornadoweb.org/en/stable/websocket.html?highlight=websocket
tornado.websocket.WebSocketHandler中封装的三个方法如下：
class EchoWebSocket(tornado.websocket.WebSocketHandler): def open(self): #客户端连接时执行 print("WebSocket opened") def on_message(self, message): #接收到客户端消息时执行 self.write_message(u"You said: " + message) def on_close(self): #断开连接时执行 print("WebSocket closed")
简单在线聊天室实现：
app.py
#coding:utf-8 import tornado.web import tornado.websocket import tornado.ioloop import uuid Users = set() class IndexHandler(tornado.web.RequestHandler): def get(self): self.render('index.html') class ChatHandler(tornado.websocket.WebSocketHandler): def open(self): self.id = str(uuid.uuid4()) Users.add(self) def on_message(self, message): for client in Users: content = client.render_string('message.html',id=self.id,msg=message) client.write_message(content) def on_close(self): delattr(self,'id') Users.remove(self) settings={ 'template_path':'templates', 'static_path':'statics', 'static_url_prefix':'/statics/', } app = tornado.web.Application([ (r'/',IndexHandler), (r'/chat',ChatHandler), ],**settings) if __name__ == '__main__': app.listen(8000) tornado.ioloop.IOLoop.instance().start()
index.html
<!DOCTYPE html> <html lang="zh-CN"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Title</title> <style> #content{ border:solid gray 2px; height:400px; margin:20px 0px 0px 100px; overflow: auto; } </style> </head> <body> <div style="width: 750px; margin: 0 auto"> <h3>websocket聊天室</h3> <div id="content" > </div> <div style="margin-left:100px"> <input type="text" id="msg"/> <button onclick="sendMsg();">发送</button> <button onclick="closeCon();">断开连接</button> </div> </div> <script src="/statics/jquery-3.3.1.min.js"></script> <script> var web = new WebSocket("ws://127.0.0.1:8000/chat"); web.onopen=function () { var newTag = document.createElement('div'); newTag.innerHTML='[连接成功]'; document.getElementById('content').appendChild(newTag); }; web.onerror=function (error) { console.log('Error:'+error); }; web.onmessage=function (event) { console.log(event); $('#content').append(event.data); //document.getElementById('content').append(event.data); 添加为字符窜，不是tag标签？ //document.getElementById('content').appendChild(event.data); 失败？ }; web.onclose=function () { var newTag = document.createElement('div'); newTag.innerHTML='[断开连接]'; document.getElementById('content').appendChild(newTag); }; function sendMsg() { var mstag = document.getElementById('msg'); web.send(mstag.value); mstag.value=''; }; function closeCon() { web.close(); var newTag = document.createElement('div'); newTag.innerHTML='[断开连接]'; document.getElementById('content').appendChild(newTag); }; </script> </body> </html>
message.html
<div style="margin: 20px; background-color: green">{{id}}:{{msg}}</div>
参考文章：
http://www.cnblogs.com/wupeiqi/p/6558766.html
https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_servers
https://www.cnblogs.com/aguncn/p/5059337.html
https://www.cnblogs.com/JetpropelledSnake/p/9033064.html
