Tomcat视角看一次http请求

一次HTTP请求

如上图，一次完整的HTT请求，大致包含如下步骤：

域名解析
TCP连接(三次握手)
建立连接后发送请求数据
服务器处理请求
发送响应数据
浏览器解析响应数据
浏览器渲染前端页面

http请求过程看似简单，但如果你对任何事情都抱有一定的好奇心，要从底层根本性的了解以上各个环节，其实及其复杂，需要的知识量也很丰富，基本涵盖了你大学学的所有专业科目(不信？后面我们慢慢来看)。今天我们站着Tomcat的视角，去了解http请求在Tomcat中到底经历了些什么。

我们将围绕以下步骤进行：Tomcat接收请求、请求数据读取、容器处理业务逻辑、多次请求如何衔接、简单响应等。

底层概念介绍

1、关于网络

　　我们所熟知的中间件(mysql、redis、kafka、rabbitmq、dubbo等)其实都是网络上的一个运行的服务，它们从客服端接收数据，通过复杂的逻辑处理后返回客户端。Tomcat当然也不例外，它可以接收来自客户端的多种协议的请求，包括：HTTP、APR等。这里的接收请求和返回请求，自然就涉及到网络编程，在java中网络编程被JDK封装成了Socket接口，我们可以把tomcat理解成一个启动后就一直在运行的Socket服务端程序，它在不停的监听来自外界的请求(其实是TCP请求，是否理解了不管你是http还是其他应用层协议，其实底层都是TCP请求？)。熟悉Socket的同学肯定要问了，那Tomcat是BIO还是NIO呢？没错，只要是网络传输，就逃不掉这个问题。这里先简单说下加，早期的Tomcat是BIO的，具体的版本是7.0(包括)之前都是默认BIO，支持配置成NIO，8.0开始全面默认NIO。当然8.0后的NIO和之前的NIO是有区别的，后续文章我会提到。BIO模型处理流程如下图：

2、底层数据存储

　　数据通过网络传输到Tomcat所在的服务器，首先会在操作系统缓冲区revbuf中，然后tomcat会从revbuf中读取数据，最后存放在自身JVM中。这个步骤看似简单，但是确实现代操作系统的一个经典理论。操作系统和tomcat存放数据都是使用的字节数组，我们仅以Tomcat中字节数据buf为例：

数据到达Tomcat中后，都会经历以上的字节数组，而后再封装成request对象传递给容器。

源码分析

1、Acceptor

 1 /**
 2      * The background thread that listens for incoming TCP/IP connections and
 3      * hands them off to an appropriate processor.
 4      */
 5     /**
 6      * 连接接收类，用于接受来自客户端的所有请求
 7      */
 8     protected class Acceptor extends AbstractEndpoint.Acceptor {
 9 
10         @Override
11         public void run() {
12 
13             int errorDelay = 0;
14 
15             // Loop until we receive a shutdown command
16             while (running) {
17 
18                 // Loop if endpoint is paused
19                 // 内层循环，暂时中断处理
20                 while (paused && running) {
21                     state = AcceptorState.PAUSED;
22                     try {
23                         Thread.sleep(50);
24                     } catch (InterruptedException e) {
25                         // Ignore
26                     }
27                 }
28 
29                 //关闭
30                 if (!running) {
31                     break;
32                 }
33                 state = AcceptorState.RUNNING;
34 
35                 try {
36                     //if we have reached max connections, wait
37                     // 继承AQS，控制总连接数不能超过最大值
38                     countUpOrAwaitConnection();
39 
40                     Socket socket = null;
41                     try {
42                         // Accept the next incoming connection from the server
43                         // socket
44                         // 接收socket请求，最重要的入口！！！
45                         socket = serverSocketFactory.acceptSocket(serverSocket);
46                     } catch (IOException ioe) {
47                         countDownConnection();
48                         // Introduce delay if necessary
49                         errorDelay = handleExceptionWithDelay(errorDelay);
50                         // re-throw
51                         throw ioe;
52                     }
53                     // Successful accept, reset the error delay
54                     errorDelay = 0;
55 
56                     // Configure the socket
57                     if (running && !paused && setSocketOptions(socket)) {
58                         // Hand this socket off to an appropriate processor
59                         if (!processSocket(socket)) {
60                             countDownConnection();
61                             // Close socket right away
62                             closeSocket(socket);
63                         }
64                     } else {
65                         countDownConnection();
66                         // Close socket right away
67                         closeSocket(socket);
68                     }
69                 } catch (IOException x) {
70                     if (running) {
71                         log.error(sm.getString("endpoint.accept.fail"), x);
72                     }
73                 } catch (NullPointerException npe) {
74                     if (running) {
75                         log.error(sm.getString("endpoint.accept.fail"), npe);
76                     }
77                 } catch (Throwable t) {
78                     ExceptionUtils.handleThrowable(t);
79                     log.error(sm.getString("endpoint.accept.fail"), t);
80                 }
81             }
82             state = AcceptorState.ENDED;
83         }
84     }

Acceptor继承了AbstractEndpoint类的内部类Acceptor，这个Acceptor实现了Runnable接口，表明自己是一个多线程的任务类，可放入线程池。Tomcat启动后，外层Acceptor就开始执行run方法。

11行，实现了Runnable接口中的run方法，可以看见该run方法中是一个while循环，只有当running=false，循环才跳出。

45行，这里是关键，tomcat启动后，如果没有请求进来，当前主线程将在此处进行阻塞。直到第一个请求到来，便获取当前请求的socket。

59行，将得到的socekt交由processSocket()方法处理。

其他的代码都是一些异常情况的判断，这里忽略。

2、processSocket

 1 protected boolean processSocket(Socket socket) {
 2         // Process the request from this socket
 3         try {
 4             SocketWrapper<Socket> wrapper = new SocketWrapper<Socket>(socket);
 5             wrapper.setKeepAliveLeft(getMaxKeepAliveRequests());
 6             wrapper.setSecure(isSSLEnabled());
 7             // During shutdown, executor may be null - avoid NPE
 8             if (!running) {
 9                 return false;
10             }
11             // 将包装好的socketProcessor扔线程池
12             getExecutor().execute(new SocketProcessor(wrapper));
13         } catch (RejectedExecutionException x) {
14             log.warn("Socket processing request was rejected for:"+socket,x);
15             return false;
16         } catch (Throwable t) {
17             ExceptionUtils.handleThrowable(t);
18             // This means we got an OOM or similar creating a thread, or that
19             // the pool and its queue are full
20             log.error(sm.getString("endpoint.process.fail"), t);
21             return false;
22         }
23         return true;
24     }

process方法将socket包装成SocketWrapper，再设置为SocketProcessor的属性

12行，SocketProcessor类实现了Runnable接口，这里相当于将当前请求的socket扔到了线程池当中。

3、SocketProcessor

 1 /**
 2      * This class is the equivalent of the Worker, but will simply use in an
 3      * external Executor thread pool.
 4      */
 5     protected class SocketProcessor implements Runnable {
 6 
 7         protected SocketWrapper<Socket> socket = null;
 8         protected SocketStatus status = null;
 9 
10         public SocketProcessor(SocketWrapper<Socket> socket) {
11             if (socket==null) throw new NullPointerException();
12             this.socket = socket;
13         }
14 
15         public SocketProcessor(SocketWrapper<Socket> socket, SocketStatus status) {
16             this(socket);
17             this.status = status;
18         }
19 
20         @Override
21         public void run() {
22             boolean launch = false;
23             synchronized (socket) {
24                 try {
25                     SocketState state = SocketState.OPEN;
26 
27                     try {
28                         // SSL handshake
29                         // 这里处理HTTPS相关的逻辑
30                         serverSocketFactory.handshake(socket.getSocket());
31                     } catch (Throwable t) {
32                         ExceptionUtils.handleThrowable(t);
33                         if (log.isDebugEnabled()) {
34                             log.debug(sm.getString("endpoint.err.handshake"), t);
35                         }
36                         // Tell to close the socket
37                         state = SocketState.CLOSED;
38                     }
39 
40                     /**
41                      * handler处理socket
42                      */
43                     if ((state != SocketState.CLOSED)) {
44                         if (status == null) {
45                             state = handler.process(socket, SocketStatus.OPEN_READ);
46                         } else {
47                             state = handler.process(socket,status);
48                         }
49                     }
50                     if (state == SocketState.CLOSED) {
51                         // Close socket
52                         if (log.isTraceEnabled()) {
53                             log.trace("Closing socket:"+socket);
54                         }
55                         countDownConnection();
56                         try {
57                             socket.getSocket().close();
58                         } catch (IOException e) {
59                             // Ignore
60                         }
61                     } else if (state == SocketState.OPEN ||
62                             state == SocketState.UPGRADING ||
63                             state == SocketState.UPGRADING_TOMCAT  ||
64                             state == SocketState.UPGRADED){
65                         socket.setKeptAlive(true);
66                         socket.access();
67                         launch = true;
68                     } else if (state == SocketState.LONG) {
69                         socket.access();
70                         waitingRequests.add(socket);
71                     }
72                 } finally {
73                     if (launch) {
74                         try {
75                             getExecutor().execute(new SocketProcessor(socket, SocketStatus.OPEN_READ));
76                         } catch (RejectedExecutionException x) {
77                             log.warn("Socket reprocessing request was rejected for:"+socket,x);
78                             try {
79                                 //unable to handle connection at this time
80                                 handler.process(socket, SocketStatus.DISCONNECT);
81                             } finally {
82                                 countDownConnection();
83                             }
84 
85 
86                         } catch (NullPointerException npe) {
87                             if (running) {
88                                 log.error(sm.getString("endpoint.launch.fail"),
89                                         npe);
90                             }
91                         }
92                     }
93                 }
94             }
95             socket = null;
96             // Finish up this request
97         }
98 
99     }

这个类主要是处理买个socket连接中接收到的http请求

45行，将socket交由handler处理，其他的都是异常情况处理

4、handler

  1 public SocketState process(SocketWrapper<S> wrapper,
  2                 SocketStatus status) {
  3              23 
 24             try {
 25                 if (processor == null) {
 26                     processor = recycledProcessors.poll();
 27                 }
 28                 if (processor == null) {
 29                     processor = createProcessor();
 30                 }
 31 
 32                 initSsl(wrapper, processor);
 33 
 34                 SocketState state = SocketState.CLOSED;
 35                 do {
 36                     if (status == SocketStatus.DISCONNECT &&
 37                             !processor.isComet()) {
 38                         // Do nothing here, just wait for it to get recycled
 39                         // Don't do this for Comet we need to generate an end
 40                         // event (see BZ 54022)
 41                     } else if (processor.isAsync() || state == SocketState.ASYNC_END) {
 42                         state = processor.asyncDispatch(status);
 43                         if (state == SocketState.OPEN) {
 44                             // release() won't get called so in case this request
 45                             // takes a long time to process, remove the socket from
 46                             // the waiting requests now else the async timeout will
 47                             // fire
 48                             getProtocol().endpoint.removeWaitingRequest(wrapper);
 49                             // There may be pipe-lined data to read. If the data
 50                             // isn't processed now, execution will exit this
 51                             // loop and call release() which will recycle the
 52                             // processor (and input buffer) deleting any
 53                             // pipe-lined data. To avoid this, process it now.
 54                             state = processor.process(wrapper);
 55                         }
 56                     } else if (processor.isComet()) {
 57                         state = processor.event(status);
 58                     } else if (processor.getUpgradeInbound() != null) {
 59                         state = processor.upgradeDispatch();
 60                     } else if (processor.isUpgrade()) {
 61                         state = processor.upgradeDispatch(status);
 62                     } else {
 63                         // 处理socket
 64                         state = processor.process(wrapper);
 65                     }
 66 
 67                     if (processor.isAsync()) {
 68                         state = processor.asyncPostProcess();
 69                     }
 70 
 71                     if (state == SocketState.UPGRADING) {
 72                         // Get the HTTP upgrade handler
 73                         HttpUpgradeHandler httpUpgradeHandler =
 74                                 processor.getHttpUpgradeHandler();
 75                         // Release the Http11 processor to be re-used
 76                         release(wrapper, processor, false, false);
 77                         // Create the upgrade processor
 78                         processor = createUpgradeProcessor(
 79                                 wrapper, httpUpgradeHandler);
 80                         // Mark the connection as upgraded
 81                         wrapper.setUpgraded(true);
 82                         // Associate with the processor with the connection
 83                         connections.put(socket, processor);
 84                         // Initialise the upgrade handler (which may trigger
 85                         // some IO using the new protocol which is why the lines
 86                         // above are necessary)
 87                         // This cast should be safe. If it fails the error
 88                         // handling for the surrounding try/catch will deal with
 89                         // it.
 90                         httpUpgradeHandler.init((WebConnection) processor);
 91                     } else if (state == SocketState.UPGRADING_TOMCAT) {
 92                         // Get the UpgradeInbound handler
 93                         org.apache.coyote.http11.upgrade.UpgradeInbound inbound =
 94                                 processor.getUpgradeInbound();
 95                         // Release the Http11 processor to be re-used
 96                         release(wrapper, processor, false, false);
 97                         // Create the light-weight upgrade processor
 98                         processor = createUpgradeProcessor(wrapper, inbound);
 99                         inbound.onUpgradeComplete();
100                     }
101                     if (getLog().isDebugEnabled()) {
102                         getLog().debug("Socket: [" + wrapper +
103                                 "], Status in: [" + status +
104                                 "], State out: [" + state + "]");
105                     }
106                 } while (state == SocketState.ASYNC_END ||
107                         state == SocketState.UPGRADING ||
108                         state == SocketState.UPGRADING_TOMCAT);
109 
110                 if (state == SocketState.LONG) {
111                     // In the middle of processing a request/response. Keep the
112                     // socket associated with the processor. Exact requirements
113                     // depend on type of long poll
114                     connections.put(socket, processor);
115                     longPoll(wrapper, processor);
116                 } else if (state == SocketState.OPEN) {
117                     // In keep-alive but between requests. OK to recycle
118                     // processor. Continue to poll for the next request.
119                     connections.remove(socket);
120                     release(wrapper, processor, false, true);
121                 } else if (state == SocketState.SENDFILE) {
122                     // Sendfile in progress. If it fails, the socket will be
123                     // closed. If it works, the socket either be added to the
124                     // poller (or equivalent) to await more data or processed
125                     // if there are any pipe-lined requests remaining.
126                     connections.put(socket, processor);
127                 } else if (state == SocketState.UPGRADED) {
128                     // Need to keep the connection associated with the processor
129                     connections.put(socket, processor);
130                     // Don't add sockets back to the poller if this was a
131                     // non-blocking write otherwise the poller may trigger
132                     // multiple read events which may lead to thread starvation
133                     // in the connector. The write() method will add this socket
134                     // to the poller if necessary.
135                     if (status != SocketStatus.OPEN_WRITE) {
136                         longPoll(wrapper, processor);
137                     }
138                 } else {
139                     // Connection closed. OK to recycle the processor. Upgrade
140                     // processors are not re-used but recycle is called to clear
141                     // references.
142                     connections.remove(socket);
143                     if (processor.isUpgrade()) {
144                         processor.getHttpUpgradeHandler().destroy();
145                         processor.recycle(true);
146                     } else if (processor instanceof org.apache.coyote.http11.upgrade.UpgradeProcessor) {
147                         // NO-OP
148                     } else {
149                         release(wrapper, processor, true, false);
150                     }
151                 }
152                 return state;
153             } catch(java.net.SocketException e) {
154                 // SocketExceptions are normal
155                 getLog().debug(sm.getString(
156                         "abstractConnectionHandler.socketexception.debug"), e);
157             } catch (java.io.IOException e) {
158                 // IOExceptions are normal
159                 getLog().debug(sm.getString(
160                         "abstractConnectionHandler.ioexception.debug"), e);
161             }
162             // Future developers: if you discover any other
163             // rare-but-nonfatal exceptions, catch them here, and log as
164             // above.
165             catch (OutOfMemoryError oome) {
166                 // Try and handle this here to give Tomcat a chance to close the
167                 // connection and prevent clients waiting until they time out.
168                 // Worst case, it isn't recoverable and the attempt at logging
169                 // will trigger another OOME.
170                 getLog().error(sm.getString("abstractConnectionHandler.oome"), oome);
171             } catch (Throwable e) {
172                 ExceptionUtils.handleThrowable(e);
173                 // any other exception or error is odd. Here we log it
174                 // with "ERROR" level, so it will show up even on
175                 // less-than-verbose logs.
176                 getLog().error(
177                         sm.getString("abstractConnectionHandler.error"), e);
178             }
179             // Make sure socket/processor is removed from the list of current
180             // connections
181             connections.remove(socket);
182             // Don't try to add upgrade processors back into the pool
183             if (!(processor instanceof org.apache.coyote.http11.upgrade.UpgradeProcessor)
184                     && !processor.isUpgrade()) {
185                 release(wrapper, processor, true, false);
186             }
187             return SocketState.CLOSED;
188         }

68行，继续将socket往下传

  1 public SocketState process(SocketWrapper<S> socketWrapper)
  2         throws IOException {
  3         RequestInfo rp = request.getRequestProcessor();
  4         rp.setStage(org.apache.coyote.Constants.STAGE_PARSE);
  5 
  6         // Setting up the I/O
  7         setSocketWrapper(socketWrapper);
  8         /**
  9          * 设置socket的InputStream和OutStream，供后面读取数据和响应使用
 10          */
 11         getInputBuffer().init(socketWrapper, endpoint);
 12         getOutputBuffer().init(socketWrapper, endpoint);
 13 
 14         // Flags
 15         keepAlive = true;
 16         comet = false;
 17         openSocket = false;
 18         sendfileInProgress = false;
 19         readComplete = true;
 20         if (endpoint.getUsePolling()) {
 21             keptAlive = false;
 22         } else {
 23             keptAlive = socketWrapper.isKeptAlive();
 24         }
 25 
 26         /**
 27          * 长连接相关，判断当前socket是否继续处理接下来的请求
 28          */
 29         if (disableKeepAlive()) {
 30             socketWrapper.setKeepAliveLeft(0);
 31         }
 32 
 33         /**
 34          * 处理socket中的请求，在长连接的模式下，会一直执行当前循环
 35          */
 36         while (!getErrorState().isError() && keepAlive && !comet && !isAsync() &&
 37                 upgradeInbound == null &&
 38                 httpUpgradeHandler == null && !endpoint.isPaused()) {
 39 
 40             // Parsing the request header
 41             try {
 42                 /**
 43                  * 1、设置socket超时时间
 44                  * 2、第一次从socket中读取数据
 45                  */
 46                 setRequestLineReadTimeout();
 47 
 48                 // 读取HTTP请求行数据
 49                 if (!getInputBuffer().parseRequestLine(keptAlive)) {
 50                     if (handleIncompleteRequestLineRead()) {
 51                         break;
 52                     }
 53                 }
 54 
 55                 // Process the Protocol component of the request line
 56                 // Need to know if this is an HTTP 0.9 request before trying to
 57                 // parse headers.
 58                 prepareRequestProtocol();
 59 
 60                 if (endpoint.isPaused()) {
 61                     // 503 - Service unavailable
 62                     response.setStatus(503);
 63                     setErrorState(ErrorState.CLOSE_CLEAN, null);
 64                 } else {
 65                     keptAlive = true;
 66                     // Set this every time in case limit has been changed via JMX
 67                     // 设置请求行和请求头大小，注意，这个很重要！
 68                     request.getMimeHeaders().setLimit(endpoint.getMaxHeaderCount());
 69                     // 设置做多可设置cookie数量
 70                     request.getCookies().setLimit(getMaxCookieCount());
 71                     // Currently only NIO will ever return false here
 72                     // Don't parse headers for HTTP/0.9
 73                     if (!http09 && !getInputBuffer().parseHeaders()) {
 74                         // We've read part of the request, don't recycle it
 75                         // instead associate it with the socket
 76                         openSocket = true;
 77                         readComplete = false;
 78                         break;
 79                     }
 80                     if (!disableUploadTimeout) {
 81                         setSocketTimeout(connectionUploadTimeout);
 82                     }
 83                 }
 84             } catch (IOException e) {
 85                 if (getLog().isDebugEnabled()) {
 86                     getLog().debug(
 87                             sm.getString("http11processor.header.parse"), e);
 88                 }
 89                 setErrorState(ErrorState.CLOSE_NOW, e);
 90                 break;
 91             } catch (Throwable t) {
 92                 ExceptionUtils.handleThrowable(t);
 93                 UserDataHelper.Mode logMode = userDataHelper.getNextMode();
 94                 if (logMode != null) {
 95                     String message = sm.getString(
 96                             "http11processor.header.parse");
 97                     switch (logMode) {
 98                         case INFO_THEN_DEBUG:
 99                             message += sm.getString(
100                                     "http11processor.fallToDebug");
101                             //$FALL-THROUGH$
102                         case INFO:
103                             getLog().info(message, t);
104                             break;
105                         case DEBUG:
106                             getLog().debug(message, t);
107                     }
108                 }
109                 // 400 - Bad Request
110                 response.setStatus(400);
111                 setErrorState(ErrorState.CLOSE_CLEAN, t);
112                 getAdapter().log(request, response, 0);
113             }
114 
115             if (!getErrorState().isError()) {
116                 // Setting up filters, and parse some request headers
117                 rp.setStage(org.apache.coyote.Constants.STAGE_PREPARE);
118                 try {
119                     prepareRequest();
120                 } catch (Throwable t) {
121                     ExceptionUtils.handleThrowable(t);
122                     if (getLog().isDebugEnabled()) {
123                         getLog().debug(sm.getString(
124                                 "http11processor.request.prepare"), t);
125                     }
126                     // 500 - Internal Server Error
127                     response.setStatus(500);
128                     setErrorState(ErrorState.CLOSE_CLEAN, t);
129                     getAdapter().log(request, response, 0);
130                 }
131             }
132 
133             if (maxKeepAliveRequests == 1) {
134                 keepAlive = false;
135             } else if (maxKeepAliveRequests > 0 &&
136                     socketWrapper.decrementKeepAlive() <= 0) {
137                 keepAlive = false;
138             }
139 
140             // Process the request in the adapter
141             if (!getErrorState().isError()) {
142                 try {
143                     rp.setStage(org.apache.coyote.Constants.STAGE_SERVICE);
144                     /**
145                      * 将封装好的请求和响应对象,交由容器处理
146                      * service-->host-->context-->wrapper-->servlet
147                      * 这里非常重要，我们所写的servlet代码正是这里在调用，它遵循了Servlet规范
148                      * 这里处理完，代表程序员开发的servlet已经执行完毕
149                      */
150                     adapter.service(request, response);
151                     // Handle when the response was committed before a serious
152                     // error occurred.  Throwing a ServletException should both
153                     // set the status to 500 and set the errorException.
154                     // If we fail here, then the response is likely already
155                     // committed, so we can't try and set headers.
156                     if(keepAlive && !getErrorState().isError() && (
157                             response.getErrorException() != null ||
158                                     (!isAsync() &&
159                                     statusDropsConnection(response.getStatus())))) {
160                         setErrorState(ErrorState.CLOSE_CLEAN, null);
161                     }
162                     setCometTimeouts(socketWrapper);
163                 } catch (InterruptedIOException e) {
164                     setErrorState(ErrorState.CLOSE_NOW, e);
165                 } catch (HeadersTooLargeException e) {
166                     getLog().error(sm.getString("http11processor.request.process"), e);
167                     // The response should not have been committed but check it
168                     // anyway to be safe
169                     if (response.isCommitted()) {
170                         setErrorState(ErrorState.CLOSE_NOW, e);
171                     } else {
172                         response.reset();
173                         response.setStatus(500);
174                         setErrorState(ErrorState.CLOSE_CLEAN, e);
175                         response.setHeader("Connection", "close"); // TODO: Remove
176                     }
177                 } catch (Throwable t) {
178                     ExceptionUtils.handleThrowable(t);
179                     getLog().error(sm.getString("http11processor.request.process"), t);
180                     // 500 - Internal Server Error
181                     response.setStatus(500);
182                     setErrorState(ErrorState.CLOSE_CLEAN, t);
183                     getAdapter().log(request, response, 0);
184                 }
185             }
186 
187             // Finish the handling of the request
188             rp.setStage(org.apache.coyote.Constants.STAGE_ENDINPUT);
189 
190             if (!isAsync() && !comet) {
191                 if (getErrorState().isError()) {
192                     // If we know we are closing the connection, don't drain
193                     // input. This way uploading a 100GB file doesn't tie up the
194                     // thread if the servlet has rejected it.
195                     getInputBuffer().setSwallowInput(false);
196                 } else {
197                     // Need to check this again here in case the response was
198                     // committed before the error that requires the connection
199                     // to be closed occurred.
200                     checkExpectationAndResponseStatus();
201                 }
202                 /**
203                  * 当前请求收尾工作
204                  * 判断请求体是否读取完毕，没有则读取完毕，并修正pos
205                  */
206                 endRequest();
207             }
208 
209             rp.setStage(org.apache.coyote.Constants.STAGE_ENDOUTPUT);
210 
211             // If there was an error, make sure the request is counted as
212             // and error, and update the statistics counter
213             if (getErrorState().isError()) {
214                 response.setStatus(500);
215             }
216             request.updateCounters();
217 
218             if (!isAsync() && !comet || getErrorState().isError()) {
219                 if (getErrorState().isIoAllowed()) {
220                     /**
221                      * 根据修正完的pos和lastValid,初始化数组下标，以便继续处理下一次请求
222                      */
223                     getInputBuffer().nextRequest();
224                     getOutputBuffer().nextRequest();
225                 }
226             }
227 
228             if (!disableUploadTimeout) {
229                 if(endpoint.getSoTimeout() > 0) {
230                     setSocketTimeout(endpoint.getSoTimeout());
231                 } else {
232                     setSocketTimeout(0);
233                 }
234             }
235 
236             rp.setStage(org.apache.coyote.Constants.STAGE_KEEPALIVE);
237 
238             if (breakKeepAliveLoop(socketWrapper)) {
239                 break;
240             }
241         }
242 
243         rp.setStage(org.apache.coyote.Constants.STAGE_ENDED);
244 
245         if (getErrorState().isError() || endpoint.isPaused()) {
246             return SocketState.CLOSED;
247         } else if (isAsync() || comet) {
248             return SocketState.LONG;
249         } else if (isUpgrade()) {
250             return SocketState.UPGRADING;
251         } else if (getUpgradeInbound() != null) {
252             return SocketState.UPGRADING_TOMCAT;
253         } else {
254             if (sendfileInProgress) {
255                 return SocketState.SENDFILE;
256             } else {
257                 if (openSocket) {
258                     if (readComplete) {
259                         return SocketState.OPEN;
260                     } else {
261                         return SocketState.LONG;
262                     }
263                 } else {
264                     return SocketState.CLOSED;
265                 }
266             }
267         }
268     }

这个方法是关键中的关键，几乎大部分的逻辑都在这里

11-12行，获取socket的InputStream和OutputStream，这2个对象分别是从操作系统缓冲区中读取数据以及写入数据到缓冲区。

1 @Override
2     protected void init(SocketWrapper<Socket> socketWrapper,
3             AbstractEndpoint<Socket> endpoint) throws IOException {
4         inputStream = socketWrapper.getSocket().getInputStream();
5     }

46行，设置socket超时时间，以及第一次读取socket中数据，这里可能只是读取部分数据，后面会多次读取。

49行，读取HTTP请求行数据

73行，读取请求头数据

150行，将封装好的request和response对象传给容器，最终给到servlet处理业务逻辑，这行执行完成，表明程序员开发的业务逻辑已经执行完毕

206行，当前请求收尾：如果servlet中没有读取获取只读取部分请求体，这里读取完毕，并修正下面pos值

223行，根据pos和lastValid值，初始化buf数组下下标：lastValid=lastValid-pos；pos=0

这几个步骤处理比较复杂，主要有以下几点：

1、while循环用于保证长连接情况下一个socket能处理多个HTTP请求。

2、buf数组用户存放读取自socket的所有请求数据，包括：HTTP请求行，请求头，以及请求体。演变过程如下

　　初始化大小为8KB的字节数组buf，下标pos=0,lastValid=0

每次都是从socket中读取一定长度的数据到buf数组，注意：一次读取的数据没有任何规律可言，可能只有请求行，也可能包括请求行和请求头，甚至还有可能包括请求体。

一个重点：HTTP请求GET方法，请求入参大小有限制。我们很多人大概都知道这个机制，可是并不知道，为啥又限制，以及限制到底是多大，其实Tomcat在读取请求数据的时候，分了两种情况

　　1、请求行和请求头

Tomcat读取请求行和请求头后，最终数组会变成

当请求头读完后，数组会包含两部分，如果读取的过程中，lastValid==buf.length，这时会抛出异常：iib.requestheadertoolarge.error！原来如此，其实Tomcat是想请求行和请求头一起打包放在数组中，只要大小不超过8KB，都OK，如果超过就抛异常，现在明白了啊，并不是GET方法请求入参大小有限制，当然，这里都是默认的大小，程序员可以自行修改配置。

　　2、读取请求体

上面演示了读取请求行和请求头的，它们只在一个buf数组中完成。而请求体不一样，我们知道，请求体的数据大小可以是任意的，因此Tomcat不能做任何限制

如果buf中剩余大小>4500，Tomcat每次回从socket中读取大小为buf.length-end的数据放入buf中，pos=end，lastValid=pos+实际读取数据的长度。注意这里每次存放的位置都是从end开始，可以看出请求体读取的时候是采用覆盖的方式，即上次读取的数据下个循环在读取的时候就被覆盖了。因此也可以看出请求体的读取如果业务代码不做保存，那就没法再次读取的，切记！

当buf数组中剩余大小不足4500时，Tomcat便会重新初始化一个新的数组赋给变量buf(原先的数组由于没有了引用，交由JVM垃圾回收)，在新的buf中完成请求体的读取，具体逻辑同上。

下面我们来欣赏一下数据读取的代码，感受下作者高深的编程思想

 1 protected boolean fill(boolean block) throws IOException {
 2 
 3         int nRead = 0;
 4 
 5         /**
 6          * 这个核心就是读取socket中数据到缓冲区buf中，循环读取，2种情况
 7          * 1、请求行和请求头：不能超过缓冲区大小(默认8kb)，如果超过，则抛异常，读完后将parsingHeader设置为false
 8          * 2、请求行：没有任何大小限制，循环读取，如果剩下的少于4500个字节，则会重新创建buf数组，从头开始读取，直到读完位置，注意！buf原先引用的数组们，等待GC
 9          */
10         if (parsingHeader) {
11 
12             /**
13              * 从socket中读取数据大于tomcat中缓冲区buf的长度，直接抛异常,这里有两点
14              * 1、这个就是我们很多时候很多人说的，get请求url不能过长的原因，其实是header和url等总大小不能超过8kb
15              * 2、这里的buf非常总要，它是InternalInputBuffer的属性，是一个字节数据，用户暂存从socket中读取的数据，比如：请求行，请求头、请求体
16              */
17             if (lastValid == buf.length) {
18                 throw new IllegalArgumentException
19                     (sm.getString("iib.requestheadertoolarge.error"));
20             }
21 
22             // 将socket中的数据读到缓冲区buf中，注意！这里就是BIO之所以难懂的关键所在，它会阻塞！
23             // 这个方法会阻塞，如果没有数据可读，则会一直阻塞，有数据，则移动lastValid位置
24             nRead = inputStream.read(buf, pos, buf.length - lastValid);
25             if (nRead > 0) {
26                 lastValid = pos + nRead;
27             }
28 
29         } else {
30 
31             if (buf.length - end < 4500) {
32                 // In this case, the request header was really large, so we allocate a
33                 // brand new one; the old one will get GCed when subsequent requests
34                 // clear all references
35                 buf = new byte[buf.length];
36                 end = 0;
37             }
38             pos = end;
39             lastValid = pos;
40             nRead = inputStream.read(buf, pos, buf.length - lastValid);
41             if (nRead > 0) {
42                 lastValid = pos + nRead;
43             }
44 
45         }
46         // 原则上这个方法要么阻塞着，要么就返回true
47         return (nRead > 0);
48 
49     }

　　3、请求收尾

如果请求是长连接，那么一次HTTP处理完成后，socket通道并未关闭，那么怎么样再次处理下个请求发送的数据呢？

我们可以想象，如果是长连接的，也就是客户端在发送上一个请求的请求体后，立马又发送了下一个请求的请求行。所以分两组情况：

1、最后一次读取请求体，刚好读完

pos=lastValid=0

　　2、最后一次读取请求体，顺便读出了部分下一次请求的数据

可以看出：首先要讲pos修正为请求体实际截止的位置，然后再初始化下标:lastValid=lastValid-pos，pos=0。

到这里一次请求几把已经结束，如果是长连接，便会再次进去while循环，继续读取客户端发送的数据；如果非长连接，则就关闭socket链接，请求结束！

总结

我们从Tomcat角度深入了解了HTTP请求的处理过程，其中涉及网络、操作系统、内存等知识。经过这次学习，我对以下几点有了更深的理解

1、何为应用层协议，HTTP请求的本质其实也是TCP传输

2、长连接的本质

3、Tomcat底层数据存储模式

4、HTTP请求是如果到达servlet的，这是很多MVC框架的基石

5、Tomcat作者过人的设计思想和高深的编程功底

posted @ 2020-05-17 22:20 蓝了个枫阅读(1286) 评论(0) 编辑收藏举报

刷新页面返回顶部