okhttp详解

okhttp详解

概论

okhttp是一个网络库，其功能主要有两点：

1. 请求的发起与响应的接收
2. 多个请求的管理

当前版本为4.9.0。

请求的发起与响应的接收

这个过程是U型的，如拆轮子系列：拆 OkHttp中的流程图所显示的：

先是向下一步一步地加工报文，到最底端将报文发送到服务端，然后拿到服务器返回的报文后，再向上一步一步将报文转化为开发者友好的Response。

请求报文

请求报文分为三个部分：

1. 请求行
2. 请求头
3. 请求体

请求行是对该请求的大致描述，请求头是对请求行的补充以及对请求体的约束，请求体是客户端想要传递给服务器的数据。

请求行

请求行分为三个部分，每个部分用空格隔开：

1. Http方法，一般为GET、POST、HEAD、PUT、DELETE、OPTIONS、TRACE、CONNECT；
2. 请求目标，就是url，例如https://www.baidu.com；
3. HTTP版本，例如HTTP/1.1

综合一下就是：

GET https://www.baidu.com HTTP/1.1

请求头

请求头的结构比较简单，以键值对的形式存在，以回车符和换行符隔开，形式如下：

Bdpagetype: 2
Bdqid: 0xed9518d40011b00b
Cache-Control: private
Connection: keep-alive
Content-Encoding: gzip
Content-Type: text/html;charset=utf-8
Date: Sun, 21 Feb 2021 13:35:58 GMT
Expires: Sun, 21 Feb 2021 13:35:58 GMT
Server: BWS/1.1
Set-Cookie: BDSVRTM=42; path=/
Set-Cookie: BD_HOME=1; path=/
Set-Cookie: H_PS_PSSID=33425_33506_33403_33344_31253_26350; path=/; domain=.baidu.com
Strict-Transport-Security: max-age=172800
Traceid: 1613914558067414836217119616857332101131
X-Ua-Compatible: IE=Edge,chrome=1

请求体

请求体比较灵活，没有固定的格式，可以根据服务的需要自定义格式；
但是有几种常见的格式：

1. Json

{
	"rate": 0.0867625,
	"displayable": true
}

服务器可以将这种数据格式转化为对应的数据类；

2. 表单 (application/x-www-form-urlencoded)

name=tom&password=1234&realme=tomson

3. 二进制表单(multipart/form-data)

------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="url"

https://www.baidu.com/
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="name"

waffle
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="desk"; filename="桌子.jpg"
Content-Type: image/jpeg

...contents of 桌子.jpg...

其中------WebKitFormBoundary7MA4YWxkTrZu0gW用于分割不同的字段；

响应报文

响应报文的格式与请求报文的格式十分相似，唯一不同的是响应行不同；

响应头分为三个部分，HTTP版本、状态码和状态码的文本描述；

HTTP/1.1 200 OK

关于状态码，请看这篇文章。

拦截器

val interceptors = mutableListOf<Interceptor>()
interceptors += client.interceptors
interceptors += RetryAndFollowUpInterceptor(client)
interceptors += BridgeInterceptor(client.cookieJar)
interceptors += CacheInterceptor(client.cache)
interceptors += ConnectInterceptor
if (!forWebSocket) {
    interceptors += client.networkInterceptors
}
interceptors += CallServerInterceptor(forWebSocket)

以上七种拦截器的具体功能稍后再说，先解释其具体的工作过程，一个请求是如何通过这一U型过程的。

下面这一段代码是请求的开端：

val chain = RealInterceptorChain(
        call = this,
        interceptors = interceptors,
        index = 0,
        exchange = null,
        request = originalRequest,
        connectTimeoutMillis = client.connectTimeoutMillis,
        readTimeoutMillis = client.readTimeoutMillis,
        writeTimeoutMillis = client.writeTimeoutMillis
    )

    var calledNoMoreExchanges = false
    try {
      val response = chain.proceed(originalRequest)
      if (isCanceled()) {
        response.closeQuietly()
        throw IOException("Canceled")
      }
      return response
    } catch (e: IOException) {
      calledNoMoreExchanges = true
      throw noMoreExchanges(e) as Throwable
    } finally {
      if (!calledNoMoreExchanges) {
        noMoreExchanges(null)
      }
    }

下面这一段代码是RealInterceptorChain::proceed的具体内容；

override fun proceed(request: Request): Response {
    check(index < interceptors.size)

    calls++

    if (exchange != null) {
      check(exchange.finder.sameHostAndPort(request.url)) {
        "network interceptor ${interceptors[index - 1]} must retain the same host and port"
      }
      check(calls == 1) {
        "network interceptor ${interceptors[index - 1]} must call proceed() exactly once"
      }
    }

    // index 代表了当前调用链中应该使用拦截器的下标
    // 因此 next 代表了调用下一个拦截器的Chain
    val next = copy(index = index + 1, request = request)
    // 获取当前环节所需拦截器
    val interceptor = interceptors[index]

    @Suppress("USELESS_ELVIS")
    val response = interceptor.intercept(next) ?: throw NullPointerException(
        "interceptor $interceptor returned null")

    if (exchange != null) {
      check(index + 1 >= interceptors.size || next.calls == 1) {
        "network interceptor $interceptor must call proceed() exactly once"
      }
    }

    check(response.body != null) { "interceptor $interceptor returned a response with no body" }

    return response
  }

下面是Interceptor的基本结构：

override fun intercept(chain: Interceptor.Chain): Response{
    // 对Request的处理加工
    val response = chain.proceed(request)
    // 对Response的处理加工
    
    return response
}

通过这种责任链的设计模式，保证了报文的处理过程。

client.interceptors

这是用户自定义的拦截器，默认是空的列表；

RetryAndFollowUpInterceptor

此拦截器并没有对Request的加工，只有对Response的加工；

对Response的加工分为两个部分：

1. 失败重试

try {
    response = realChain.proceed(request)
    newExchangeFinder = true
} catch (e: RouteException) {
      // The attempt to connect via a route failed. The request will not have been sent.
      if (!recover(e.lastConnectException, call, request, requestSendStarted = false)) {
        throw e.firstConnectException.withSuppressed(recoveredFailures)
      } else {
        recoveredFailures += e.firstConnectException
      }
      newExchangeFinder = false
      continue
} catch (e: IOException) {
      // An attempt to communicate with a server failed. The request may have been sent.
      if (!recover(e, call, request, requestSendStarted = e !is ConnectionShutdownException)) {
        throw e.withSuppressed(recoveredFailures)
      } else {
         recoveredFailures += e
      }
      newExchangeFinder = false
       continue
  }

private fun recover(
    e: IOException,
    call: RealCall,
    userRequest: Request,
    requestSendStarted: Boolean
  ): Boolean {
    // The application layer has forbidden retries.
    if (!client.retryOnConnectionFailure) return false

    // We can't send the request body again.
    if (requestSendStarted && requestIsOneShot(e, userRequest)) return false

    // This exception is fatal.
    if (!isRecoverable(e, requestSendStarted)) return false

    // No more routes to attempt.
    if (!call.retryAfterFailure()) return false

    // For failure recovery, use the same route selector with a new connection.
    return true
  }

在处理过程中出现异常且不满足重试条件的，直接抛出异常，不再重试；

2. 重定向

 // Attach the prior response if it exists. Such responses never have a body.
 if (priorResponse != null) {
   response = response.newBuilder()
       .priorResponse(priorResponse.newBuilder()
    .body(null)
    .build())
       .build()
 }

 val exchange = call.interceptorScopedExchange
// 根据状态码，生成一个实例，如果不是重定向，则返回null
 val followUp = followUpRequest(response, exchange)

// 不是重定向直接返回response
 if (followUp == null) {
   if (exchange != null && exchange.isDuplex) {
     call.timeoutEarlyExit()
   }
   closeActiveExchange = false
   return response
 }

// 是重定向，但是只能发起一次请求，同样直接返回response
 val followUpBody = followUp.body
 if (followUpBody != null && followUpBody.isOneShot()) {
   closeActiveExchange = false
   return response
 }

 response.body?.closeQuietly()

// 重定向次数大于20，抛出异常
 if (++followUpCount > MAX_FOLLOW_UPS) {
   throw ProtocolException("Too many follow-up requests: $followUpCount")
 }

// 进行request的刷新，进入下一次循环
 request = followUp
 priorResponse = response

BridgeInterceptor

1. 根据Request的内容添加请求头

val userRequest = chain.request()
    val requestBuilder = userRequest.newBuilder()

    val body = userRequest.body
    if (body != null) {
      val contentType = body.contentType()
      if (contentType != null) {
        requestBuilder.header("Content-Type", contentType.toString())
      }

      val contentLength = body.contentLength()
      if (contentLength != -1L) {
        requestBuilder.header("Content-Length", contentLength.toString())
        requestBuilder.removeHeader("Transfer-Encoding")
      } else {
        requestBuilder.header("Transfer-Encoding", "chunked")
        requestBuilder.removeHeader("Content-Length")
      }
    }

    if (userRequest.header("Host") == null) {
      requestBuilder.header("Host", userRequest.url.toHostHeader())
    }

    if (userRequest.header("Connection") == null) {
      requestBuilder.header("Connection", "Keep-Alive")
    }

    // If we add an "Accept-Encoding: gzip" header field we're responsible for also decompressing
    // the transfer stream.
    var transparentGzip = false
    if (userRequest.header("Accept-Encoding") == null && userRequest.header("Range") == null) {
      transparentGzip = true
      requestBuilder.header("Accept-Encoding", "gzip")
    }

    val cookies = cookieJar.loadForRequest(userRequest.url)
    if (cookies.isNotEmpty()) {
      requestBuilder.header("Cookie", cookieHeader(cookies))
    }

    if (userRequest.header("User-Agent") == null) {
      requestBuilder.header("User-Agent", userAgent)
    }

2. 获得Response后保留cookie，并且解码响应体

    val networkResponse = chain.proceed(requestBuilder.build())

    cookieJar.receiveHeaders(userRequest.url, networkResponse.headers)

    val responseBuilder = networkResponse.newBuilder()
        .request(userRequest)

    // 对gzip的响应体进行解码
    if (transparentGzip &&
        "gzip".equals(networkResponse.header("Content-Encoding"), ignoreCase = true) &&
        networkResponse.promisesBody()) {
      val responseBody = networkResponse.body
      if (responseBody != null) {
        val gzipSource = GzipSource(responseBody.source())
        val strippedHeaders = networkResponse.headers.newBuilder()
            .removeAll("Content-Encoding")
            .removeAll("Content-Length")
            .build()
        responseBuilder.headers(strippedHeaders)
        val contentType = networkResponse.header("Content-Type")
        responseBuilder.body(RealResponseBody(contentType, -1L, gzipSource.buffer()))
      }
    }

    return responseBuilder.build()

CacheInterceptor

1. 根据Resquest获取缓存中的Response，如果缓存有效则直接返回；如果无效则向下执行；

    val call = chain.call()
    val cacheCandidate = cache?.get(chain.request())

    val now = System.currentTimeMillis()

    // CacheStrategy 有两个字段，networkRequest和cacheResponse
    // 当cacheResponse不为空时，缓存有效；
    // 当networkRequest不为空时，缓存无效；
    val strategy = CacheStrategy.Factory(now, chain.request(), cacheCandidate).compute()
    val networkRequest = strategy.networkRequest
    val cacheResponse = strategy.cacheResponse

    cache?.trackResponse(strategy)
    val listener = (call as? RealCall)?.eventListener ?: EventListener.NONE

    if (cacheCandidate != null && cacheResponse == null) {
      // The cache candidate wasn't applicable. Close it.
      cacheCandidate.body?.closeQuietly()
    }

    // If we're forbidden from using the network and the cache is insufficient, fail.
    if (networkRequest == null && cacheResponse == null) {
      return Response.Builder()
          .request(chain.request())
          .protocol(Protocol.HTTP_1_1)
          .code(HTTP_GATEWAY_TIMEOUT)
          .message("Unsatisfiable Request (only-if-cached)")
          .body(EMPTY_RESPONSE)
          .sentRequestAtMillis(-1L)
          .receivedResponseAtMillis(System.currentTimeMillis())
          .build().also {
            listener.satisfactionFailure(call, it)
          }
    }

    // 缓存有效，直接返回
    if (networkRequest == null) {
      return cacheResponse!!.newBuilder()
          .cacheResponse(stripBody(cacheResponse))
          .build().also {
            listener.cacheHit(call, it)
          }
    }

    // 缓存命中记录
    if (cacheResponse != null) {
      listener.cacheConditionalHit(call, cacheResponse)
    } else if (cache != null) {
      listener.cacheMiss(call)
    }

2. 发起请求，更新缓存

    var networkResponse: Response? = null
    try {
      networkResponse = chain.proceed(networkRequest)
    } finally {
      // If we're crashing on I/O or otherwise, don't leak the cache body.
      if (networkResponse == null && cacheCandidate != null) {
        cacheCandidate.body?.closeQuietly()
      }
    }

    // If we have a cache response too, then we're doing a conditional get.
    if (cacheResponse != null) {
      if (networkResponse?.code == HTTP_NOT_MODIFIED) {
        val response = cacheResponse.newBuilder()
            .headers(combine(cacheResponse.headers, networkResponse.headers))
            .sentRequestAtMillis(networkResponse.sentRequestAtMillis)
            .receivedResponseAtMillis(networkResponse.receivedResponseAtMillis)
            .cacheResponse(stripBody(cacheResponse))
            .networkResponse(stripBody(networkResponse))
            .build()

        networkResponse.body!!.close()

        // Update the cache after combining headers but before stripping the
        // Content-Encoding header (as performed by initContentStream()).
        cache!!.trackConditionalCacheHit()
        cache.update(cacheResponse, response)
        return response.also {
          listener.cacheHit(call, it)
        }
      } else {
        cacheResponse.body?.closeQuietly()
      }
    }

    val response = networkResponse!!.newBuilder()
        .cacheResponse(stripBody(cacheResponse))
        .networkResponse(stripBody(networkResponse))
        .build()

    if (cache != null) {
      if (response.promisesBody() && CacheStrategy.isCacheable(response, networkRequest)) {
        // Offer this request to the cache.
        val cacheRequest = cache.put(response)
        return cacheWritingResponse(cacheRequest, response).also {
          if (cacheResponse != null) {
            // This will log a conditional cache miss only.
            listener.cacheMiss(call)
          }
        }
      }

      if (HttpMethod.invalidatesCache(networkRequest.method)) {
        try {
          cache.remove(networkRequest)
        } catch (_: IOException) {
          // The cache cannot be written.
        }
      }
    }

    return response

ConnectInterceptor

这个拦截器只做了一件事：打开客户端到服务端的Socket连接；

val realChain = chain as RealInterceptorChain
val exchange = realChain.call.initExchange(chain)
val connectedChain = realChain.copy(exchange = exchange)
return connectedChain.proceed(realChain.request)

可以看到其关键就在initExchange函数上；

其主要调用栈如下：

graph TD; a(RealCall::initExchange) --> b(ExchangeFinder::find) b--> c(ExchangeFinder::findHealthyConnection) c --> d(ExchangeFinder::findConnection)

1. RealCall::initExchange获得一个Exchange用来发送请求和接收响应；
2. 在开始处理请求之前，我们需要ExchangeFinder::find来获取一个对报文编码和解码的工具类；
3. 但是其编码解码过程，又依赖于对服务器的连接；

寻找连接的代码如下：

  private fun findConnection(
    connectTimeout: Int,
    readTimeout: Int,
    writeTimeout: Int,
    pingIntervalMillis: Int,
    connectionRetryEnabled: Boolean
  ): RealConnection {
    if (call.isCanceled()) throw IOException("Canceled")

    // 复用call中connection
    val callConnection = call.connection // This may be mutated by releaseConnectionNoEvents()!
    if (callConnection != null) {
      var toClose: Socket? = null
      synchronized(callConnection) {
        if (callConnection.noNewExchanges || !sameHostAndPort(callConnection.route().address.url)) {
          toClose = call.releaseConnectionNoEvents()
        }
      }

      // If the call's connection wasn't released, reuse it. We don't call connectionAcquired() here
      // because we already acquired it.
      if (call.connection != null) {
        check(toClose == null)
        return callConnection
      }

      // The call's connection was released.
      toClose?.closeQuietly()
      eventListener.connectionReleased(call, callConnection)
    }

    // We need a new connection. Give it fresh stats.
    refusedStreamCount = 0
    connectionShutdownCount = 0
    otherFailureCount = 0

    // 尝试从连接池中获取
    if (connectionPool.callAcquirePooledConnection(address, call, null, false)) {
      val result = call.connection!!
      eventListener.connectionAcquired(call, result)
      return result
    }

    // Nothing in the pool. Figure out what route we'll try next.
    val routes: List<Route>?
    val route: Route
    if (nextRouteToTry != null) {
      // Use a route from a preceding coalesced connection.
      routes = null
      route = nextRouteToTry!!
      nextRouteToTry = null
    } else if (routeSelection != null && routeSelection!!.hasNext()) {
      // Use a route from an existing route selection.
      routes = null
      route = routeSelection!!.next()
    } else {
      // Compute a new route selection. This is a blocking operation!
      var localRouteSelector = routeSelector
      if (localRouteSelector == null) {
        localRouteSelector = RouteSelector(address, call.client.routeDatabase, call, eventListener)
        this.routeSelector = localRouteSelector
      }
      val localRouteSelection = localRouteSelector.next()
      routeSelection = localRouteSelection
      routes = localRouteSelection.routes

      if (call.isCanceled()) throw IOException("Canceled")

      // Now that we have a set of IP addresses, make another attempt at getting a connection from
      // the pool. We have a better chance of matching thanks to connection coalescing.
      if (connectionPool.callAcquirePooledConnection(address, call, routes, false)) {
        val result = call.connection!!
        eventListener.connectionAcquired(call, result)
        return result
      }

      route = localRouteSelection.next()
    }

    // Connect. Tell the call about the connecting call so async cancels work.
    val newConnection = RealConnection(connectionPool, route)
    call.connectionToCancel = newConnection
    try {
      newConnection.connect(
          connectTimeout,
          readTimeout,
          writeTimeout,
          pingIntervalMillis,
          connectionRetryEnabled,
          call,
          eventListener
      )
    } finally {
      call.connectionToCancel = null
    }
    call.client.routeDatabase.connected(newConnection.route())

    // If we raced another call connecting to this host, coalesce the connections. This makes for 3
    // different lookups in the connection pool!
    if (connectionPool.callAcquirePooledConnection(address, call, routes, true)) {
      val result = call.connection!!
      nextRouteToTry = route
      newConnection.socket().closeQuietly()
      eventListener.connectionAcquired(call, result)
      return result
    }

    synchronized(newConnection) {
      connectionPool.put(newConnection)
      call.acquireConnectionNoEvents(newConnection)
    }

    eventListener.connectionAcquired(call, newConnection)
    return newConnection
  }

其查找过程如下：

1. 尝试复用call.connection;
2. 尝试在连接池中查找；
3. 更换路由再次在连接池中查找；
4. 新建一个RealConnection；

networkInterceptors

自定义的拦截器，默认为空元素的列表；

CallServerInterceptor

虽然这是责任链的最后一步，但是能讲的确实不多，代码都相对浅显；

response = responseBuilder
          .request(request)
          .handshake(exchange.connection.handshake())
          .sentRequestAtMillis(sentRequestMillis)
          .receivedResponseAtMillis(System.currentTimeMillis())
          .build()

还有一些对特殊状态码的处理，不再赘述；

多个请求的管理

同步和异步

1. 同步请求过程

  override fun execute(): Response {
    // 对执行状态进行校验
    check(executed.compareAndSet(false, true)) { "Already Executed" }
    // 开始计算超时
    timeout.enter()
    // 调用请求开始回调
    callStart()
    try {
      // 将call添加到列表，详细分析见下面
      client.dispatcher.executed(this)
      // 实际执行请求，上面已详细讲述
      return getResponseWithInterceptorChain()
    } finally {
      // 将call从列表中移除
      client.dispatcher.finished(this)
    }
  }

2. 异步请求过程

  override fun enqueue(responseCallback: Callback) {
    // 对执行状态进行校验
    check(executed.compareAndSet(false, true)) { "Already Executed" }
    // 调用请求开始回调，这个调用感觉有点不对，应该在真正执行请求之前，这里只是在排队
    callStart()
    // 将call加入到排队队列
    client.dispatcher.enqueue(AsyncCall(responseCallback))
  }

3. Dispatcher

Dispatcher是Call的管理者；

下面是它的一些重要字段：

// 最大并发数量
var maxRequests = 64
// 每个域名的最大并发数量
var maxRequestsPerHost = 5
// 线程池，用于执行异步请求
val executorService: ExecutorService
get() {
   if (executorServiceOrNull == null) {
        executorServiceOrNull = ThreadPoolExecutor(0, Int.MAX_VALUE, 60, TimeUnit.SECONDS,
            SynchronousQueue(), threadFactory("$okHttpName Dispatcher", false))
      }
      return executorServiceOrNull!!
 }

  /** 准备执行的异步请求队列 */
  private val readyAsyncCalls = ArrayDeque<AsyncCall>()

  /** 正在执行的异步请求队列 */
  private val runningAsyncCalls = ArrayDeque<AsyncCall>()

  /** 正在执行的同步请求队列 */
  private val runningSyncCalls = ArrayDeque<RealCall>()

重要方法：

// 将异步请求添加到排队队列
internal fun enqueue(call: AsyncCall) {
    synchronized(this) {
      readyAsyncCalls.add(call)

      // 复用RealCall
      if (!call.call.forWebSocket) {
        val existingCall = findExistingCallWithHost(call.host)
        if (existingCall != null) call.reuseCallsPerHostFrom(existingCall)
      }
    }
    // 开始队列处理
    promoteAndExecute()
  }

private fun promoteAndExecute(): Boolean {
    // 确认当前线程没有被锁
    this.assertThreadDoesntHoldLock()

    // 此次准备执行的异步请求列表
    val executableCalls = mutableListOf<AsyncCall>()
    val isRunning: Boolean
    synchronized(this) {
      val i = readyAsyncCalls.iterator()
      while (i.hasNext()) {
        val asyncCall = i.next()
		
        // 大于阈值，停止添加
        if (runningAsyncCalls.size >= this.maxRequests) break // Max capacity.
        // 大于阈值，跳过这个请求
        if (asyncCall.callsPerHost.get() >= this.maxRequestsPerHost) continue // Host max capacity.
		
        // 从排队队列中移除
        i.remove()
        asyncCall.callsPerHost.incrementAndGet()
        // 添加到新的队列
        executableCalls.add(asyncCall)
        runningAsyncCalls.add(asyncCall)
      }
      isRunning = runningCallsCount() > 0
    }

    // 一个一个地丢到线程池里执行
    for (i in 0 until executableCalls.size) {
      val asyncCall = executableCalls[i]
      asyncCall.executeOn(executorService)
    }

    return isRunning
  }