Curl request 'Expect: 100-Continue' Issues and Risks

https://support.airship.com/hc/en-us/articles/213492003--Expect-100-Continue-Issues-and-Risks

Last modified March 13, 2018 21:35

 

API POST requests sent using HTTP clients and with an Expect: 100-Continue request header are significantly more likely to fail that those without the Expect: 100-Continue header. 

How the Expect: 100-Continue Header Works

When Expect: 100-Continue is NOT present, HTTP follows approximately the following flow (from the client's point of view):

1. The request initiates a TCP connection to the server.
2. When the connection to the server is established, the full request--which includes both the request headers and the request body--is transmitted to the server.
3. The client waits for a response from the server (comprised of response headers and a response body).
4. If HTTP keep-alives are supported, the request is optionally repeated from step 2.

When the client is using the Expect: 100-Continue feature, the following events occur:

1. The request initiates a TCP connection to the server.
2. When the connection to the server is established, the request--including the headers, the Expect: 100-Continue header, without the request body--is then transmitted to the server.
3. The client then waits for a response from the server.
• If the status code is a final status code, using the prior steps above the client retries the request without Expect: 100-Continue header.
• If the status code is 100-Continue, the request body is sent to the server.
4. The client will then wait for a response from the server (comprised of response headers and a response body).
5. If HTTP keep-alives are supported, the request is optionally repeated from step 2.

Why use Expect: 100-Continue?

API POST requests that include the Expect: 100-Continue header save bandwidth between the client and the server, because the server can reject the API request before the request body is even transmitted. For API POST requests with very large request bodies (such as file uploads), the server can, for example, check for invalid authentication and reject the request before the push body was sent, resulting in significant bandwidth savings.

Without Expect: 100-Continue:

Without the Expect: 100-Continue feature, the entire API request, including the (potentially large) push body would have to be transmitted before the server could even determine if the syntax or authentication is valid. However, since the majority of our API requests have small POST bodies, the benefits of separating the request header from the request body is negligible. 

Problems when the request header and body are sent separately

Because of the high volume of requests that Urban Airship handles, many levels of complexity exist between our clients and the servers responsible for responding to API requests. This is not an abnormal phenomenon for most server configurations and strategies, but it does introduce a risk of elevated request failures to any API POST requests using the Expect: 100-Continue header. This is due to the fact that the request header and the request body are sent separately from one another, and must travel through the same connection throughout the entire API server infrastructure.

With the number of proxies, load-balancing servers, and back-end request processing servers that are implemented, requests with the Expect: 100-Continue header have an increased probability of becoming separated from one another, and hence returning with an error.

What To Expect:

We've always attempted to support Expect: 100-Continue. However, we have determined that our customers that use Expect: 100-Continue are receiving a sub-optimal quality of service due to elevated request failures.

Additionally, the majority of our API requests have small POST bodies, and as a result the benefits of separating the request header from the request body are negligible. These reasons have motivated us to disable support for Expect: 100-Continue service-wide.

Our Recommendations:

We recommend against the use of Expect: 100-Continue.

If you receive an HTTP Error 417 (Expectation failed), retry the request without Expect: 100-Continue.