One way of HRS happening is when two servers in a request chain do not agree on the body parsing. Back in 2005, Linhart et al. first documented HRS. However, only recently it gained a huge popularity thanks to the research of few researchers like James Kettle, Evan Custodio and Regis Leroy. They demonstrated how HRS can be weaponized for severe attacks including web cache poisoning, request hijacking and response queue poisoning. The animation below shows an example of how HRS works.

HRS Example

Basically what happens is a confusion created in a request (in the animation, the semicolon added to the Transfer-Encoding header) makes the reverse proxy unable to recognize the Transfer-Encoding header and therefore choose Content-Length for body parsing, while it is unable to deter the origin server from parsing the body with the Transfer-Encoding header. As a result, a request is smuggled into the request buffer of the origin server. Because the same connection is usually kept being used (for performance reasons) between the reverse proxy and backend servers, the next request coming from another user is contaminated by the smuggled request. All attacks originate from this contamination.

Our Research

To create a confusion between servers, the previous research had mainly focused on Transfer-Encoding and Content-Length headers as the subject of mutations (mostly simple ones like adding a single character). The main goal of our paper, “T-Reqs: HTTP Request Smuggling with Differential Fuzzing” which was recently published by ACM CCS 2021, is to search beyond these two headers for confusions with more complex mutations.

To reach the goal, we developed T-Reqs, a grammar-based HTTP fuzzer. This fuzzer generates valid HTTP requests from a user-defined grammar. The elements in the grammar can be marked to allow mutations. They can either be marked string-mutable or tree-mutable. String-mutable elements undergo character-level changes (e.g., “POST /abc HTTP/1.1” becomes “?OST /abc HTTP/1.1”), while tree-mutable ones are structurally transformed (e.g., “POST /abc HTTP/1.1” becomes “POST /abc /abc HTTP/1.1”). The number of mutations and the pool of elements and characters to be used in mutations are configurable among other things. The figure below shows further examples for both string (on the left) and tree (on the right) mutations.

T-Reqs mutation types

We tested popular HTTP servers with T-Reqs and found that mutations in all parts of a request including the request line, request headers and request body can cause body parsing discrepancies. Some of the successful mutation categories required multiple string mutations and some involved tree mutations. Please see the section 5.2 in the paper for more details.

We also found that significant portion of these discrepancies can actually be exploited for HRS. Again, mutations in all request parts were found to be successful in causing HRS. While we were testing each discrepancy for HRS, we collected information about the conditions affecting the success of body discrepancies to become HRS. We list these conditions and how they affected each discrepancy in section 5.3 in the paper.

We shared the findings of the paper with the security teams of tested HTTP servers. We worked together with most of them to provide additional data and help them mitigate the issues. We know at least one HTTP server has already included T-Reqs in its test tool set. You can check out T-Reqs here.