Semantics of informational responses.

[ioquatix]

I would like to discuss how we may improve the clarity of the specification with respect to informational responses. Thanks in advance for everyone's efforts in discussing this matter.

Firstly, let's clarify, a response is a status code, a set of headers, and an optional body, as shown in https://www.rfc-editor.org/rfc/rfc9110#section-3.9. Furthermore, a response may use the 1xx status code, which is considered "Informational" and "Interim" and "prior to completing a final response".

The 1xx (Informational) class of status code indicates an interim response for communicating connection
status or request progress prior to completing the requested action and sending a final response.

By this definition, one may expect that responses with a 1xx status code will always be followed by a non-1xx final response. This is important for ensuring secure response handling with persistent connections. For example, a persistent connection is only secure if, after making a request, all responses are consumed relating to that request. As HTTP/1.x provides no additional request/response delineation, I believe it's important that we have a clear definition of "interim" and "final" response codes.

For the sake of the argument, a client may do the following:

connection = connection_pool.acquire

connection.send_request(method, path, headers, body)
while response = connection.read_response
  if response.final?
    process_final_response(response)
    break
  else
    process_interim_response(response)
  end
end

connection_pool.add(connection)

If, for whatever reason, the specification of final? is not well-defined, it would be entirely possible for the next user of the connection, to read a response from a previous request, causing a potentially serious security issue. Unfortunately HTTP request smuggling is a fairly common form of attack, usually due to ambiguity in request/response processing and interpretation.

My goal is to seek clarity regarding the definition of "final" response - as the author of HTTP/1, HTTP/2 and HTTP/3 (in progress) client and server implementations for Ruby, I have significant exposure to these specifications and implementation details. Recently, I was made aware that I was handling this incorrectly and sought out clarifications from the specifications (RFCs), but felt the wording of the informational responses does not reflect the reality of how they must be handled.

Specifically, 101 Switching Protocols does not appear to be an interim response, at least not in all cases, and certainly not with respect to the framing of the connection as per the above pseudo-code example. It's true that clients must handle 101 explicitly, but at the protocol/stream level, my implementation handles it the same as any other response (as per the above pseudo code, essentially). In that case, the 101 response MUST be returned to the client as a final response, so that it can complete the upgrade.

In other words, even thought the specifications state that responses with a 1xx status code will be followed by a "final response", this is not true for 101 Switching Protocols.

I believe that additional clarification should be provided around the 101 Switching Protocols status code. Perhaps under https://www.rfc-editor.org/rfc/rfc9110#section-15.2.2 we can state that "A response with a status code of 101 switching protocols, in the context of the original connection, is considered a final response, as no subsequent non-1xx response will be sent by the server."

To support this change, I present the following thoughts:

• HTTP/2 WebSockets uses CONNECT and expects a normal final response with status code 200 in order to complete the negotiation. I think this strongly suggests that the use of 101 in HTTP/1.1 for WebSockets, was at best, a stretch (hypothetically speaking, it feels more like 201 Switching Protocols).
• In my implementation of HTTP, I did not find any advantage to differentiate between HTTP/1 and HTTP/2 WebSockets (or HTTP/3). Although the version specific details of connection negotiation are a pain to implement, the actual interface for the user is identical (a bidirectional stream encapsulating the WebSocket protocol). Semantically, HTTP/1.1's 101 Switching Protocols and HTTP/2's 200 OK are the same for the sake of what the user actually cares about, and at the level of the client implementation, are considered the same "final response" before wrapping the underlying stream with WebSocket framing.
• I cannot find any example of a 101 Switching Protocols having a subsequent response at the same protocol level. It was used (but apparently deprecated) for upgrading HTTP/1 to HTTP/2 in the past, however, this "final response" is operating at a totally different semantic level to the original request protocol, so I don't feel that we should consider this example.
• I think the proposed clarification is essentially resolving an internal conflict of the specification, whereby 1xx responses should be followed by a final response. In other words, such a clarification seeks to resolve ambiguity but does not introduce any new semantics or definitions that aren't already present in the specification.
• Assuming that all 1xx responses will be non-final, in general, may affect the secure development of HTTP/1.1 proxy servers. Who is to say that future 1xx status codes wouldn't be introduced that are considered "final"? While it's true the various HTTP working groups consider this carefully, it's also true that there are significant security implications https://www.rfc-editor.org/rfc/rfc8297.html#section-3.

My hope, by clarifying this, is that I can be confident in saying "101 Switching Protocols" is, for all intents and purposes, considered a final response, with respect to the protocol and framing of the original connection. The specification is, in my humble opinion, currently, both at odds and in support of the above statement. I'd like to fix that.

[mnot]

Don't get too hung up on the high-level semantic categories of responses. While one could argue that 101 is more appropriately cast as a final response (e.g., 2xx), there are counter-arguments, including that the operation on the target resource never actually completes from a HTTP standpoint when the protocol switches.

101 does specify its semantics with precision. While new status codes are required to conform to the general requirements of the category they're in, there are many exceptions due to history -- see eg a very similar situation with regard to whether a status code can have content.

[MikeBishop]

I totally see your point -- 101 indicates that the protocol being spoken on the connection has changed, and the request's response will occur in the new protocol. As far as HTTP is concerned, it can be viewed as final, since there's no more HTTP to do; as far as the request is concerned, it's not, since the request is the starting point for the new protocol. (As you alluded to, this is reflected in the now-deprecated Upgrade: h2c case, where the server sends a response to the request in HTTP/2 framing following the 101.)

Personally, I think our mistake was in removing Upgrade/101 support in HTTP/2+; the CONNECT + :protocol hack is a workaround when we admitted that we actually did still need the ability to switch to a different protocol within a stream. Had we known that from the start, you might see Upgrade headers and 101 status codes still being used for WebSockets et al. That's not going to happen at this point.

Both Upgrade and CONNECT have semantics that don't sit well with the rest of HTTP, and unfortunately one just has to special-case them. I don't think changing the specs in this respect will make them fit any better, just differently.