Understanding the Request Line

A Fiddler user recently sent me the following question:

Eric-- I noticed a “minor” difference between Raw and Hex Inspectors when examining requests in Fiddler. When I access www.microsoft.com, the Raw view looks like this:

    GET http://www.microsoft.com/ HTTP/1.1
    ...
    Host: www.microsoft.com

While the Hex view looks like this:

    GET / HTTP/1.1
    ...
    Host: www.microsoft.com

Where is this difference  in the Request Line coming from? I’ve noticed that if I use a socket to directly send a GET for the absolute URL, some servers return an error.

When a client like a browser believes that it is sending a request to a proxy, it always sends an absolute URL in the Request Line. When it thinks that it is sending its request directly to the server, it typically only sends only the path component of the URL in the Request Line. Under RFC2616 (Section 5.1.2), servers MUST accept either form, but as he noticed, some servers will not accept an absolute URL in the Request Line and return a HTTP/4xx or HTTP/5xx error code instead.

In the scenario above, the browser sends GET http://www.microsoft.com/ HTTP/1.1 to Fiddler and Fiddler sends GET / HTTP/1.1 when sending the received request directly to the server.

Fiddler’s Inspectors have the choice of displaying either the relative or absolute URL in the Request Line. Displaying the absolute URL simplifies request tampering, while displaying only the relative path more closely replicates what servers actually receive.