HTTP

HTTP (the Hypertext Transfer Protocol) is a client-server protocol used for file exchange on the World Wide Web. HTTP is the means of communications between a web browser (a program which acts as an HTTP client) and a web server (a program which acts as an HTTP server). Both the HTTP request and response consist of a file, or array of bytes, in which each byte must be a member of the ASCII character set. Besides protocol overhead, the request optionally may contain one or more named parameters elaborating on the request, and the response optionally may contain an ASCII-text file "payload". By using character encoding algorithms, the response can effectively send binary information such as image or audio. When binary files are returned, the byte stream is encoded as ASCII text by the HTTP server for transport across the internet and must then be decoded back to its binary format upon reception by the HTTP client. Web browsers are not the only type of HTTP client program. HTTP is also used by search engines to index the World Wide Web, as well as by so-called spam-bots which scrape web pages to obtain information for malicious purposes.

The HTTP specifications

The two HTTP specifications in active use today contain precise, elegant language which is accessible to lay readers if they carefully consider the definitions provided near the top of the documents:

• W3C RFC 1945, Hypertext Transfer Protocol -- HTTP/1.0, which specifies how the browser and server communicate with each other
• W3C RFC 2616, Hypertext Transfer Protocol -- HTTP/1.1, which adds caching, proxies and keep-alive connections to the original specification

HTTP and the Transmission Control Protocol

HTTP requests and responses rely upon the Internet's Transmission Control Protocol (TCP) for error-free transmission between client programs and server programs. HTTP is one of several well-known TCP applications and is assigned the well-known TCP port number of 80. Thus, browsing to a URL such as http://diatom.ansp.org/ is actually equivalent to browsing to http://diatom.ansp.org:80/. If two web server programs were to execute simultaneously on a single host computer, it would be necessary for one of them to be configured to use a TCP port number above 8000 (since well-known TCP ports are supposed to be below 8000). A URL such as http://diatom.ansp.org:8080/ might be used, for example, to address an HTTP request to a second web server on a computer whose DNS name is diatom.ansp.org, assuming the second HTTP server program were configured to listen on TCP port 8080. If two web servers were to run simultaneously on one computer and both attempted to listen for requests on TCP port 80 (the HTTP default), neither would work correctly.

HTTP and URL's

HTTP requests and responses consist of several header lines and, optionally, a body. The location of the server program and the exact web page or file the client is requesting are both precisely identified in a Uniform Resource Identifier (URI).

Example conversation between an HTTP server and client

The following is a typical HTTP message exchange of the kind which is carried out every time a page is loaded in a web browser. In the following example, the user has entered the URL http://www.lth.se/ in a browser and clicked. The browser then sends an HTTP request, using TCP, to the computer whose DNS name is www.lth.se. The www.lth.se computer must have an HTTP server programming executing and configured to listen on TCP port 80. If this program is present, the TCP program on www.lth.se passes the HTTP request to the HTTP server program.

and the server responds with the web server's default page. Comments below are shown in italics and are not part of the actual conversation.

Request:

GET / HTTP/1.1                          Please transmit your root page, using HTTP 1.1,
Host: www.lth.se                        located at www.lth.se.
User-Agent: Mozilla/5.0 (Linux i686)    I am Mozilla 5.0, running on i686 Linux.
Accept: text/html                       I understand HTML-coded documents.
Accept-Language: sv, en-gb              I prefer pages in Swedish and British English.
Accept-Encoding: gzip, deflate          You may compress your content as gzip or deflate, if you wish.
Accept-Charset: utf-8, ISO-8859-1       I understand text encoded in Unicode and Latin-1.
                                        (Empty line signifies end of request)

Response:

HTTP/1.1 200 OK                         Request is valid; I am complying according to HTTP 1.1 (code 200).
Date: Wed, 26 May 2010 10:33:59         The time (where I am) is 10:33 [...].
Server: Apache/2.2.3 (Red Hat)          I am Apache 2.2.3, running on Red Hat Linux.
Content-Length: 54283                   The content you requested is 54,283 bytes long.
Content-Type: text/html; charset=utf-8  Prepare to receive text encoded in Unicode, to be interpreted as HTML.
<html>                                  (The webpage www.lth.se/ follows)
  <head>
    <title>Some page title</title>
...

These are a few additional example responses that could also occur:

Response:

HTTP/1.1 400 Bad Request                Request does not conform to HTTP/1.1, and I did not understand it.
                                        Please do not repeat the request in its current form.

Response:

HTTP/1.1 404 Not Found                  Request comforms with HTTP/1.1, but the resource requested was not found here.
                                        Please do not repeat the request for this resource.

Response:

HTTP/1.1 403 Forbidden                  Request is conformant and valid, but I refuse to comply under HTTP/1.1.
                                        Please do not repeat the request.

HTTP request methods

HTTP clients can use one of eight request methods:

• HEAD
• GET
• POST
• PUT
• DELETE
• TRACE
• OPTIONS
• CONNECT

In practice, it is mainly the GET, POST and HEAD methods that are used in web applications, although protocols like WebDAV make use of others.

Status codes

An HTTP response (sent by a server in reply to an HTTP request) includes a status header, which informs the client whether the request succeeded. The status header is made up of a "status code" and a "reason phrase" (descriptive text).

Status codes classes

Status codes are grouped into classes:

• 1xx (informational) : Request received, continuing process
• 2xx (success) : The action was successfully received, understood, and accepted
• 3xx (redirect) : Further action must be taken in order to complete the request
• 4xx (client error) : The request contains bad syntax or cannot be fulfilled
• 5xx (server error) : The server failed to fulfill an apparently valid request.

For example, if the client requests a non-existent document, the status code will be "404 Not Found".

According to the W3C consortium :

HTTP applications are not required to understand the meaning of all registered status codes, though such understanding is obviously desirable. However, applications MUST understand the class of any status code, as indicated by the first digit, and treat any unrecognized response as being equivalent to the x00 status code of that class, with the exception that an unrecognized response MUST NOT be cached.

All W3C status codes

All the codes are described in RFC 2616.

HTTP header and cache management

The HTTP message header includes a number of fields used to facilitate cache management. One of these, Etag (entity tag) is a string valued field that represents a value that should (weak entity tag) or must (strong entity tag) change whenever the page (or other resource) is modified. This allows browsers or other clients to determine whether or not the entire resource needs to be downloaded. The HEAD method, which returns the same message header that would be included in the response to a GET request, can be used to determine if a cached copy of the resource is up to date without actually downloading a new copy. Other elements of the message header can be used, for example, to indicate when a copy should expire (no longer be considered valid), or that it should not be cached at all. This can be useful, for example, when data is generated dynamically (for example, the number of visits to a web site).

HTTP server operations

Multiple virtual servers may map onto a single physical computer. For effective server use, they must be on networks engineered to handle the traffic with them; see port scanning for Internet Service Provider checking for servers placed where traffic can create problems.

History

HTTP was designed at CERN by Tim Berners-Lee in 1989 as a way to share hypertext documents. Around 1992 with the availability of the first web browser (Mosaic), HTTP and HTML together began to be used by other sites, primarily in the scientific world. The availability of the Mosaic web browser and the NCSA HTTPd web server, both developed at the National Center for Supercomputing Applications by Marc Andreessen, were key to the explosion in popularity of HTTP and HTML that followed.

The first (1990) version of HTTP, called HTTP/0.9, was a bare-bones protocol for raw data transfer across the Internet. HTTP/1.0 (1996) improved the protocol by allowing payload message to be in a self-describing language HTML, along with the addition of metadata about the location of the requested information and other directives on how to handle the request and response.

However, HTTP/1.0 was still limited, lacking web caches and persistent connections for repeated messaging, and virtual web servers. To meet those needs, HTTP/1.1 was developed.