Simple
HomeDesignConcurrency
Search



Feedback




Serving content through the HTTP/1.1 protocol involves the establishment of a network connection (typically TCP) to an origin server and issuing a HTTP request. This request is processed by the origin server and a response is prepared and transmitted over the same TCP channel. This client server model of communication forms the basis for all Web servers.

Background

Traditional designs include select based servers, such as BOA and Zeus, and thread per request (often thread per connection) servers, such as the Java Web Server and Apache (although Apache combines this with a fork technique). Observations on performance have shown that the select based servers perform better with servers like Zeus and thttpd demonstrating relatively high throughput. This is usually because the thread per connection servers spend much of the time waiting on I/O operations, which consumes resources. The design of Simple uses techniques found in both architectures. The thread per request technique is coupled with a polling mechinism, which prevents initial I/O blocking, thus increasing the performance and concurrency of the server.

The thread per connection policy often used for servers pre Java 1.4 is very wasteful of resources. Concurrency is often a problem with such servers, as the larger the number of clients the larger the number of threads competing for resources. This design also results in degradation in performance as the number of clients increases and can often result in a failure of the server. To avoid such problems Simple uses a thread per request technique, which allows it to use a fixed thread pool that can be optimized for the host OS. This enables Simple to work well for single CPU desktop machines and provides scalability for multiprocessor machines.

Parallelism

Establishing a thread per request policy for HTTP/1.1 servers is a difficult task to achieve if concurrency is to be optimized. It can be achieved using the connection close semantics used by HTTP/1.0 servers, however this does not take advantage of the pipelining capabilities of HTTP/1.1. Coupling the pipelining capabilities of HTTP/1.1 and the thread per request policy is especially difficult to achieve when control of the transaction is handed to a third party object, like a Java Servlet. This is because the Servlet object acquires an OutputStream object from the request and interacts with that independently of any policy the Servlet engine might have. Requiring the Servlet to observe the I/O policies of the host server reduces the transparency of the system. The problems that arise are ones regarding resource consumption. If the Servlet is writing dynamic content the Servlet engine must buffer the output so that when the transaction is finished it can set a HTTP Content-Length header specifying the length of the output. This allows the host server to maintain the pipeline, as the connection close semantics are not required to delimit the resulting content, see RFC 2616. If however the Servlet knows the length of the output and wishes to set the Content-Length header directly (avoiding the use of a large buffer) this introduces the problem of handing control to a third party object. Simple provides a monitoring mechanism which acts as a client interpreting the HTTP/1.1 message header before the response commits (the HTTP headers are written to the TCP channel). This enables the response to interpret the HTTP I/O so that if the third party object violates its contract, that is, its specified message delimiter, the server can take the appropriate action (prioritizing maintaining the pipeline).

The underlying monitoring of the request and response enables the server to provide a highly concurrent API. The Servlet Specification requires that the HTTP request is handled by the Servlet.service method alone. This means that once the service method has finished the response is committed and the content is sent. The Simple API provides the capability to hand control of the request or response to an asynchronous thread without any change in semantics. This makes the API highly concurrent throughout. This enables the concept of an active service to be used. Comparing this with the Servlet API would be to say that the Servlet could do the following without any change in semantics.

   public void service(ServletRequest req, ServletResponse resp) {
      RequestDispatcher service = req.getNamedDispatcher("name");
      new Thread(new Runnable(){
         public void run() {
            try {
               service.forward(req, resp);
            }catch(ServletException e){
               // ....
            }
         }
      }).start();
   }

This, according to the Java Servlet Specification, is clearly illegal, as the Servlet engine will attempt to close and commit the response once the Servlet.service method has finished. This type of processing is however completely supported in the Simple API, it requires only that the OutputStream issued is closed once the HTTP transaction has completed. The closure of the issued OutputStream within the Servlet API is discouraged because of the use of the RequestDispatcher.include technique. Closing the connection during an include would influence the intended output (although the ServletResponse can issue a proxy OutputStream that cannot be closed, this is a poor design practice). The use of active services has advantages when there are time consuming tasks that the service must perform, such as interactions with databases or applications in an n-tier architecture. It also complements the thread per request policy using a fixed thread pool. Communications with applications in an n-tier architecture, such as Enterprise Java Beans (EJB) and databases, means that a thread from the fixed pool is occupied with time consuming I/O, thus reducing the concurrency of the server. If however the service can operate in a different thread, then the thread from the pool can be recovered. This asynchronous processing however requires that the HTTP I/O is monitored. When the input and output have finished then the server must be able to dispatch the next request from the pipeline regardless of weather the service has finished.

The use of active services cannot be done according to the Java Servlet Specification. The Servlet.service method must finish before the next request on the pipeline can be processed. The Simple API uses the Service object as an alternative to the Servlet object for processing requests. The RequestDispatcher.forward technique can be used to forward the request to another Service. This can be useful if services wish to chain functionality (the Simple API does not use the RequestDispatcher.include technique because it fosters the idea that everything should be a Service). The underlying monitoring mechanism enables handing of the HTTP transaction to another Service completely transparently, as illustrated below.

   public void handle(Request req, Response resp) {
      Service service = engine.lookup("name", false);
      new Thread(new Runnable(){
         public void run(){
            try {
               service.handle(req, resp);
            }catch(Exception e){
               // ....
            }
         }
      }).start();
   }





SourceForge Logo