The goal of monitoring is to enable the server to determine when it can proceed with the next request from a HTTP pipeline. This is done independently of the Service objects. The initial I/O used by Simple incorporates a poll to determine the pipelines that can be processed. Each pipeline has a dedicated Poller object which reads the HTTP message header from the pipeline. The key objective of the poller is to ensure that the server does not block waiting for the HTTP header which would require thread per connection semantics. Instead the poller reads bytes in chunks from the underlying Socket object. When it has read the full HTTP header from the pipeline it dispatches the transaction to a ProtocolHandler for processing. The transaction is dispatched using the Request and Response objects, each object is given a Monitor, which is used to assist monitoring of the HTTP I/O.

Polling

The polling of the pipeline is done by determining the number of bytes available from the socket without blocking. This data is read from the socket and scanned for the CRLF CRLF header ending, see RFC 2616. If the pipeline is not producing data the poller schedules itself for a period of time with a PollerHandler. This will reprocess the poller after the specified period of time giving the client time to transmit the HTTP header. This process is iterated on each poll with an increasing wait period. The wait period is exponential so that the more inactive the pipeline the longer it waits for the next poll. If the wait period grows beyond a specified threshold the TCP connection is closed which frees up resources held by the pipeline such as file descriptors and memory.

This polling technique works well in processing pipelines ensuring that a fixed amount of threads can process an arbitrary amount of connected clients. Once the HTTP header has been read it is used to create a Request object which can be handed to the ProtocolHandler. The exponential back-off technique ensures that the server does not expend CPU cycles polling inactive pipelines and the threshold wait period allows the server to close inactive TCP connections freeing resources. The wait period is configurable and is very sensitive to the observed and actual performance of the entire system. If the threshold is high then clients requesting a page over a pipeline will think that the server is overloaded and slow in processing the request when in fact the poller is scheduled within a queue waiting to be reprocessed. When the time has expired then the request is processed this results in observed poor performance (a high wait has benefits for DOS attacks and slow network connections). If the wait is kept low then clients reusing the pipeline will observe little delay (this has its benefits when the server is under little load as it increases observed performance). The client will observe delay regardless because the server is busy processing requests. So the client will observe little difference between a highly loaded server and a relatively dormant server depending on the configured wait period. This technique enables an arbitrary number of connections to be processed concurrently. This ensures that each pipeline is given equal priority and that requests are taken from each pipeline in sequence. Because the server uses a fixed thread pool dispatched requests are handled just as fast regardless of the load on the server.

Monitors

The monitoring of the pipeline is done using monitored input and output streams. These monitors are configured by interpreting the HTTP headers. If the HTTP request specifies that there is content with the request then an MonitoredInputStream is used to determine when the client has read all data from the issued InputStream and notifies an InputMonitor when the request content has been read. If response content is given to the client then a MonitoredOutputStream performs the same task for the response output. When both request and response have finished with there respective streams the monitor reprocesses the Poller ensuring the the next bytes read from the input stream is the next HTTP request and that the output can receive a HTTP response on the output stream. The monitoring system used by the server is shown below.

The streams issued to the service object are acquired using the Request.getInputStream and Response.getOutputStream methods. The service can configure the connection semantics using the Response so that the response content can be chunked (see RFC 2616), fixed for a specified Content-Length, or closed as in HTTP/1.0. The streams returned each perform monitoring so that the poller can be reprocessed once the service has finished with the stream.

The monitoring streams provide the service with illusion that it is communicating directly with the client by providing the correct responses once the content has been read from the stream. Once the MonitoredInputStream determines that the entire HTTP request content has been read it returns -1 for the InputStream.read operation this ensures that the service cannot read the HTTP request header from the next request on the pipeline.

The monitoring of the content output performs that task of monitoring output written to the pipeline. This ensures that if there is insufficient content to fulfill a Content-Length header that the server can take appropriate action (which is immediate closure of the TCP connection according to RFC 2616). This also ensures that the service cannot write more than is promised by the interpretation of the HTTP response header so that the pipeline can be maintained. This form of monitoring enables transparent concurrency to be achieved. Service implementations can process the HTTP transactions without restrictions on time taken within the Service.handle method and requests can be forwarded to other Service implementations asynchronously.