Links User Guide Reference Apache Tomcat Development | | Introduction |
With usage of APR or NIO APIs as the basis of its connectors, Tomcat is
able to provide a number of extensions over the regular blocking IO
as provided with support for the Servlet API.
IMPORTANT NOTE: Usage of these features requires using the APR or NIO
HTTP connectors. The classic java.io HTTP connector and the AJP connectors
do not support them.
|
| Comet support |
Comet support allows a servlet to process IO aynchronously, recieving
events when data is available for reading on the connection (rather than
always using a blocking read), and writing data back on connections
asychnonously (most likely responding to some event raised from some
other source).
| Comet Terms |
Comet Connection - For the readability of this document we will be referring an open ended HTTP request and an open
ended HTTP response that are mapped to a Cometprocessor as a "Comet Connection".
Comet Processor - a servlet that implements the org.apache.catalina.CometProcessor interface.
Used to process comet connections and the events on the connection
Comet Event - an event triggered by the container related to a Comet Connection
Open Ended Request - a HTTP request that is either chunked or has a very large content length, allowing
data to be sent at different times. This is how Tomcat gets around the request/response methodology of
the HTTP protocol allowing Comet Processors to send and receive data on the socket synchronously and asynchronously.
Open Ended Response - see Open Ended Request
Active Comet Connection, a comet connection that currently is associated with a TCP connection and an open ended request/response.
Blocking Comet Connection, an invocation of the read/write methods will block until data was received/sent.
Non blocking Comet Connection, an invocation of the read/write methods will not block.
Comet operation - comet connections can register themselves for a set of events to be
notified of. The different operations are:
OP_READ - results in a READ event when data has been received
OP_WRITE - results in a WRITE event when the socket is ready to receive data
OP_CALLBACK - results in a CALLBACK event on a Tomcat thread.
Connection Centric - Comet connections are based on actual IO events on a TCP connection from the TCP layer.
This is different from servlets that are based on HTTP requests and responses, by the HTTP protocol above the TCP layer.
Closing a comet connection - may not actually mean that the underlying TCP connection is closed. Tomcat still
respects the maxKeepAliveRequests attribute of the connector, and may decide to keep the connection
open. This is the case in the event of a connection timeout, the event ERROR/TIMEOUT is signaled and the
TCP connection is reused for the next HTTP request.
|
| Comet Lifecycle |
The lifecycle and event cycle of a Comet Connection is slightly different than a regular servlet.
Instead the life/event cycle is very "connection centric" and based on IO events, rater then
a request/response cycle like a normal HTTP request. This is the most common mistake developers make when they
start writing comet servlets, is that they don't realize that it is all based around these different events, and
that the events are some what connection centric.
A comet interaction is started by the container invoking the event method on the Comet Processor
and the event will have a BEGIN type. For a deeper explanation of types, keep reading.
Once a BEGIN event has been invoked, the Comet Connection is active. At this type, the Comet Event object
reference can be used to reference the HttpServletRequest/HttpServletResponse objects for asynchronous actions
such as reading or writing from the streams or readers/writers. At this point the comet connection is considered active
or initiated.
Once the Comet connection is active, the comet processor will receive events from the container.
These events are defined in the section below.
A comet processor may register itself for events, and receive the events when they happen.
In a similar fashion, a comet processor may unregister itself from comet events and
use the isReadable/isWriteable methods on the comet event to detect the state
of the comet connection. The registered events will be in effect until they are unregistered,
the comet connection is closed or terminated.
By default a comet connection gets registered for READ events upon initiation.
The event registration can be compared to poll or select structures of different operating systems,
and also bear some resemblances with the java.nio API.
Since Comet Connections deal directly with the IO layer of the container/connection and read and writes can be done
asynchronously, caution and care should be excersized when programming this. the comet connection itself is not thread safe
as such an implementation would suffer performance issues, instead it is up to the comet processor developer to ensure that
thread safety is reached and properly handled.
As an example, registering the comet connection for READ events and at the same time performing asynchronous reads
on the input stream or the reader, can cause data corruption and dead locks.
Same way, using multiple threads to do writes to the output stream or writer can have the same effect.
To be thread safe, one can synchronize on the event object, but you will also need to coordinate it with
events that are registered.
There is an event that is not IO based, that is the CometEvent.EventType.CALLBACK event.
This is an event that is forced by Tomcat when the OP_CALLBACK operation is registered.
Using the OP_CALLBACK operation, Tomcat will spawn a Tomcat worker thread that you can use to piggy back
on for reading/writing data or performing other operations, and not having to have spawn and synchronize
your own background threads.
An example scenarion would be where you have one thread pulling content for different comet clients.
When the background thread has content for a client, it can store it in the request object as an attribute,
then register for the OP_CALLBACK event. Once the CALLBACK event, then the application can use Tomcat's worker
thread to write the data to the client.
Tomcat guarantees that only one thread will be invoking the CometProcessor event for a particular client,
so by using the Tomcat worker threads to perform your actions, you are thread safe without the expense of
locks or synchronized methods.
Another usage scenario for the CALLBACK event, is when you close the comet connection asynchronously and you want
it processed by tomcat without depending on a timeout or another IO event. This would look like
 | | | |
...
event.close();
event.register(CometEvent.CometOperation.OP_CALLBACK);
...
| | | | |
|
| Comet Operations |
The previous section touched a little bit on the comet connection lifecycle.
It is important to remember that comet events are based around IO events of an actual socket.
To clarify the Comet API, it has been created to resemble the java.nio channel/selector APIs.
In the case of Comet, Tomcat is the selector and using the CometEvent object, you can
register and unregister your Comet event for different event type notifications.
We call the parameter to the CometEvent.register/unregister method a comet operation.
This is similar to the interestOps method of a SelectionKey in the java.nio implementation.
The Comet implementation of register and unregister has been greatly simplified to not force the
comet developer to implement complex synchronizations around the register and unregister code.
It is important to realize, just like the java.nio API, that once an operation has been registered, it will
remain registered until it is unregistered. If you have registered OP_READ, then the comet connection will
fire READ events, every time data arrives until your unregister the OP_READ operation.
OP_CALLBACK/OP_WRITE work in the same way, essentially, register(OP_CALLBACK|OP_WRITE) will keep spawning
CALLBACK/WRITE events until you unregister the operation(s).
|
| CometEvent |
Servlets which implement the org.apache.catalina.CometProcessor
interface will have their event method invoked rather than the usual service
method, according to the event which occurred. The event object gives
access to the usual request and response objects, which may be used in the
usual way. The main difference is that those objects remain valid and fully
functional at any time between processing of the BEGIN event until processing
an END or ERROR event.
The following event types exist:
- EventType.BEGIN: will be called at the beginning
of the processing of the connection. It can be used to initialize any relevant
fields using the request and response objects. During the BEGIN event you may also configure
your comet connection for blocking or non blocking mode. using the
configureBlocking(boolean)
method on the comet event object. Between the end of the processing
of this event, and the beginning of the processing of the end or error events,
it is possible to use the response object to write data on the open connection.
Note that the response object and depedent OutputStream and Writer are still
not synchronized, so when they are accessed by multiple threads,
synchronization is mandatory. After processing the initial event, the request
is considered to be committed.
- EventType.READ: This indicates that input data is available, and that one read can be made
without blocking. The
available() and ready() methods of the InputStream or
Reader may be used to determine if there is a risk of blocking: the servlet
should read while data is reported available, and can make one additional read
should read while data is reported available. Not reading all the data, is not recommended,
as it may lead to unexpected behavior depending on the connector implementation.
When encountering a read error, the servlet should report it by propagating the exception properly.
Throwing an exception will cause the error event to be invoked, and the connection
will be closed.
Alternately, it is also possible to catch any exception, perform clean up
on any data structure the servlet may be using, and using the close() method
of the event object. It is not recommended to attempt reading data from the request
object outside of the execution of this method/event if the comet connection is registered for
the READ event. Instead unregister the read event to perform asynchronous reads.
On some platforms, like Windows, a client disconnect is indicated by a READ event.
Reading from the stream may result in -1, an IOException or an EOFException.
Make sure you properly handle all these three cases.
If you don't catch the IOException, Tomcat will instantly invoke your event chain with an ERROR as
it catches the error for you, and you will be notified of the error at that time.
- EventType.WRITE: If you wish to be notified whether you can write data to the underlying TCP socket,
register your comet connection for this event. Tomcat will invoke this event, and you can write to the response
object. This event is not needed nor should be used unless you are running the comet connection in non blocking mode.
- EventType.CALLBACK: When a comet connection is registered using the OP_CALLBACK operation,
Tomcat will generate the CALLBACK event periodically. The CALLBACK will always
be invoked using a Tomcat worker thread, just like the other event types.
- EventType.END: End may be called to end the processing of the request. Fields that have
been initialized in the begin method should be reset. After this event has
been processed, the request and response objects, as well as all their dependent
objects will be recycled and used to process other requests. End will also be
called when data is available and the end of file is reached on the request input
(this usually indicates the client has pipelined a request).
- EventType.ERROR: Error will be called by the container in the case where an IO exception
or a similar unrecoverable error occurs on the connection. Fields that have
been initialized in the begin method should be reset. After this event has
been processed, the request and response objects, as well as all their dependent
objects will be recycled and used to process other requests.
There are some event subtypes which allow finer processing of events (note: some of these
events require usage of the org.apache.catalina.valves.CometConnectionManagerValve valve):
- EventSubType.TIMEOUT: The connection timed out (sub type of ERROR); note that this ERROR
type is not fatal, and the connection will not be closed unless the servlet uses the close
method of the event.
- EventSubType.CLIENT_DISCONNECT: The client connection was closed (sub type of ERROR).
method of the event.
- EventSubType.IOEXCEPTION: An IO exception occurred, such as invalid content, for example,
an invalid chunk block (sub type of ERROR).
- EventSubType.WEBAPP_RELOAD: The web application is being reloaded (sub type of END).
- EventSubType.SESSION_END: The servlet ended the session (sub type of END).
As described above, the typical lifecycle of a Comet request will consist in a series of
events such as: BEGIN -> READ -> READ -> READ -> ERROR/TIMEOUT. At any time, the servlet
may end processing of the request by using the close method of the event object.
|
| CometFilter |
Similar to regular filters, a filter chain is invoked when comet events are processed.
These filters should implement the CometFilter interface (which works in the same way as
the regular Filter interface), and should be declared and mapped in the deployment
descriptor in the same way as a regular filter. The filter chain when processing an event
will only include filters which match all the usual mapping rules, and also implement
the CometFiler interface.
|
| Example code snippets |
Let's start with a real world example of how you could use Comet to handle AJAX requests.
An AJAX request is a complete HTTP request, followed by a complete HTTP response.
The next request is a new HTTP request.
There are many AJAX frameworks, and they can be used with regular servlets, so why
would I want to do this using Comet?
The answer is simply, with Comet you can do the response asynchronously and therefor
release the Tomcat thread to handle other requests.
| | | |
Non blocking read and writes
public class ExampleDelayedAjaxResponse implements CometProcessor {
...
public class DelayedResponder extends Thread {
public void run() {
...
Client[] clients = getClients();
for (int i=0; i<clients.length; i++ ) {
CometEvent event = client.getEvent();
byte[] data = getDelayedResponse(event);
if (data!=null) {
if (event.isWriteable()) {
event.getHttpServletResponse().getOutputStream().write(data);
if (event.isWriteable()) { //did we write it all?
event.close(); //close the event, in anticipation of the next request
event.register(OP_CALLBACK); //triggers an END event
} else { //we didn't write it all, trigger a WRITE event when we are done with the write
event.register(OP_WRITE);
}
} else {
//we are not able to write, let us know when we can
event.register(OP_WRITE);
}
//remove the client from the async thread
//since we have received the data for this client.
clients.remove(event);
}
}
...
}
}
...
public void event(CometEvent event) throws IOException, ServletException {
...
if ( event.getEventType() == CometEvent.EventType.BEGIN ) {
//configure non blocking
event.configureBlocking(false);
//deregister for READ since we want to enable pipe lining on the connection
//for the next HTTP request
event.unregister(OP_READ);
//add the client to the list
clients.add(event);
} if ( event.getEventType() == CometEvent.EventType.READ ) {
//read client Id and stock list from client
//and add the event to our list
assert("this should never happen");
} if ( event.getEventType() == CometEvent.EventType.WRITE ) {
//unregister from the write event
event.unregister(OP_WRITE);
//we can now write
byte[] data = getDelayedResponse(event);
if ( data != null ) {
event.getHttpServletResponse().getOutputStream().write(data);
}
if( data==null || event.isWriteable() ) {
event.close();
}
} if ( event.getEventType() == CometEvent.EventType.END ) {
clients.remove(event);
} else if (...) {
...
}
...
}
}
Blocking read and writes
public class ExampleDelayedAjaxResponse implements CometProcessor {
...
public class DelayedResponder extends Thread {
public void run() {
...
Client[] clients = getClients();
for (int i=0; i<clients.length; i++ ) {
CometEvent event = client.getEvent();
byte[] data = getDelayedResponse(event);
if (data!=null) {
//register for a write, better do that on a thread pool thread
//since write is blocking
event.register(OP_WRITE);
//remove the client from the async thread
clients.remove(event);
}
}
...
}
}
...
public void event(CometEvent event) throws IOException, ServletException {
...
if ( event.getEventType() == CometEvent.EventType.BEGIN ) {
//configure blocking
event.configureBlocking(true);
//deregister for READ since we want to enable pipe lining on the connection
//for the next HTTP request
event.unregister(OP_READ);
//add the client to the list
clients.add(event);
} if ( event.getEventType() == CometEvent.EventType.READ ) {
//read client Id and stock list from client
//and add the event to our list
assert("this should never happen");
} if ( event.getEventType() == CometEvent.EventType.WRITE ) {
//unregister from the write event
event.unregister(OP_WRITE);
//we can now write
byte[] data = getDelayedResponse(event);
//note we don't have to check for null data here
event.getHttpServletResponse().getOutputStream().write(data);
event.close();
} if ( event.getEventType() == CometEvent.EventType.END ) {
clients.remove(event);
} else if (...) {
...
}
...
}
}
| | | | |
Imagine you are writing a servlet that is updating a set of
stock tickers. You have a back ground thread that is receiving
updates for tickers as they happen, and you wish to push out these
to all the tickers that have the stocks in their list.
In the following example, you can accomplish maximum through put by
taking advantage of Tomcat's non blocking Comet features.
When the StockUpdater thread is running, it receives a set of stock updates.
It gets all the clients that are registered for the stocks that have changed.
For each client, there is an associated CometEvent object, the StockUpdater
checks if it can write without blocking, if so it writes directly, otherwise
it registers the client for a WRITE event, that will tell the
system that we can write the data now.
This is a perfect example of how we can take advantage of the combination of the write event
and the isWriteable method to determine, when we can write the data.
As with any kind of non blocking IO, you will need to synchronize your code,
this has not been done in the example below since the focus is on the
data delivery, not synchronization.
| | | |
public class ExampleCometStockStreamer implements CometProcessor {
...
public class StockUpdater extends Thread {
public void run() {
...
StockUpdates[] updates = fetchUpdates();
Client[] clients = getClients(updates);
for (int i=0; i<clients.length; i++ ) {
CometEvent event = client.getEvent();
StockUpdates[] clientList = getClientUpdates(client,updates);
client.setAndMergeNextUpdates(clientList);
if (event.isWriteable()) {
byte[] data = getUpdateChunk(client.getNextUpdates());
event.getHttpServletResponse().getOutputStream().write(data);
} else {
event.register(OP_WRITE);
}
}
...
}
}
...
public void event(CometEvent event) throws IOException, ServletException {
...
if ( event.getEventType() == CometEvent.EventType.BEGIN ) {
//configure non blocking
event.configureBlocking(false);
} if ( event.getEventType() == CometEvent.EventType.READ ) {
//read client Id and stock list from client
//and add the event to our list
String clientId = readClientInfo(event,stocks);
clients.add(clientId, event, stocks);
} if ( event.getEventType() == CometEvent.EventType.WRITE ) {
//unregister from the write event
event.unregister(OP_WRITE);
//we can now write
byte[] data = getUpdateChunk(client.getNextUpdates());
event.getHttpServletResponse().getOutputStream().write(data);
} else if (...) {
...
}
...
}
}
| | | | |
The above stock ticker example is extremely powerful,
but also creates a great deal of complexity trying to
synchronize between the registration of interested events
between the threads.
Another option is to simplify this by using blocking IO.
That implementation would look like this.
Notice that writing data to the client is only done
upon an event and never asynchronously. Assuming that the data written is smaller
than the socket network buffer, this write is almost always guaranteed
to be written without a delay. Should the write be blocked,
the system is still concurrent, as the writing happens on a thread
from Tomcat's thread pool.
In this case, the only synchronization that needs to be done is in between
client.getNextUpdates() and client.setAndMergeNextUpdates(clientList).
| | | |
public class ExampleCometStockStreamer implements CometProcessor {
...
public class StockUpdater extends Thread {
public void run() {
...
StockUpdates[] updates = fetchUpdates();
Client[] clients = getClients(updates);
for (int i=0; i<clients.length; i++ ) {
StockUpdates[] clientList = getClientUpdates(client,updates);
client.setAndMergeNextUpdates(clientList);
client.getEvent().register(OP_WRITE);
}
...
}
}
...
public void event(CometEvent event) throws IOException, ServletException {
...
if ( event.getEventType() == CometEvent.EventType.BEGIN ) {
//configure blocking
event.configureBlocking(true);
} if ( event.getEventType() == CometEvent.EventType.READ ) {
//read client Id and stock list from client
//and add the event to our list
String clientId = readClientInfo(event,stocks);
clients.add(clientId, event, stocks);
} if ( event.getEventType() == CometEvent.EventType.WRITE ) {
Client client = clients.get(event);
//unregister from the write event
event.unregister(OP_WRITE);
//we can now write
byte[] data = getUpdateChunk(client.getNextUpdates());
event.getHttpServletResponse().getOutputStream().write(data);
} else if (...) {
...
}
...
}
}
| | | | |
Imagine that you wish to write a pseudo transactional system,
(please take the word transaction with a grain of salt),
and be able to do your own write scheduling.
In the next example we are going to demonstrate the ability to
use the isReadable() and isWriteable() methods in a poller sense,
and do all writes and reads asynchronously on a single thread.
Our goal here is to implement a comet servlet that reads a client request,
then writes a chunk of data when the request has been received.
The servlet will not write the next chunk until the first request has been read
and first chunk has been written to all clients.
The code below is far from optimal, but it demonstrates the ability
to not rely on any IO events, and schedule yourself when you wish to read
or write data. All operations are non blocking, so the AllWriterThread
will never block on any operation.
In the example below,
we just do a busy spin cycle.
other
| | | |
public class ExampleAllReadThenWriteComet implements CometProcessor {
...
public class AllWriterThread extends Thread {
byte[] dataChunks = ...;
public void run() {
...
for (int i=0; i<dataChunks.length; i++ ) {
for (int j=0; j<clients.size(); j++) {
boolean done = false;
while (!done) {
//first read the first request
//but only if our previous write was completed
if ( clients[j].getEvent().isWriteable() && clients[j].getEvent().isReadable() ) {
done = readClientData(clients[j]); //returns true if all data has been received for a request
}
}
done = false;
while (!done) {
//write the response
if ( clients[j].getEvent().isWriteable() {
clients[j].getEvent().getHttpServletResponse().write(dataChunks[i]);
done = true;
}
}
}
}
...
}
}
...
public void event(CometEvent event) throws IOException, ServletException {
...
if ( event.getEventType() == CometEvent.EventType.BEGIN ) {
//configure non blocking
event.configureBlocking(false);
//disable all events
event.unregister(event.getRegisteredOps());
//add the event to our client list
clients.add(event);
//start our writer if all clients have arrived
if (clients.size()==5) {
AllWriterThread thread = new AllWriterThread();
thread.start();
}
} if ( event.getEventType() == CometEvent.EventType.READ ) {
} if ( event.getEventType() == CometEvent.EventType.WRITE ) {
} else if (...) {
...
}
...
}
}
| | | | |
Ok, so the previous example was kind of silly, but we demonstrated that
you are able to read/write on a single thread, in a non blocking fashion.
Now we are going to achieve the exact same functionality, but not using
any asynchronous data, instead we are going to use
blocking IO and tomcat's worker threads
| | | |
public class ExampleAllReadThenWriteComet implements CometProcessor {
...
byte[] dataChunks = ...;
...
public void event(CometEvent event) throws IOException, ServletException {
...
if ( event.getEventType() == CometEvent.EventType.BEGIN ) {
//configure blocking
event.configureBlocking(true);
//disable all events
event.unregister(event.getRegisteredOps());
//add the event to our client list
clients.add(event);
//if all our clients have arrived, register them for read.
if (atomicClientCounter.addAndGet(1)==5) {
atomicClientReadCounter.set(5);
for (Client c : clients) {
c.getEvent().register(OP_READ);
}
}
} if ( event.getEventType() == CometEvent.EventType.READ ) {
event.unregister(OP_READ);
Client client = clients.get(event);
readClientData(client);
if (atomicClientReadCounter.addAndGet(-1) == 0 ) {
//all clients have read
atomicClientWriteCounter.set(5);
for (Client c : clients) {
c.getEvent().register(OP_WRITE);
}
}
} if ( event.getEventType() == CometEvent.EventType.WRITE ) {
event.unregister(OP_WRITE);
Client client = clients.get(event);
writeNextChunk(client);
if (atomicClientWriteCounter.addAndGet(-1) == 0 ) {
//all clients have been written, start reading the next request
atomicClientReadCounter.set(5);
for (Client c : clients) {
c.getEvent().register(OP_READ);
}
}
} else if (...) {
...
}
...
}
}
| | | | |
|
|
| Asynchronous writes |
When APR or NIO is enabled, Tomcat supports using sendfile to send large static files.
These writes, as soon as the system load increases, will be performed
asynchronously in the most efficient way. Instead of sending a large response using
blocking writes, it is possible to write content to a static file, and write it
using a sendfile code. A caching valve could take advantage of this to cache the
response data in a file rather than store it in memory. Sendfile support is
available if the request attribute org.apache.tomcat.sendfile.support
is set to Boolean.TRUE.
Any servlet can instruct Tomcat to perform a sendfile call by setting the appropriate
response attributes. When using sendfile, it is best to ensure that neither the
request or response have been wrapped, since as the response body will be sent later
by the connector itself, it cannot be filtered. Other than setting the 3 needed
response attributes, the servlet should not send any response data, but it may use
any method which will result in modifying the response header (like setting cookies).
- org.apache.tomcat.sendfile.filename: Canonical filename of the file which will be sent as
a String
- org.apache.tomcat.sendfile.start: Start offset as a Long
- org.apache.tomcat.sendfile.start: End offset as a Long
|
|
