The Apache Tomcat Servlet/JSP Container

Apache Tomcat 6.0

Apache Logo

Links

User Guide

Reference

Apache Tomcat Development

Apache Tomcat 6.0

The Tomcat JDBC Connection Pool

Introduction

The JDBC Connection Pool org.apache.tomcat.jdbc.pool is a replacement or an alternative to the commons-dbcp connection pool.

So why do we need a new connection pool?

Here are a few of the reasons:

  1. commons-dbcp is single threaded, in order to be thread safe commons-dbcp locks the entire pool, even during query validation.
  2. commons-dbcp is slow - as the number of logical CPUs grow, the performance suffers, the above point shows that there is not support for high concurrency Even with the enormous optimizations of the synchronized statement in Java 6, commons-dbcp still suffers in speed and concurrency.
  3. commons-dbcp is complex, over 60 classes. tomcat-jdbc-pool, is 8 classes, hence modifications for future requirement will require much less changes.
  4. commons-dbcp uses static interfaces. This means you can't compile it with JDK 1.6, or if you run on JDK 1.6/1.7 you will get NoSuchMethodException for all the methods not implemented, even if the driver supports it.
  5. The commons-dbcp has become fairly stagnant. Sparse updates, releases, and new feature support.
  6. It's not worth rewriting over 60 classes, when something as a connection pool can be accomplished with as a much simpler implementation.
  7. Tomcat jdbc pool implements a fairness option not available in commons-dbcp and still performs faster than commons-dbcp
  8. Tomcat jdbc pool implements the ability retrieve a connection asynchronously, without adding additional threads to the library itself
  9. Tomcat jdbc pool is a Tomcat module, it depends on Tomcat JULI, a simplified logging framework used in Tomcat.

Features added over other connection pool implementations

  1. Support for highly concurrent environments and multi core/cpu systems.
  2. Dynamic implementation of interface, will support java.sql and javax.sql interfaces for your runtime environment (as long as your JDBC driver does the same), even when compiled with a lower version of the JDK.
  3. Validation intervals - we don't have to validate every single time we use the connection, we can do this when we borrow or return the connection, just not more frequent than an interval we can configure.
  4. Run-Once query, a configurable query that will be run only once, when the connection to the database is established. Very useful to setup session settings, that you want to exist during the entire time the connection is established.
  5. Ability to configure custom interceptors. This allows you to write custom interceptors to enhance the functionality. You can use interceptors to gather query stats, cache session states, reconnect the connection upon failures, retry queries, cache query results, and so on. Your options are endless and the interceptors are dynamic, not tied to a JDK version of a java.sql/javax.sql interface.
  6. High performance - we will show some differences in performance later on
  7. Extremely simple, due to the very simplified implementation, the line count and source file count are very low, compare with c3p0 that has over 200 source files(last time we checked), Tomcat jdbc has a core of 8 files, the connection pool itself is about half that. As bugs may occur, they will be faster to track down, and easier to fix. Complexity reduction has been a focus from inception.
  8. Asynchronous connection retrieval - you can queue your request for a connection and receive a Future<Connection> back.

How to use

Usage of the Tomcat connection pool has been made to be as simple as possible, for those of you that are familiar with commons-dbcp, the transition will be very simple. Moving from other connection pools is also fairly straight forward.

Additional features

The Tomcat connection pool offers a few additional features over what most other pools let you do:

  • initSQL - the ability to run a SQL statement exactly once, when the connection is created
  • validationInterval - in addition to running validations on connections, avoid running them too frequently.
  • jdbcInterceptors - flexible and pluggable interceptors to create any customizations around the pool, the query execution and the result set handling. More on this in the advanced section.
  • fairQueue - Set the fair flag to true to achieve thread fairness or to use asynchronous connection retrieval
Inside the Apache Tomcat Container

The Tomcat Connection pool is configured as a resource described in The Tomcat JDBC documentation With the only difference being that you have to specify the factory attribute and set the value to org.apache.tomcat.jdbc.pool.DataSourceFactory

Standalone

The connection pool only has another dependency, and that is on tomcat-juli.jar. To configure the pool in a stand alone project using bean instantiation, the bean to instantiate is org.apache.tomcat.jdbc.pool.DataSource. The same attributes (documented below) as you use to configure a connection pool as a JNDI resource, are used to configure a data source as a bean.

Attributes

To provide a very simple switch to and from commons-dbcp and tomcat-jdbc-pool, Most attributes are the same and have the same meaning.

JNDI Factory and Type
AttributeDescription
factory

factory is required, and the value should be org.apache.tomcat.jdbc.pool.DataSourceFactory

type

Type should always be javax.sql.DataSource

Common Attributes

These attributes are shared between commons-dbcp and tomcat-jdbc-pool, in some cases default values are different.

AttributeDescription
defaultAutoCommit

(boolean) The default auto-commit state of connections created by this pool. If not set, default is JDBC driver default (If not set then the setAutoCommit method will not be called.)

defaultReadOnly

(boolean) The default read-only state of connections created by this pool. If not set then the setReadOnly method will not be called. (Some drivers don't support read only mode, ex: Informix)

defaultTransactionIsolation

(String) The default TransactionIsolation state of connections created by this pool. One of the following: (see javadoc )
* NONE
* READ_COMMITTED
* READ_UNCOMMITTED
* REPEATABLE_READ
* SERIALIZABLE
If not set, the method will not be called and it defaults to the JDBC driver.

defaultCatalog

(String) The default catalog of connections created by this pool.

driverClassName

(String) The fully qualified Java class name of the JDBC driver to be used. The driver has to be accessible from the same classloader as tomcat-jdbc.jar

username

(String) The connection username to be passed to our JDBC driver to establish a connection. Note, at this point, DataSource.getConnection(username,password) is not using the credentials passed into the method.

password

(String) The connection password to be passed to our JDBC driver to establish a connection. Note, at this point, DataSource.getConnection(username,password) is not using the credentials passed into the method.

maxActive

(int) The maximum number of active connections that can be allocated from this pool at the same time. The default value is 100

maxIdle

(int) The maximum number of connections that should be kept in the pool at all times. Default value is maxActive:100 Idle connections are checked periodically (if enabled) and connections that been idle for longer than minEvictableIdleTimeMillis will be released. (also see testWhileIdle)

minIdle

(int) The minimum number of established connections that should be kept in the pool at all times. The connection pool can shrink below this number if validation queries fail. Default value is derived from initialSize:10 (also see testWhileIdle)

initialSize

(int)The initial number of connections that are created when the pool is started. Default value is 10

maxWait

(long) The maximum number of milliseconds that the pool will wait (when there are no available connections) for a connection to be returned before throwing an exception. Default value is 30000 (30 seconds)

testOnBorrow

(boolean) The indication of whether objects will be validated before being borrowed from the pool. If the object fails to validate, it will be dropped from the pool, and we will attempt to borrow another. NOTE - for a true value to have any effect, the validationQuery parameter must be set to a non-null string. Default value is false

testOnReturn

(boolean) The indication of whether objects will be validated before being returned to the pool. NOTE - for a true value to have any effect, the validationQuery parameter must be set to a non-null string. The default value is false.

testWhileIdle

(boolean) The indication of whether objects will be validated by the idle object evictor (if any). If an object fails to validate, it will be dropped from the pool. NOTE - for a true value to have any effect, the validationQuery parameter must be set to a non-null string. The default value is false and this property has to be set in order for the pool cleaner/test thread is to run (also see timeBetweenEvictionRunsMillis)

validationQuery

(String) The SQL query that will be used to validate connections from this pool before returning them to the caller. If specified, this query does not have to return any data, it just can't throw a SQLException. The default value is null. Example values are SELECT 1(mysql), select 1 from dual(oracle), SELECT 1(MS Sql Server)

timeBetweenEvictionRunsMillis

(long) The number of milliseconds to sleep between runs of the idle connection validation/cleaner thread. This value should not be set under 1 second. It dictates how often we check for idle, abandoned connections, and how often we validate idle connections. The default value is 5000 (5 seconds).

numTestsPerEvictionRun

(int) Property not used in tomcat-jdbc-pool.

minEvictableIdleTimeMillis

(long) The minimum amount of time an object may sit idle in the pool before it is eligable for eviction. The default value is 60000 (60 seconds).

accessToUnderlyingConnectionAllowed

(boolean) Property not used. Access can be achieved by calling unwrap on the pooled connection. see javax.sql.DataSource interface, or call getConnection through reflection.

removeAbandoned

(boolean) Flag to remove abandoned connections if they exceed the removeAbandonedTimout. If set to true a connection is considered abandoned and eligible for removal if it has been in use longer than the removeAbandonedTimeout Setting this to true can recover db connections from applications that fail to close a connection. See also logAbandoned The default value is false.

removeAbandonedTimeout

(long) Timeout in seconds before an abandoned(in use) connection can be removed. The default value is 60 (60 seconds). The value should be set to the longest running query your applications might have.

logAbandoned

(boolean) Flag to log stack traces for application code which abandoned a Connection. Logging of abandoned Connections adds overhead for every Connection borrow because a stack trace has to be generated. The default value is false.

connectionProperties

(String) The connection properties that will be sent to our JDBC driver when establishing new connections. Format of the string must be [propertyName=property;]* NOTE - The "user" and "password" properties will be passed explicitly, so they do not need to be included here. The default value is null.

poolPreparedStatements

(boolean) Property not used. The default value is false.

maxOpenPreparedStatements

(int) Property not used. The default value is false.

Tomcat JDBC Enhanced Attributes
AttributeDescription
initSQL

(String) A custom query to be run when a connection is first created. The default value is null.

jdbcInterceptors

(String) A semicolon separated list of classnames extending org.apache.tomcat.jdbc.pool.JdbcInterceptor class. These interceptors will be inserted as an interceptor into the chain of operations on a java.sql.Connection object. The default value is null.
Predefined interceptors:
org.apache.tomcat.jdbc.pool.interceptor.ConnectionState - keeps track of auto commit, read only, catalog and transaction isolation level.
org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer - keeps track of opened statements, and closes them when the connection is returned to the pool.

validationInterval

(long) avoid excess validation, only run validation at most at this frequency - time in milliseconds. If a connection is due for validation, but has been validated previously within this interval, it will not be validated again. The default value is 30000 (30 seconds).

jmxEnabled

(boolean) Register the pool with JMX or not. The default value is true.

fairQueue

(boolean) Set to true if you wish that calls to getConnection should be treated fairly in a true FIFO fashion. This uses the org.apache.tomcat.jdbc.pool.FairBlockingQueue implementation for the list of the idle connections. The default value is false. This flag is required when you want to use asynchronous connection retrieval.

useEquals

(boolean) Set to true if you wish the ProxyConnection class to use String.equals instead of == when comparing method names. This property does not apply to added interceptors as those are configured individually. The default value is false.

Advanced usage
JDBC interceptors

To see an example of how to use an interceptor, take a look at org.apache.tomcat.jdbc.pool.interceptor.ConnectionState. This simple interceptor is a cache of three attributes, transaction isolation level, auto commit and read only state, in order for the system to avoid not needed roundtrips to the database.

Further interceptors will be added to the core of the pool as the need arises. Contributions are always welcome!

Interceptors are of course not limited to just java.sql.Connection but can be used to wrap any of the results from a method invokation as well. You could build query performance analyzer that provides JMX notifications when a query is running longer than the expected time.

JDBC interceptors

Configuring JDBC interceptors is done using the jdbcInterceptors property. The property contains a list of semi colon separated class names. If the classname if not fully qualified it will be prefixed with the org.apache.tomcat.jdbc.pool.interceptor. prefix.
Example:
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
is the same as
jdbcInterceptors="ConnectionState;StatementFinalizer"

Interceptors can have properties as well. These would be configured within the paranthesis of the class names. Example:
jdbcInterceptors="ConnectionState;StatementFinalizer(useWeakReferences=true,useEquals=true)"

org.apache.tomcat.jdbc.pool.JdbcInterceptor

Abstract base class for all interceptors, can not be instantiated.

AttributeDescription
useEquals

(String as boolean) A custom query to be run when a connection is first created. The default value is false.

org.apache.tomcat.jdbc.pool.interceptor.ConnectionState

Caches the connection for the following attributes autoCommit, readOnly, transactionIsolation and catalog. It is a performance enhancement to avoid roundtrip to the database when getters are called or setters are called with an already set value.

AttributeDescription
org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer

Keeps track of all statements created using createStatement, prepareStatement or prepareCall and closes these statements when the connection is returned to the pool.

AttributeDescription
org.apache.tomcat.jdbc.pool.interceptor.SlowQueryReport

Keeps track of query performance and issues log entries when queries exceed a time threshold of fail. The log level used is WARN

AttributeDescription
threshold

(int as String) The number of milliseconds a query has to exceed before issuing a log alert. The default value is 1000 milliseconds.

maxQueries

(int as String) The maximum number of queries to keep track of in order to preserve memory space The default value is 1000.

org.apache.tomcat.jdbc.pool.interceptor.SlowQueryReportJmx

Extends the SlowQueryReport and in addition to log entries it issues JMX notification for monitoring tools to react to. Inherits all the attributes from its parent class. This class uses Tomcat's JMX engine so it wont work outside of the Tomcat container.

AttributeDescription
Code Example

Other examples of Tomcat configuration for JDBC usage can be found in the Tomcat documentation.

Plain Ol' Java

Here is a simple example of how to create and use a data source.

        import java.sql.Connection;
        import java.sql.ResultSet;
        import java.sql.Statement;
        
        import org.apache.tomcat.jdbc.pool.DataSource;
        import org.apache.tomcat.jdbc.pool.PoolProperties;
        
        public class SimplePOJOExample {
        
            public static void main(String[] args) throws Exception {
                PoolProperties p = new PoolProperties();
                p.setUrl("jdbc:mysql://localhost:3306/mysql?autoReconnect=true");
                p.setDriverClassName("com.mysql.jdbc.Driver");
                p.setUsername("root");
                p.setPassword("password");
                p.setJmxEnabled(true);
                p.setTestWhileIdle(false);
                p.setTestOnBorrow(true);
                p.setValidationQuery("SELECT 1");
                p.setTestOnReturn(false);
                p.setValidationInterval(30000);
                p.setTimeBetweenEvictionRunsMillis(30000);
                p.setMaxActive(100);
                p.setInitialSize(10);
                p.setMaxWait(10000);
                p.setRemoveAbandonedTimeout(60);
                p.setMinEvictableIdleTimeMillis(30000);
                p.setMinIdle(10);
                p.setLogAbandoned(true);
                p.setRemoveAbandoned(true);
                p.setJdbcInterceptors("org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer");
                DataSource datasource = new DataSource();
                datasource.setPoolProperties(p); 
                
                Connection con = null;
                try {            
                  con = datasource.getConnection();
                  Statement st = con.createStatement();
                  ResultSet rs = st.executeQuery("select * from user");
                  int cnt = 1;
                  while (rs.next()) {
                      System.out.println((cnt++)+". Host:" +rs.getString("Host")+" User:"+rs.getString("User")+" Password:"+rs.getString("Password"));
                  }
                  rs.close();
                  st.close();
                } finally {
                  if (con!=null) try {con.close();}catch (Exception ignore) {}
                }  
            }
        
        }
    
As a Resource

And here is an example on how to configure a resource for JNDI lookups

    <Resource name="jdbc/TestDB" 
              auth="Container" 
              type="javax.sql.DataSource" 
              factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
              testWhileIdle="true"
              testOnBorrow="true"
              testOnReturn="false"
              validationQuery="SELECT 1"
              validationInterval="30000"
              timeBetweenEvictionRunsMillis="30000"
              maxActive="100" 
              minIdle="10" 
              maxWait="10000" 
              initialSize="10"
              removeAbandonedTimeout="60"
              removeAbandoned="true"
              logAbandoned="true"
              minEvictableIdleTimeMillis="30000" 
              jmxEnabled="true"
              jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
              username="root" 
              password="password" 
              driverClassName="com.mysql.jdbc.Driver"
              url="jdbc:mysql://localhost:3306/mysql?autoReconnect=true"/>

    
    
Asynchronous Connection Retrieval

The Tomcat JDBC connection pool supports asynchronous connection retrieval without adding additional threads to the pool library. It does this by adding a method to the data source called Future<Connection> getConnectionAsync(). In order to use the async retrieval, two conditions must be met:
1. You must configure the fairQueue property to be true.
2. You will have to cast the data source to org.apache.tomcat.jdbc.pool.DataSource
An example of using the async feature is show below.

      
        Connection con = null;
        try {            
          Future<Connection> future = datasource.getConnectionAsync();
          while (!future.isDone()) {
              System.out.println("Connection is not yet available. Do some background work");
              try {
                  Thread.sleep(100); //simulate work
              }catch (InterruptedException x) {
                  Thread.currentThread().interrupted();
              }
          }
          con = future.get(); //should return instantly 
          Statement st = con.createStatement();
          ResultSet rs = st.executeQuery("select * from user");
      

Interceptors

Interceptors are a powerful way to enable, disable or modify functionality on a specific connection or its sub components. There are many different use cases for when interceptors are useful. By default, and for performance reasons, the connection pool is stateless. The only state the pool itself inserts are defaultAutoCommit, defaultReadOnly, defaultTransactionIsolation, defaultCatalog if these are set. These 4 properties are only set upon connection creation. Should these properties be modified during the usage of the connection, the pool itself will not reset them.

An interceptor has to extend the org.apache.tomcat.jdbc.pool.JdbcInterceptor class. This class is fairly simple, You will need to have a no arg constructor

       
         public JdbcInterceptor() {
         }
       

When a connection is borrowed from the pool, the interceptor can initialize or in some other way react to the event by implementing the

       
         public abstract void reset(ConnectionPool parent, PooledConnection con);
       
method. This method gets called with two parameters, a reference to the connection pool itself ConnectionPool parent and a reference to the underlying connection PooledConnection con.

When a method on the java.sql.Connection object is invoked, it will cause the

         public Object invoke(Object proxy, Method method, Object[] args) throws Throwable
       
method to get invoked. The Method method is the actual method invoked, and Object[] args are the arguments. To look at a very simple example, where we demonstrate how to make the invokation to java.sql.Connection.close() a noop if the connection has been closed
       
        if (CLOSE_VAL==method.getName()) {
            if (isClosed()) return null; //noop for already closed.
        } 
        return super.invoke(proxy,method,args);
        
There is an observation being made. It is the comparison of the method name. One way to do this would be to do "close".equals(method.getName()). Above we see a direct reference comparison between the method name and static final String reference. According to the JVM spec, method names and static final String end up in a shared constant pool, so the reference comparison should work. One could of course do this as well:
       
        if (compare(CLOSE_VAL,method)) {
            if (isClosed()) return null; //noop for already closed.
        } 
        return super.invoke(proxy,method,args);
        
The compare(String,Method) will use the useEquals flag on an interceptor and do either reference comparison or a string value comparison when the useEquals=true flag is set.

Pool start/stop
When the connection pool is started or closed, you can be notifed. You will only be notified once per interceptor class even though it is an instance method. and you will be notified using an interceptor currently not attached to a pool.

       public void poolStarted(ConnectionPool pool) {
       }
       
       public void poolClosed(ConnectionPool pool) {
       }
       
When overriding these methods, don't forget to call super if you are extending a class other than JdbcInterceptor

Configuring interceptors
Interceptors are configured using the jdbcInterceptors property or the setJdbcInterceptors method. An interceptor can have properties, and would be configured like this

       String jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState(useEquals=true,fast=yes)"
       

Interceptor properties
Since interceptors can have properties, you need to be able to read the values of these properties within your interceptor. Taking an example like the one above, you can override the setProperties method.

	   @Override
	   public void setProperties(Map<String, InterceptorProperty> properties) {
	       super.setProperties(properties);
	       final String myprop = "myprop";
	       InterceptorProperty p1 = properties.get(myprop);
	       if (p1!=null) {
	           setMyprop(Long.parseLong(p1.getValue()));
	       }
	   }
        


Copyright © 1999-2008, Apache Software Foundation