magazine resources subscribe about advertising

 

 

 








 CD Home < Web Techniques < 2001 < August  

The Threadmaster

By Jim Jagielski

After several years of design and development, Apache 2.0 is ready for prime time. The most visible—and perhaps most awaited—change is native support for Unix threads. The proper use of threads can greatly improve the performance of most applications, so it's not surprising that threads generate a lot of discussion in development circles.

The Windows port of Apache 1.3 is, in fact, threaded. But most people use Apache with Unix-based systems, so the thread support in Apache 2.0 is a big step forward. At their most basic, threads are lightweight processes. In Unix, each application runs as a separate process. The operating system starts a new process whenever a user launches a new application. In many cases, the application itself can "fork" a new process to complete some tasks. But forking new processes is expensive: Creating them costs an appreciable amount of server memory and time. If your server is creating a lot of processes, the end result will be very slow response times to requests and actions.

Threads were designed to avoid this bottleneck. Using threads drains fewer server resources than forking a new process. One process can create many threads, all sharing the same memory space. This, too, saves time. For these reasons, it's preferable to use threads when an application needs to perform several tasks, each with the same priority.

Caveats

Threads are great, but it was obvious early in the Apache 2.0 design cycle that restricting all operations to threads was shortsighted. First of all, some people run Apache on operating systems that don't support threads. I'm a member of the core Apache development team, and one of our goals when developing Apache 2.0 was to retain support for as many platforms as possible, no matter how esoteric.

Of course, even for those platforms that do support threads, not all do so in a robust manner (like versions of FreeBSD older than 4.3). Most operating systems implement Posix Threads (also called pthreads), but their efficiency and reliability varies on different platforms. However, there are other implementations, such as pth, written by Ralf Engelschall. If your platform doesn't natively support threads or if the implementation is flaky, you should research these alternatives.

Additionally, implementing threads isn't always as easy as the concept implies. Because threads share memory with other threads, programs must be written as "thread safe," which requires careful planning and debugging. Often, developers have to rewrite entire applications (or significant chunks of them) when they want to add support for threads. The usual problem spots are data structures that are declared local and static.

The final concern is more subtle and subjective: Because using threads can speed up performance, developers sometimes neglect to optimize the rest of the code in their applications. They often assume that because an application uses threads, it will run faster no matter what, so there's no reason to spend time tweaking the rest of the code. In reality, threads by themselves are no guarantee of speed.

In response to numerous problems associated with threads, the Apache 2.0 group implemented the concept of Multiple Processing Modules (MPMs). The modules let developers choose the underlying method for handling requests. Each MPM is best suited for a specific environment. In Unix, there are three main MPMs: prefork, threaded, and perchild.

The Prefork MPM

The prefork MPM avoids threads completely. In fact, it causes Apache 2.0 to operate in the same way as Apache 1.3. All request and task handling is done via Unix processes. To avoid the nasty overhead of forking a new process for each request, the Web server creates several processes as soon as it starts. These processes then wait around, listening for requests, and as each request comes in, one process takes responsibility for it. When a request is finished, the associated process rejoins its siblings waiting for more incoming work. The name "preforking" comes from the fact that the processes are forked before the work is done.

This MPM keeps enough idle processes available to handle bursts of requests, but not so many that server resources are wasted on unused processes. Two main Apache directives control this behavior: MinSpareServers and MaxSpareServers. Although these directives refer to servers, they truly mean processes. The MinSpareServers directive defines the minimum number of idle Unix processes that Apache should guarantee are always available. For example, if MinSpareServers is set to 10 and Apache sees that only 6 idle processes are available (because the others are busy handling requests), the server will fork 4 new processes. Idle processes are sort of like insurance: You keep them available just in case you get a big burst of activity. Without enough idle processes, Apache would be forced to spend time creating new ones at the most inopportune time—when users are trying to access Web pages.

MaxSpareServers handles the other end of the spectrum. When Apache sees too many idle processes floating around, it terminates them to avoid wasting server resources. If MaxSpareServers is set to 20, and Apache sees 25 idle processes, it will start killing off some of those idle processes to get the number below the setting. The combination of these two directives balances responsiveness with resources.

There's also a directive called MaxClients that plays an important role in the prefork MPM. This directive limits the total number of processes that Apache can have active at once, regardless of its state. This prevents Apache from monopolizing the operating system's available resources, which could prevent other applications from performing properly.

The big advantage of the prefork MPM is that if your platform doesn't support threads, or if some of your Apache modules aren't thread safe, you can continue using Apache 2.0. Also, because a dedicated process handles each request, the server as a whole is extremely reliable and robust. If a buggy module causes that process to fail and "dump core," only that request is affected. So if your primary goal is reliability, or if you're using Apache modules that may not be completely bug free or thread safe, then choose the prefork MPM.

The Threaded MPM

One alternative to prefork is the threaded MPM. When Apache starts, this MPM creates several child processes, similar to what the prefork MPM does. These processes, however, don't deal directly with requests. Instead, each is used as the controller for task threads that actually handle the requests. Each has a set number of threads. As the server gets busier, Apache forks off additional processes as required to increase the pool of available threads. In this MPM, the number of processes can vary, but the amount of threads per process remains static. This results in a fast, scalable server.

With this MPM (and as with the prefork MPM), you would use the MaxClients directive to denote the absolute maximum number of available processes. However, you also need to determine the number of threads that each process will start. Set this value with the ThreadsPerChild directive. Thus, the absolute total number of requests that Apache 2.0 could handle at the same time is the product of the MaxClients and ThreadsPerChild values.

The threaded MPM also makes use of the MinSpareThreads and MaxSpareThreads directives to control when new processes are created, and when old ones are removed. Based on the values you provide, Apache counts the total number of idle (spare) threads in all processes, and adds or removes processes to keep that number between MinSpareThreads and MaxSpareThreads. So if, for example, the number of idle threads is less than MinSpareThreads, Apache will fork off an additional process to add to the thread pool.

Because the threaded MPM forks child processes as needed, it tends to favor robustness over scalability. For example, if you were using third-party modules that weren't as reliable as you'd like, you'd most likely favor a mix of more child processes with fewer threads. This way, if a process dies, it only takes down a small number of requests. As your confidence in the module's reliability increases, you can ramp up the number of threads to increase performance. The threaded MPM is very forgiving, in that it lets you tune the runtime operation of Apache on the fly. For platforms that support threads, threaded is the default MPM for Apache 2.0.

The Perchild MPM

perchild is the third MPM for Unix, and it follows the path that prefork and threaded laid down. With prefork, you adjust the number of processes and don't use threads at all. With threaded, you use a set number of threads per process, and the number of processes varies. With perchild, you keep the number of child processes constant and adjust the number of threads as necessary. (This is based on an older MPM known as dexter.)

For perchild, you no longer have to worry about minimum or maximum numbers of idle processes. Instead, you control the number of idle threads—as you did with the threaded MPM. You do this by using the MinSpareThreads and MaxSpareThreads directives. The behavior of these directives with perchild is somewhat different from their behavior with the threaded MPM. In threaded, these were absolute numbers that limited the total threads in all processes. In perchild, these directives specify thread limits for each process. Apache tries to keep the number of idle threads in each process between these boundaries.

You also need to define the total number of processes that Apache will start. This is done with the NumServers directive, not with MaxClients (as you might have expected) for perchild. Because the number of processes doesn't change, the idea of a maximum number really doesn't make sense, and the new directive helps clarify that distinction.

Finally, you need to limit the total number of threads ever available per process, and this is done using the MaxThreadsPerChild directive. Thus, the absolute limit in simultaneous requests is the product of NumServers and MaxThreadsPerChild.

Because perchild favors threads above processes, it's the most scalable of all MPMs. It can be the fastest as well. Unfortunately, this also makes it the least robust. You could, for example, have a setup with only two processes, each with 1000 threads. If one process dies because of a buggy module, you've lost half of your capability and possibly dropped half of your requests. As a result, the perchild MPM is favored by developers who use thread-safe modules or serve mostly static content.

The perchild MPM has one more nifty feature that sets it apart: It lets you control the user ID under which various processes and virtual hosts run. This is useful because it extends down to the module level. The full implications are beyond the scope of this article, but expect a follow-up article soon with more detail.

A Stitch in Time

All of us have anticipated Apache's support for threads under Unix. Instead of just adding threads as a replacement for processes, the Apache group built in more control and balance for processes and threads. MPMs let you easily and logically partition the actual mechanics of how Apache handles requests, and the underlying platform. MPM implementation is robust enough that new and more specific MPMs can be added as needed to better support platforms in the future. Apache 2.0 now has MPMs specifically designed for Windows, BeOS, and OS/2 to take full advantage of these operating systems.

The Apache Web server group's focus is extensibility, modularity, and control. MPMs focus on the third leg, giving you control over the robustness and scalability of your applications. Tune each as you see fit.


Jim has been active on the Web since the 80s. Best known as one of the core Apache developers, he can be reached at jim@jaguNET.com.




Copyright © 2003 CMP Media LLC