CPU Sizing for Concurrent Client Connections to EAServer
How many simultaneous client connections do you think is reasonable for a server?
This document helps you understand how to answer that question, by examining
response times in a sample application scenario.
Scenario
Test_Servlet1 - with a known response time of 1.5 seconds -- is deployed
to EAServer on ServerMachine1.
ServerMachine1 is an Windows or Solaris box, with 2 CPUs.
Each minute, there are 600 concurrent client requests for Test_Servlet1.
Result
ServerMachine1 begins thrashing, EAServer generates messages that the server is busy, and/or may hang.
Explanation
ServerMachine1 is undersized.
When there are 600 requests per minute, that averages out to 10 requests per
second.
Since there are 2 CPUs, each CPU receives half of the requests; 5 requests
per second.
In order to process 5 requests per second, the request would have to complete
in 0.2 second.
Since the response time for Test_Servlet1 is actually 1.2 seconds, the request
cannot be completed.
Therefore, the requests wait in line for a small slice of CPU time, then
go to the end of the line to wait for another slice, and repeat this until
it received enough slices to be able to complete. This causes a backlog,
and thrashing begins.
For example, imagine a CPU with backlog of even 50 requests for Test_Servlet1
that are waiting to complete. The first request in line gets one fiftieth
of a second, then goes to the end of the line to wait for another fiftieth
of a second, and this continues. The request may never get to finish when
more and more requests are coming in and waiting in line.
To make it worse, many of the end-users start hitting the browser Refresh button
since a response did not come back quickly enough.
(The presence of many disconnect errors in the EAServer log file
can be a telltale sign that this is happening.)
Although a Unix machine would show 100% CPU usage in this type of scenario,
Windows may only show 60-70% CPU usage.
To identify the problem, use a CPU monitoring tool (such as PSTAT on Windows)
to view the number of threads waiting, and their average execution time.
The Solution
Upgrade the server.
As an average guideline, assuming the application code is designed and tuned
properly:
- Solaris: consider allocating 100 concurrent requests per CPU
- Windows: allocate 75 concurrent requests per CPU, for the first two CPUs.
After 2 CPUs, testing has shown that performance is not linear, so you will
need to stress test your application to determine appropriate sizing.
For more specific tuning suggestions, see the TechWave 2003 presentation Application
Server Benchmarks and Sizing.
Although it may reduce scalability, you can also prevent a backlog of requests by limiting the maximum number of connections to EAServer.
More Information
Guidelines and suggestions for EAServer tuning are available in the
EAServer Performance and Tuning Guide.
