How Many Virtual Users Per Machine?

One of the more common questions we hear from load testers is how many virtual users can be run from one load test generating computer.  For testers wanting to exercise their web application with thousands, tens of thousands, or even hundreds of thousands of virtual users, understanding how many test generators are needed helps them appropriately plan for load testing activities.

The implications of using too few load test generators can be significant.  If the processors are too slow or the machines don’t have enough memory, it could prevent the test from accurately creating and measuring load on the application.  In particular, the test will likely both underreport the extent of the load on the server, because it’s not able to generate the load and measure performance as expected.

So how do testers know how many load generators are needed for a particular test?  Our rough rule of thumb is that for a fast load generator computer (either running Test Studio or a remote execution agent), it is possible to run about 500 virtual users per core.  A fast computer is one in which the processors are clocked at 2.2 GHz or higher, and with about 4 GB of main memory.

That’s a good starting point, but of course, your mileage may vary.  You may have a slower computer, fewer cores, less memory, or slower disk drives.  Or your scripts may be long and complex.  Alternatively, the web pages created may be larger than average size.  With additional bandwidth needs, it’s important to be able to design the load testing environment accordingly.

So testers need a more accurate way of identifying how many computers needed to generate the number of load test virtual users we need.  In particular, we have to evaluate how much of each relevant resource each virtual user consumes.

There are a number of Test Studio processes that are involved during the execution of a load test.  These include Telerik.TestStudio.Load.Controller, Telerik.TestStudio.Load.Agent, Telerik.TestStudio.Scheduling.Client, and Telerik.TestStudio.Profiler.  If the test involves remote execution agents, then Telerik.TestStudio.RemoteExecutor comes into play.  The first two start up when a load test starts, and close when the test is complete.  The others are running whenever Test Studio runs in general, but still have a role in load testing.

With all of that as background, here’s the steps that I recommend (and have experimented with) in order to more exactly determine your load generator resource requirements.

1. Close all non-essential applications and processes.  Any use of memory or processing power could affect the results.

2. Record a sample load testing script, preferably from the application under test, or capture and filter web traffic to create a script.

3. Run that sample as a single-user load test in order to get baseline resource utilization.

4. Record the amount of memory and CPU percentage used by the different processes.

5. This provides a baseline for a single virtual user.  Run the script again with ten virtual users.

6. Multiply those numbers by the desired number of virtual users.

To further validate that number, run the script again with a hundred virtual users.  Look at the increase in CPU and memory for the next level of usage, and use that to estimate the resource needed for the required number of virtual users.

At the same time, it’s important to monitor network traffic.  While network is generally less of a limiting factor than server resources, it is an important consideration in making sure that the test is accurate.

Once you have an idea of the memory and CPU footprint for virtual users, use these figures to define the memory and CPU requirements for a larger test.  Prepare a graph that illustrates how much memory and CPU increases when users are added, in order to get an idea of how these resources are used in generating and managing virtual users.

They will, of course, be different for different load generation computers.  And memory is somewhat fungible, in that it consists of both main memory and swap files.  But knowing this information will help you determine the number of execution agents you need to create the right load test for your application.

For more information on load testing with Test Studio, check out the Telerik website.  Also look at the new load testing features in Test Studio 2013 R2, and in particular load testing Web services.

Peter Varhol

About the author

Peter Varhol

Peter Varhol is an Evangelist for Telerik’s TestStudio. He’s been a software developer and software product manager, technology journalist, and university professor among the many roles in his past, and believes that his best talent is explaining concepts and practices to others. He’s on Twitter at @pvarhol.