Testertools Forum

1. Pamela
   Member

   Reputation:297

    

   Hello -

   I would like to initiate a discussion on the drawbacks related to Performance and Load testing.

   When we read literature on Load tools we are told of the many benefits of Load testing a web application to ensure scalability, uptime, reliability e.t.c.

   But what are the drawbacks?
   What negative experiences have people had?

   I guess that some negative points are:-

   Cost - You need to spend money in choosing the correct product. Purchasing the tool and hardware. Hiring specialist testers. Training them e.t.c

   Poor planning - Without a plan with clearly defined goals it is possible to produce meaningless results.

   Inacurrate simulation - Difficult to simulate expected usage behaviour correctly. This would lead in meaningless results.

   Inaccurate workload definition - Predicting how many people use a site is a difficult exercise. Little investigation would lead in meaningless results.

   As a test consultant I have to make sure that the client is aware of the Risks associated with performance testing. Some of these Risks can be construed as excuses for when things go wrong. :-)

   Posted 1 year ago #

2. Davidcrp
   Member

   Reputation:291

    

   I think you covered the majority.

   Most times the goals and results far outway the drawbacks. What customer will use your site if it takes 30 seconds for a page to be displayed? Not many that is for sure. These points must always be stressed.

   If you read the book "Scaling for E-Business.. Technologies, Models, Performance, and Capacity Planning" by Menasce and Almedia you will be armed with the vital information that your customer will need to hear.

   Let's not forget the goal of performance /load testing is to measure these site characteristics: volumes (hits, page views, transactions, searches), arrival rates, response times by class, user session time, number of concurrent users, and processor and disk utilization, etc.

   There are several reasons for performance/load testing.

   Measuring end to user response time.
   Defining optimal hardware configuration.
   Checking reliability.
   Checking hardware or software upgrades.
   Evaluating new products.
   Identifying bottlenecks.
   Measuring system capacity.

   Here is also a good article on IBM's site: http://www-4.ibm.com/software/developer/library/w-capacity/?dwzone=web

   Posted 1 year ago #

3. Patricia
   Member

   Reputation:218

    

   One other point I want to provide is also the fact that a single tool is probably not enough to do true performance testing.

   Monitors should be placed on every component involved in the application to get an idea of true load. Disk, Proc, Mem usage on every box. Response time on DB, web servers, ftp sites, etc.

   Some of this is overkill, but if you need more fuel to the fire, maybe that could provide it ;-)

   I don't put monitors on all pieces all the time, but after some initial risk assessment, I put monitors on the pieces that I see as critical.

   Load/Perf testing also can be very time consuming. I like to get this phase of testing up and running as soon as the first release of the software is delivered... better know sooner than later what the hazards might be.

   Posted 1 year ago #

4. Brandan
   Member

   Reputation:227

    

   Another drawback can be dissimilar environments. It is generally not always easy to replicate environments exactly (which figures into your cost perspective) and thus your results (unless you do statistical analysis and capacity modeling) can be contraindicative to what you will really see in a production environment.

   However, the areas you cover are dead-on accurate. For example, one thing that people very rarely consider (and I tout all the time on this forum) is the modeling of burst traffic and heavy-tail distributions for distributed environments, particularly open-ended ones such as the WWW. Lately, what has not been taken account of is different access methods that might be used and doing cross scenarios for these.

   The other problem is similar to the problem in quantum theory: if you observe too much, you disturb that which you are observing. But if you observe too little you do not get a good picture. Thus a balance must be struck between things like time-series analysis and queuing models. Placing monitors all over is great - but not if they start to impact the results you see in non-verifiable ways. As Alanah mentions, one way is to only monitor that which you perceive to be critical.

   Posted 1 year ago #

RSS feed for this topic

Reply

You must log in to post.