QA - Load and Performance Tool Selection Criteria
- Load and Performance Objectivces: The selected tools need to enable you to go after your load and performance objectives / goals. If you have not defined what you want to get out of your load and performance tests, you are not ready to select a tool. It seems obvious, but it is a easy trap to fall into.
- Customer Perception: Will the usage of a certain tool give you credibility in the eyes of your customers? One of the reasons we selected the use LoadRunner at Lombardi Software, when we were a startup, was that it showed our customers / potential customers that we were a mature player and used enterprise tools.
- Ownership and Control: This question becomes especially important when making a selection between a 'hosted performance tool' or deploying your own. A hosted tool is convenient because the 'host' will do most of the work, from coding your usage scenarios, executing the tests, generating reports and providing assistance. The price you pay is that you cannot run your tests whenever and how oftern you want to. Plus your assets are 'held captive' by the third party. Deploying/owning your own tool is nice because it gives you flexibility to run the tests whenever and how often. Plus you own the assets. On the other side, you have to invest a considerable amount of time in the development and maintenance of the scripts. Also, do we as an Engineering team have the knowledge to iterpret the results of the test execution. Who is the best person for the job?