Software Testing Tools Question:
Software Test Automation Tool Evaluation Criteria Part 2
Answer:
Performance and Stress Testing
* Performance and stress testing tool is integrated with GUI testing tool
* Supports stress, load, and performance testing Allows for simulation of users without requiring use of physical workstations
* Ability to support configuration testing (that is, tests can be run on different hardware and software configurations)
* Ability to submit a variable script from a data pool of library of scripts/data entries and logon IDs/password
* Supports resource monitoring (memory, disk space, system resources)
* Synchronization ability so that a script can access a record in database at the same time to determine locking, deadlock conditions, and concurrency control problems
* Ability to detect when events have completed in a reliable fashion
* Ability to provide client to server response times
* Ability to provide graphical results
* Ability to provide performance measurements of data loading
Version Control
* Does the tool come with integrated version control capability?
* Can the tool be integrated with other version control tools
Test Planning and Management
* Test planning and management tool is integrated with
GUI testing tool
* Test planning and management tool is integrated with requirements management tool
* Test planning and management tool follows specific industry standard on testing process (such as SEI/CMM, ISO)
* Supports test execution management
* Allows for test planning--does the tool support planning, managing, and analyzing testing efforts? Can the tool reference test plans, matrices, and product specifications to create traceability?
* Allows for measuring test progress
* Allows for various reporting activities
Pricing
* Is the price within the estimated price range?
* What type of licensing is being used (floating, fixed)?
* Is the price competitive?
Vendor Qualifications
* Maturity of product
* Market share of product
* Vendor qualifications, such as financial stability and length of existence. What is the vendor's track record?
* Are software patches provided, if deemed necessary?
* Are upgrades provided on a regular basis?
* Customer support
* Training is available
* Is a tool Help feature available? Is the tool well documented?
* Availability and access to tool user groups
* Performance and stress testing tool is integrated with GUI testing tool
* Supports stress, load, and performance testing Allows for simulation of users without requiring use of physical workstations
* Ability to support configuration testing (that is, tests can be run on different hardware and software configurations)
* Ability to submit a variable script from a data pool of library of scripts/data entries and logon IDs/password
* Supports resource monitoring (memory, disk space, system resources)
* Synchronization ability so that a script can access a record in database at the same time to determine locking, deadlock conditions, and concurrency control problems
* Ability to detect when events have completed in a reliable fashion
* Ability to provide client to server response times
* Ability to provide graphical results
* Ability to provide performance measurements of data loading
Version Control
* Does the tool come with integrated version control capability?
* Can the tool be integrated with other version control tools
Test Planning and Management
* Test planning and management tool is integrated with
GUI testing tool
* Test planning and management tool is integrated with requirements management tool
* Test planning and management tool follows specific industry standard on testing process (such as SEI/CMM, ISO)
* Supports test execution management
* Allows for test planning--does the tool support planning, managing, and analyzing testing efforts? Can the tool reference test plans, matrices, and product specifications to create traceability?
* Allows for measuring test progress
* Allows for various reporting activities
Pricing
* Is the price within the estimated price range?
* What type of licensing is being used (floating, fixed)?
* Is the price competitive?
Vendor Qualifications
* Maturity of product
* Market share of product
* Vendor qualifications, such as financial stability and length of existence. What is the vendor's track record?
* Are software patches provided, if deemed necessary?
* Are upgrades provided on a regular basis?
* Customer support
* Training is available
* Is a tool Help feature available? Is the tool well documented?
* Availability and access to tool user groups
| Previous Question | Next Question |
| Software Test Automation Tool Evaluation Criteria Part 1 | Comparing SilkTest and WinRunner Part 1 |