Software Testing Tools Question:
Software Test Automation Tool Evaluation Criteria Part 1
Answer:
Ease of Use
* Learning curve
* Easy to maintain the tool
* Easy to install--tool may not be used if difficult to install
Tool Customization
* Can the tool be customized (can fields in tool be added or deleted)?
* Does the tool support the required test procedure naming convention?
Platform Support
* Can it be moved and run on several platforms at once, across a network (that is, cross-Windows support, Win95, and WinNT)?
Multiuser Access
* What database does the tool use? Does it allow for scalability?
* Network-based test repository--necessary when multiple access to repository is required
Defect Tracking
* Does the tool come with an integrated defect-tracking feature?
Tool Functionality
* Test scripting language--does the tool use a flexible, yet robust scripting language? What is the complexity of the scripting language: Is it 4 GL? Does it allow for modular script development?
* Complexity of scripting language
* Scripting language allows for variable declaration and use; allows passing of parameters between functions
* Does the tool use a test script compiler or an interpreter?
* Interactive test debugging--does the scripting language allow the user to view variable values, step through the code, integrate test procedures, or jump to other external procedures?
* Does the tool allow recording at the widget level (object recognition level)?
* Does the tool allow for interfacing with external .dll and .exe files?
* Published APIs--language interface capabilities
* ODBC support--does the tool support any ODBC-compliant database?
* Is the tool intrusive (that is, does source code need to be expanded by inserting additional statements)?
* Communication protocols--can the tool be adapted to various communication protocols (such as TCP/IP, IPX)?
* Custom control support--does the tool allow you to map to additional custom controls, so the tool is still compatible and usable?
* Ability to kick off scripts at a specified time; scripts can run unattended
* Allows for adding timers
* Allows for adding comments during recording
* Compatible with the GUI programming language and entire hardware and software development environment used for application under test (i.e., VB, Powerbuilder)
* Can query or update test data during playback (that is, allows the use of SQL statements)
* Supports the creation of a library of reusable function
* Allows for wrappers (shells) where multiple procedures can be linked together and are called from one procedure
* Test results analysis--does the tool allow you to easily see whether the tests have passed or failed (that is, automatic creation of test results log)?
* Test execution on script playback--can the tool handle error recovery and unexpected active windows, log the discrepancy, and continue playback (automatic recovery from errors)?
* Allows for synchronization between client and server
* Allows for automatic test procedure generation
* Allows for automatic data generation
Reporting Capability
* Ability to provide graphical results (charts and graphs)
* Ability to provide reports
* What report writer does the tool use?
* Can predefined reports be modified and/or can new reports be created?
* Learning curve
* Easy to maintain the tool
* Easy to install--tool may not be used if difficult to install
Tool Customization
* Can the tool be customized (can fields in tool be added or deleted)?
* Does the tool support the required test procedure naming convention?
Platform Support
* Can it be moved and run on several platforms at once, across a network (that is, cross-Windows support, Win95, and WinNT)?
Multiuser Access
* What database does the tool use? Does it allow for scalability?
* Network-based test repository--necessary when multiple access to repository is required
Defect Tracking
* Does the tool come with an integrated defect-tracking feature?
Tool Functionality
* Test scripting language--does the tool use a flexible, yet robust scripting language? What is the complexity of the scripting language: Is it 4 GL? Does it allow for modular script development?
* Complexity of scripting language
* Scripting language allows for variable declaration and use; allows passing of parameters between functions
* Does the tool use a test script compiler or an interpreter?
* Interactive test debugging--does the scripting language allow the user to view variable values, step through the code, integrate test procedures, or jump to other external procedures?
* Does the tool allow recording at the widget level (object recognition level)?
* Does the tool allow for interfacing with external .dll and .exe files?
* Published APIs--language interface capabilities
* ODBC support--does the tool support any ODBC-compliant database?
* Is the tool intrusive (that is, does source code need to be expanded by inserting additional statements)?
* Communication protocols--can the tool be adapted to various communication protocols (such as TCP/IP, IPX)?
* Custom control support--does the tool allow you to map to additional custom controls, so the tool is still compatible and usable?
* Ability to kick off scripts at a specified time; scripts can run unattended
* Allows for adding timers
* Allows for adding comments during recording
* Compatible with the GUI programming language and entire hardware and software development environment used for application under test (i.e., VB, Powerbuilder)
* Can query or update test data during playback (that is, allows the use of SQL statements)
* Supports the creation of a library of reusable function
* Allows for wrappers (shells) where multiple procedures can be linked together and are called from one procedure
* Test results analysis--does the tool allow you to easily see whether the tests have passed or failed (that is, automatic creation of test results log)?
* Test execution on script playback--can the tool handle error recovery and unexpected active windows, log the discrepancy, and continue playback (automatic recovery from errors)?
* Allows for synchronization between client and server
* Allows for automatic test procedure generation
* Allows for automatic data generation
Reporting Capability
* Ability to provide graphical results (charts and graphs)
* Ability to provide reports
* What report writer does the tool use?
* Can predefined reports be modified and/or can new reports be created?
| Previous Question | Next Question |
| Explain Web Application Testing Tool Web Form Flooder? | Software Test Automation Tool Evaluation Criteria Part 2 |