Once you have decided which test tasks you want to automate, you can begin the tool selection process. Because the cost of investing in a new tool can be very high (see above), you need to plan the process carefully. The tool selection process comprise five steps:
- Specify the requirements for the planned usage
- Research and creation of a list of suitable candidates
- Demos and trials
- Create a shortlist based on your requirements
- Proof of Concept
Selection criteria
In Step 1, the following criteria influence requirements specification:
- Compatibility with the targeted test objects
- Tester know-how regarding the selected tool and/or the test task it supports
- Simplicity of integration into the existing development and test environment
- Potential integration with other existing (test) tools
- The platform the tool is due to run on
- Manufacturer support, reliability, and reputation
- Benefits and drawbacks of the available licensing options (for example, commercial vs. open-source, purchase vs. subscription)
- Price and cost of maintenance
These and other criteria need to be listed and prioritized. Deal-breaker (i.e., essential) criteria need to be flagged as such7.
Market research and shortlisting
The next step is to compile a list of the available tools in each category that includes a description of each. You can then check out the best-sounding choices in-house or have the manufacturer demonstrate them. These experiments and demos usually make it fairly obvious which products work best for you and which manufacturers offer the best service. The most appropriate tools are then put on a shortlist and you need to answer the following questions for the shortlisted products:
- Is the tool compatible with your test objects and your existing development environment?
- Do the attributes that got the tool onto the shortlist function as required in real-world situations?
- Can the manufacturer provide professional-grade support for standard and non-standard queries?
Leave a Reply