HSC welcomes all external visitors to this site, especially students and members of the academic community. Please use the comments box at the bottom of each page to record any comments or suggestions for improvement.

 Test Automation 

Agenda

• What is Test Automation?
• Why Test Automation is required?

1. High expectations from Testers.
2. The number of people involved and the diversity of the environment.
3. To perform Regression testing and load testing.
4. Efficient utilization of resources.

• How test automation can be achieved?
• Test Automation advantage.
• Challenges.
• Conclusion

Testing is the most important phase in a SDLC cycle and many a time the developer resorts to Test Automation for ease and better maintenance purpose for posterity. In case of turnkey situation when the test environment is in place, the Developers job for test automation is made easy. But if developers have to write the scripts and have to understand the testing tool and if the configurations for the testing tool are complex then it becomes even more likely that testing will be done manually.

Before discussing further about Test Automation lets first define Test Automation.

What is Test Automation? Test Automation is a process to use a software for testing the target software. The automation software will take care of test case execution, order of execution, reporting of bugs and maintenance of test cases.

Why Test Automation is required?

1. High expectations from Testers: With Automatic code generation becoming more common in software development there is continuous increasing gap between the software development and software testing. Testers are being asked to test more and more code in less and less time.
2. The number of people involved and the diversity of the environment: If the project is a large scale project where number of modules and developers involved are more than 3 - 4 developers per module, then it is advised to use test automation. In case of large project (in terms of KLOC and number of modules) there will be too many interfaces involved. Some times even the different software environment be involved. Test automation will reduce the risk of bug finding at later stage of software development and will ensure a stable build at each stage of software development.
3. To perform Regression testing(repetitive task) and load testing: Though some times it is not time line involved but the test input parameters are constraints. Take a case of database application where testers have to perform load testing, what happens when more than 100 request for database access are received in 1 sec. Or test could be how many different users can access a Web page under test simultaneously. Is it practically possible to have 200-300 testers just to the test case for the web application.
4. Enable a team to more fully utilize its testing resources. e. g. - automated testing can give you way to keep that equipment active during off hours thus reducing the need for more equipment.
5. For test maintainability.

Also many a times you want to rerun the test case just to ensure the success of the test case. Take a case while you have executed a manual test case and encountered a bug and now you don't remember the exact configuration or execution sequence. All the above points may not be valid for all the projects. Even if few or all of the points are valid for any particular project, is it practically possible to achieve 100% automated testing. We can have answers for all these questions once we will start with the process of automation.

How test automation can be achieved?

 Assumptions :

1. Team has complete understanding of the feature and how to test it.
2. The project is stable enough to make the investment of automation.
3. The project has standardized and effective testing procedures and practices.

I Analyze what is good/bad to automate. How to make a rational decision? Ask few questions to yourself.

Automating this particular test and running it once will cost how much more than simply running it once manually? The answer vary widely depending upon the project:

one answer could be - If I automate this test I'll probably lose 3 manual tests. But I have done a pretty complete job of test design and I really think those additional test cases would only be trivial variations of the existing tests. And I really doubt they'd find some serious new bugs. - in this case the cost of Automation is low.

Where as if the project phase is like - I don't have time to try all the obvious tests once. In the time I would spend automating this test , I guarantee I could find at least one complete new bug. - in this case the cost of Automation is high.

Above question is just one of the type. And as I said the answer vary for different project. The above two answers are just the example of two different scenarios of two different projects. One should look for the budget, effort estimate(e.g. -complexity involved in making test scripts), time spent and re-usability of the automated test cases. The size of the team and as I mentioned above it is really essential that the team has a complete understanding of the project features. And project has well defined testing standards.

Here re-usability of test cases will be an important factor to define the cost. If the automated test case that can not be reuse for the subsequent incremental release then selection of such test cases for automation is a bad choice.

All the factors stated above puts constraints on test automation for the project. And mitigation of these constraints land us in a situation where it is practically not possible to achieve 100% automation.

II Who will automate the test? If there is different testing team, at what phase of project the testing team will start working upon? Is it the developers responsibility to provide the test scripts or it is testers responsibility to take input from developers and write the test scripts for test automation. Are the developer and tester in sync? Are they comfortable with the test environment?

There could be different testing team if the project is big size project and there are phase deliveries with each feature to be tested thoroughly. And here testing team could be involved in the early phase of project development i.e. Code development and test case development will go hand in hand.

The case will be different if the project is small size project. There you may not have a different testing team and it will be developers responsibility to write the test scripts and perform the testing. In this case it will be difficult to achieve test automation. Most likely the developers will eat-up more than 50% of the testing time with the development activity. And then either there will be very few automated test cases or the quality of the automated test cases will be very poor. It is really very essential to decide on test tool at the early stages of development. Delaying of which will result in poor quality of test cases because the users are not well aware of the test environment.

III Building maintainable test suites. It is advisable to use the test suites. This will help in maintaining the test execution sequence. The execution sequence will lead to improve the quality of the product.

IV Make sure that your tests are actually testing. Following are the few approaches: 1. Conduct manual reviews for the test code. The reviewer shall have familiarity with the project feature as well as with the test tool. 2. Execute the test scripts in debug mode, of course if the facility is been provided by the test tool. 3. To check the validity of test script make some changes in the system under test. Execute the test script and check against the expected behavior.

Test Automation advantage 1. Software testing using an automatic test program will generally avoid the errors that humans make when they get tired after multiple repetitions. 2. The test program won't skip any tests by mistake. 3. The test program can also record the results of the test accurately. The results can be automatically fed into a database that may provide useful statistics on how well the software development process is going. 4. Testing your own work is generally thought of as risky since you'll be likely to overlook bugs that someone not so close to the code (and not so emotionally attached to it) will see easily. 5. The main benefit from automated tests is instant, reliable feedback: "The developers get direct feedback after implementing changes. The customer gets concrete feedback about the current state of their product. Automated tests show the real progress of the project." More changes can be made more quickly with less problems, with tests run several times a day after every change. The end result should be a better final product.

Challenges

of the code will be more beneficial to automate. Which part of code will be more stable? Which test tool will be more suitable for the project requirements?

Conclusion

Please keep yourself away from Test Automation if it just for the sake of having Test Automation. Evaluate your project for Test automation and happy testing.

Outside Resources

1. http://en.wikipedia.org/wiki/Test_automation
2. http://www.nokia.com/A4211021
3. http://www.informit.com/articles/article.asp?p=379757&rl=1
4. http://www.io.com/~wazmo/papers/seven_steps.html
5. http://www.testing.com/writings/automate.pdf
6. OpenSourceTestAutomationFrameworksPaper  Δ

Categories: Software

Comments

Add Comment
Email address(will be kept hidden)
Enter code: