QA Best Practices

Over the yeas, Accion Labs has provided services in Manual, Automation & Performance testing of Desktop, Web & Mobile applications. We have over 250 SDeTs engaged in Automation testing of various projects and along the way, we have picked up some best practices that we use on a daily basis to improve quality outcomes for our clients. Before we dive into best practices, it's useful to look at landscape of various technologies and frameworks for Q/A in today's world. From testing of non-UI components, to mobile applications and from functional testing to performance and regression, Q/A has to cover a lot of ground these days.

PHASES FOR Q/A TESTING

Q/A testing process cannot be an afterthought. In fact, Q/A starts along side software development. Unit testing is in fact something that developers need to perform to ensure that the component being developed does what is intended to do. As the software evolves, unit tests are modified, serving as an up-to-date form of documentation. Unit tests should provide the following test coverage:

• Function coverage: Each function/method executed by at least one test case.
• Statement coverage: Each line of code covered by at least one test case.
• Path coverage: Every possible path through code covered by at least one test case.

Functional Testing is the next phase in Q/A testing lifecycle and it should address concerns surrounding the correct implementation of functional requirements. Functional test suites are created from requirement use cases, with each scenario becoming a functional test. Functional testing are carried out by the test engineers which ideally should be part of a separate/independent team. Regression Testing goes along with functional testing and it should ensure that code modifications have not inadvertently introduced bugs into the system or changed existing functionality. And finally, Performance Testing should be performed to ensures that the performance of the application does not go down as load on the application increases. It offers a great opportunity to identify performance bottlenecks and remove them before software is released to market.

Q/A BEST PRACTICES AND TEST METRICS

Based on our experience, following Best Practices should be used for an effective testing and QA implementation:

1. People: Whether you call them Q/A resources or automation engineers or just testers, it's important that people who are working on Q/A have a thorough knowledge of software fundamentals. At Accion, we prefer to call them Automation Engineers. It is critical that all automation engineers have a software development background and the understand the importance of good design in the field of software development.
2. Tool Usage: Use tools for tracking and managing defects, as well as the creation of test cases and execution. Don't rely on spreadsheets or notes. Good tools can increase the maturity of the testing & QA process.
3. Metrics: Develop and create metrics to track the software quality in its current state, as well as to compare the improvement with previous versions. This helps in increasing the value and maturity of the testing process. The following are some of the metrics we recommend that you use at the Project level:
   • Requirements Coverage = (Number of requirements covered/Total number of requirements) * 100
   • Defect Distribution by Status and Phase = (Total number of defects /Functional area(s))*Status * Phase
   • Defect Open and Close Rates = (Defects found before delivery/(Defects found before delivery + defects founds after delivery)) * 100
   • Base Metrics = These should be collected by automation engineers throughout the testing effort:
     • Number of test cases
     • Number of test cases passed
     • Number of test cases executed
     • Number of test cases failed
     • Number of test cases under investigation
     • Number of test cases blocked
     • Number of test cases re-executed
     • Number of first run failures
     • Total executions
     • Total passes and failures
     • Test case execution time
     • Test execution time
   • Derived Metrics = These metrics are calculated from base metrics for reporting purposes:
     • Percentage of test cases passed
     • Percentage of test coverage
     • Percentage of defects corrected
     • Percentage of test cases blocked
     • Percentage of rework
     • Percentage of test effectiveness
     • First run fail rate
     • Defect discovery rate
     • Overall fail rate
4. Testing Environment: Implement an appropriate testing environment that allow developers to reproduce the system execution in production environments. This is very useful because it not only helps in execution of corresponding test cases but also creation of new test cases.
5. Test Data: Test data should be as close to real life as possible and it should also include scenarios that result in negative testing. Same test cases run with different test data can produce different results for many complex applications today.
6. Change Management: Like in production environment, testing environment should also track changes in configuration, ensuring not only controlled results, but that the tests are run in environments that closely resemble those of the real production environment.

So, there you have it folks. If you follow above best practices and use the test metrics in a disciplined way, quality of your software will definitely improve.