What is defect leakage? In simple words, “Defect Leakage is the metric that is used to identify the efficiency of the QA testing i.e., how many defects are missed/slipped during the QA testing. “

It is calculated using a simple formula: Number of defects found in production or UAT / Number of Defects found in testing. 

The higher the number, the more cause for alarm.

Though organizations nowadays believe in building QA culture, adoption of Agile and DevOps still many companies consider QA as the only gatekeepers for quality and fully responsible for skipping bugs in production. 

No matter how much you test, there are so many bug types in software testing, that a few bugs can still slip through the testing phase & reach production. Every software tester would have come across a situation of a missed bug in software testing. For many testers, it’s a terrifying nightmare. Such situations can be overwhelmingly stressful and scary!

So, Here is the guide for a  QA to handle skipped bugs in production. 

Avoid blaming games: Dealing with your angry customers and annoyed managers while figuring out what went wrong requires you to stay calm and avoid blaming others. Instead, take a deep breath and focus on planning next steps.

Don’t Panic. There is no such software that is bug-free. Exhaustive testing is impossible with limited time during regression, resources and budget. Additionally, there will always be differences between testing and production environments. 

Reproducing the issue: Go through the bug ticket, talk to the customers or support team, look at logs or monitoring tools and try to reproduce the issue as an early step. 

Identify a workaround: If there is a workaround, unblock your users first and let them know alternative ways of achieving the task (Ex: reinstall app, rollback to older version). If required, work with customer support & Dev teams to find out the workaround. Every missed bug to production need not have to be fixed immediately except blocker or critical issues. 

Roll out a quick fix:  Work with your team to solve the problem as quickly as possible. Do not look for a perfect solution while sending a patch, but rather one that unblocks the customer soon. Ensure that you test the patch before the fix. Sometimes the fixes introduce new testing bugs. Identify the impact areas and explore them for risks before releasing the fix.

Learn From the Mistakes: See mistakes as an opportunity to learn. Think about ways to prevent similar mistakes from happening in the future following steps 

•  After the patch is released, conduct a root cause analysis and investigate how they got through the testing phase. 

• Brainstorm or huddle with your team about the possible ways to prevent similar issues in the future. 

• Identify the gaps in your process to avoid similar bugs in the future. Strengthen or change your process as per your context.

• Make some action plan and share gap analysis  with your manager if you need help in regards to updating test data in the QA environment, reviewing acceptance criteria in the early stage, increasing test coverage or adding more automation test cases etc. 

• Sometimes you will notice that some features/modules are difficult to test. Poor testability leads to more bugs. Improve the testability of such features/modules.

• Blocker or Critical bugs should be test coverages for manual and automation if possible. 

• Document and share your learnings with your entire team to prevent similar issues in the future. 

UAT testing: Get the application tested in a real-world or staging environment by early adopters or customers.

Unit testing: Make Unit testing a part of the definition of done. Furthermore, start measuring and publishing unit test coverage in your test reports.

Ensure Sanity test: Create Sanity test to run on a new build or code changes by developer or tester to ensure the build is not breaking other parts of the system at a high level. 

Conduct bug bashes: Bug bashing fosters a shared sense of quality by involving the entire organization in testing and finding problems before the release.

When testing complex applications, you cannot find every possible bug in limited time, resources and budget. Implement risk-based testing and prioritize testing those areas of the application that are most critical to the business. Realize that QA cannot assure 100 % quality but only assess the quality of a product. Anticipate issues and have a plan to rollback without impacting the customers. Furthermore, build a culture where everyone contributes to testing. 

=============================================================