Automation to improve application quality for a global safety certification leader

Challenge

The client’s development processes resulted in substandard application quality. The situation needed immediate remediation. The development process lacked transparency in the daily tasks, onboarding fresh resources took an inordinately long time, several processes were manual, testing was negligible, the tools used for testing were minimal, and the leadership was not 100 percent focused on driving improvement. The client was keen to take corrective measures and improve application quality by increasing test coverage and by creating cross-skilled teams that were productive and could meet flexible demands.

Solution

After a study of the client’s development processes, Marlabs decided to use automation for testing, introduce dashboards for updates, create a digital knowledge management repository, and conduct frequent meetings between key players to share information on progress.

The first major challenge was the validation phase during testing. This process—which is extremely stringent—was unique to the client. The accuracy and detail required in the documentation at this stage leave zero room for errors. Any missing detail or gaps in documentation highlighted by external auditors would mean jeopardizing the entire release cycle and erasing months of work.

Having decided on automation being central to the solution, Marlabs designed a release strategy and set up a verification and validations testing service to address the client’s products and platforms. Marlabs then conducted the following activities:

• ‍In-sprint, functional, API, and regression tests: These were prioritized and automated while interventions were created to maintain and enhance the UI automation and API automation frameworks.‍
• Specialized validation (21 CFR Part 11 & US FDA Regulatory): Services were set up to ensure the safety and quality of product behavior and to meet compliance requirements.‍
• Platform testing: Delivered test strategy and end-to-end testing of the client’s compliance training system and its compliance management system for regulated industries.‍
• Technical testing: An analysis and assessment of the client’s existing automated test framework was conducted with continued enhancements to increase test scripting productivity.‍
• Exploratory testing: Coupled with risk-based and other tests, exploratory testing was done to unlock higher efficiencies.‍
• Communication: Effective governance, communication, and collaboration between the client’s teams and the leadership was established, and a realistic goals-driven 30-60-90 action plan improved application quality.‍
• Metrics and measurement: Weekly and monthly release-wise progress reports and metrics management were created.‍
• Knowledge management repository:  A dedicated "Client University" was created to include important pieces of training, details of systems used, testing conducted, and more.

Being hyper-aware of the precise documentation required during the test phase, Marlabs employed a tool that generates and reports stage-wise data in a descriptive, non-editable (and therefore tamper-proof) format. These reports capture test evidence to present to the client’s customers and auditors for the necessary production approvals.

Results

Marlabs made a difference by introducing comprehensive testing, employing the latest automation tools, and using technology better. Creating the knowledge management repository also ensured ongoing awareness across the client’s development team and has become an invaluable reference resource for new team members.

With the implementation of automation, the client experienced no defect leakages into production over two major releases that included 100+ change controls and 320+ training courses. The robust automation framework ensured the client could focus on extending automation to additional product lines. At the same time, improvements in governance resulted in consistent results.

Impact

• 100% automation regression coverage
• 30% faster onboarding and cross-training of resources
• 70% faster turnaround on outages by collaborating with internal dev teams
• 40% reduction in support calls over 12 months, indicating improvement in app quality