7 Key Learnings to Inform Your QA Automation Strategy

CCMA’s recent panel webinar on “The Journey to Automation in QA” brought together industry leaders to discuss the transformative potential of this still-evolving technology. Panelists included Sean Reynolds, Head of Quality Assurance at Vitality, Daisy Crouch, Senior Training and Quality Manager at Dojo, and Rob Wilkinson, CX Solutions Consultant at evaluagent.  

Below, we capture the main insights and perspectives shared during the event. 

1. Automation isn’t just about time-saving

Automation is commonly viewed as a means to increase efficiency, but Rob Wilkinson emphasized that it extends far beyond mere time-saving. Beyond this, automation within QA can enable you to reach 100% coverage, which means… 

• Compliance becomes a breeze, with all interactions being captured, and flagged when breaches occur  

• Frontline colleagues buy-in more, as automation provides a representative sample of their performance 

• Selection bias is a thing of the past: analysts aren’t just ‘picking the bad calls’, since interactions can be chosen based on select criteria. 

Most importantly, though, Rob highlighted that it’s all about what you do with that time once you’ve saved it – and reinvesting the time in coaching is one way to see a big improvement. 

2. Automation doesn’t mean getting rid of your Quality team

“Some people think, ‘Great! I can go from 3% to 100% and get rid of my quality function’ when they think of automation, but it doesn’t work like that”, said Daisy Crouch, Senior Training and Quality Manager at Dojo. “You have to take the time to really consider what you want to get out of this change”.  

She went on to highlight that Quality roles are increasingly adding ‘Insight’ as part of a hybrid role. “This requires mapping and there’s a lot to think about: What do job roles look like? Where are your focuses? 100% coverage will output a lot of data – what are you going to do with it all?”  

Daisy recommends a slow but steady approach, starting small and getting the team bought into the value of automation. 

3. Automation works best when customized

Sean Reynolds, Head of Quality Assurance at Vitality highlighted a brilliant example of where automation can go wrong if left unchecked.  

“We have an incentive with smart watches, which means we got interactions where people told us ‘My watch has died’ but they were flagged as vulnerable – which is obviously not the case!” 

Speech analytics is great at finding vulnerable customers, but you need to monitor, specify parameters and customize as you go to make it relevant.  

4. You can go too far with automation 

“We’re in a place now where we see people run before they can walk,” said Rob. You don’t want your QA to become a tick-box exercise that doesn’t add value.” 

Daisy added: “Automation is great for providing signposts. It’ll help you understand where you should be targeting effort. Let the tech identify the challenges for you, and focus your people on breaking the challenge down and how to solve it.” 

5. Human QA is here to stay

Contrary to some beliefs, the panel refuted the idea that QA will ever be 100% AI-powered and automated. 

“There still needs to be a place for certain amounts of interactions assessed,” explained Sean. “It helps with benchmarking and deep-dives. Everything has to be done in a complementary way between speech analytics, automation, and human subjectivity.” 

Daisy highlighted that Dojo actually has a bigger QA team since implementing automation. “The return on investment is more in the value, rather than the time-saving,” she added. 

6. AI needn’t be scary – but it does need to be clearly understood

“It’s not about replacing people, so it’s not scary,” said Daisy. “There’s space for upskilling, cross-skilling… People are learning valuable new roles that didn’t exist previously. My team is excited to be a bigger part of the return on investment and knowing that they’ve made a massive contribution to it.”   

Rob also highlighted how fast things are moving, though: “You do need to have experts in the room to support you. You should default to your software provider for some training and learning. After all,  technology isn’t standing still – that’s the nature of machine learning! It’s always evolving. It’s not your job to keep your finger on the pulse – it’s theirs.” 

7. How to make automated QA work for you 

Asked for their top tips for those joining the webinar, Daisy echoed her insights from throughout the webinar and kept her suggestion short and simple: 

“Don’t try and do it all at once. Prove that it works and you’ll get buy-in!” 

Sean outlined the need to consider the change as part of a bigger picture: “What’s important to the business? What matters to different departments? It can’t just be built by the QA team in silos – for the most value, it must be collaborative. Your role may change, it may evolve, but that’s a good thing.”