Using AI to Replace and Augment Call Center Workers

As companies adjust to compressed margins from higher labor costs, labor shortages and increasingly complex relationships with labor they are turning to artificial intelligence and automation.

Call center workers will slowly be replaced by artificial intelligence. In the near term, AI will drive call center wages lower and reduce the number of employees needed. Long term, call centers will be reserved for luxury brands and high value customers.

Recent studies have shown that employees are more productive while working alongside AI.

They have also shown that AI in call centers bring sub-par workers up to an average or above average level of customer service.

Call centers can begin taking advantage of these technologies today.

Over the coming years we will see AI model fusion. The seamless handoff between large language models like ChatGPT, advanced text to speech, and speech to text algorithms working together to carry on conversations.

Each one of these technologies is revolutionary on their own, and will be even more exciting when working together.

Let’s look at how AI and ChatGPT can be used in call centers today and the future

Using ChatGPT to improve customer service

Research from the Massachusetts Insitute of Technology (MIT) recently showed that implementing custom trained language models like ChatGPT helped low performing contact center agents to perform at an average level.

This is the first step of deploying AI into call centers.

The technology is not currently in a place where you can interact directly with AI. Humans are still needed to translate between what they are being told by a customer and interpret the internal policy on the subject related to the call.

Employees can use ChatGPT as a quick reference guide while on the phone, and provide best in class answers.

AI models are trained based on conversations of top performing employees which gives lower performing employees a guide of how to interact with customers.

Additional benefits of AI for call centers

Call center managers are tasked with hitting a series of KPIs. AI has the ability to improve most of them.

  • Lower Average Handle Time (AHT) – By putting information at the fingertips of agents, they can resolve customer inquiries faster. By deploying real time speech to text and ChatGPT models, AI could be setup to pre-emptively answer questions without agents even having to type.
  • Higher First Call Resolution (FCR) – AI can be trained on thousands of cell center interactions and corrected when it fails which improves the model over time. The improved model can better handle fringe cases that the average call center agent may see only once ever several months.
  • Improved Service Levels – Getting to the right answer faster means that more calls can be answered by the same number of agents. On the other hand, some companies may decide to reduce their workforce to save money and maintain the same level of service.

Beyond hard call center KPI’s, advancements in AI will improve overall customer satisfaction. Beyond lower wait times, and better first call resolution there are a number of intangible benefits that companies can begin to deploy today.

  • Consolidation of Customer Communication – Companies are already deploying advanced AI chatbots to help customer service. Historically chat bots were advanced decision trees that would mostly route people to contact a call center. By consolidating the information source for internal and external communications many points of contact can be diverted.
  • Call Diversion – Some call centers will focus on call volumes. However, increasing call volumes in a customer service environment usually indicate a process failure somewhere else along the customer lifecycle value chain. The best call is one that does not happen, and tools like ChatGPT should be able to answer a wide range of common questions that customers have.

Some of this technology is already here today. Microsoft Dynamics 365 can already be trained using internal data to suggest articles and pieces of information that may be useful in a customer facing environment or a call center.

Enable AI suggestions for cases, knowledge articles | Microsoft Learn

Training AI on Call Centers Transcripts

With technology like Chat GPT, Speech to Text, Machine Learning (ML) Classification Models and even AI powered Word Clouds, the humble call center is going to become an essential part of company operations in the future.

The data call centers generate is massively valuable!

Most calls are audio recorded these days for quality assurance purposes and in case they need to be audited in the future. They can also be a treasure trove of information regarding what your customer needs are, or equally importantly how to successfully market and sell products to people calling into learn more or purchase your products.

Could Call Centers be Your Biggest Data Asset?

While not as direct to customer as an internal sales team, you could gain a lot of valuable training data from transcribing calls from a call center. The voice to text transcription process would be a ton of work to do manually.

OpenAI the creators of Chat GPT recently introduced another amazing algorithm called Whisper that converts Speech to Text with amazing accuracy. Check out the following link for some examples of it transcribing fast talkers, different languages, and people with thick accents.

Introducing Whisper (

Automating Call Center Quality Assurance

Companies are increasingly turning to AI for quality assurance.

Robots can record all incoming and outgoing traffic and run the interactions through advanced AI models that test for keywords and call sentiment. They typically use the connotation of specific words to assign a positive or negative value to it and net the scores together.

More advanced AI like ChatGPT will be able to decipher the entirety of a text conversation and score it, giving managers a valuable quality control tool.

These solutions can be deployed in real-time and also augment the work of Quality Assurance Analysts who could plug into listening to a call that an AI determines may be going poorly.

While it could be an awkward transition, it could also mean that a customer achieves first time resolution by being transferred to a more seasoned call center professional.

The Current State of AI Voices and Avatars

The Wall Street Journal recently covered the state of AI text to speech and the use of digital avatars.

AI was trained on voice samples from one of their reporters, and AI does an astonishing job of re-creating the sound of their voice. There are some limitations in the inflections not being as natural sounding as they would while interacting with a human, but the results are infinitely better than Apple’s Fred Voice.

Avatars are and voices will be designed as brand extensions

Every interaction point that can be controlled by a major brand is thought through. The voice and look of a digital assistant that is working with you to resolve inquiries is no different.

Automakers even hire specialists to fine tune the digital sounds that electric cars make

Digital avatar and digital voice artists will be a job of the future. Companies will have unprecedented amounts of data to work with when deploying AI replacements for call center agents. We imagine solutions deploying dozens of voice variations to A-B test the voices that obtain the best customer satisfaction or being customized more to fit the brand.


The pieces are in place and in-development to create a seamless interaction of customers and AI. Text to speech, speech to text, and large language models like ChatGPT have all of the pieces to make it a reality.

Advancements in computer speed and technology will need to happen for the dream to become a reality.

ChatGPT-4 still takes several seconds for a response, as do speech and text conversion technologies.

Solutions are currently best fitted to act as an internal assist that employees can reference until the AI models are capable of responding and adjusting at the speed that a human can.

Scroll to Top