Generative AI has reached perhaps its highest peak of attention so far due to the latest accomplishments in generating images, text, and even holding conversations. Public demos of diffusion models such as Stable Diffusion or Midjourney, or chatbots like ChatGPT have already helped us as users to accomplish more in less time. However, there were no scientific studies to back up this hypothesis so far… until now.
A recent study conducted by scientists from Stanford University and MIT has shed new light on the impact of generative AI on the productivity of workers. The study was specifically focused on understanding the impact of using an AI-based conversational assistant (similar to ChatGPT) for customer support within a Fortune 500 company. The results were staggering: the use of the AI assistant by 5179 customer support agents resulted in a 14% increase in worker productivity.
The only technical detail we know about this AI tool is that it was based on OpenAI’s LLM GPT. Its role was to constantly monitor customer chats, providing agents with real-time suggestions for how to respond to the clients. The outcomes of the AI models were suggestions to aid them and not direct responses based on these messages, but the human agent had always mastery on what to say to the client in the end.
When comparing results before and after the adoption of the AI, the authors observed a remarkable 14% increment in productivity. This was explained by the improvement in multiple factors, including a reduction in the time it took agents to handle an individual conversation, an increase in the number of conversations they could handle per hour, and a small increase in the share of conversations that were successfully resolved.
Surprisingly (or not), these gains were even more pronounced in less-experienced and lower-skill workers, indicating that this technology can help less productive people to catch up with their more experienced peers. The authors believe this could be due to the AI assistant being able to capture and disseminate the patterns of behavior that characterize the most productive agents, aiding newcomers in moving more quickly up the learning curve.
Another interesting outcome of the study is that introducing the AI assistant improved the way customers treated agents, as measured by the sentiment analysis of their messages. This change was associated with other organizational modifications, such as a decrease in turnover, particularly for newer workers, and a decrease in the likelihood of customers escalating a call by asking to speak to an agent’s supervisor.
While these results may not be applicable to every domain that is adopting generative AI, they provide the first scientific evidence of the positive impact of this technology outside a lab in an application field that is known to use a high level of automation. We’re also eagerly waiting to see similar studies around other more human-oriented points such as workers’ satisfaction, which should be also an important variable to consider in this path towards massive AI adoption.
At Arionkoder AI Labs we’re creating new tools to facilitate using LLMs and conversational agents to digest massive amounts of information regardless of the application domain. Do you want to know more about this project? Do you want to try it out on your business? Reach out to us and let’s start thinking about it together!