twoday AI Agent is a ready-to-use solution ideal for companies aiming to rapidly implement generative AI in their operations and swiftly create business value. Common applications include enhancing customer service, automating internal processes, and streamlining data management and utilization.
In a recent project, experts from a large organization tested both a freely available AI application and twoday AI Agent, using a diverse set of realistic and complex customer support queries. twoday AI Agent, equipped with a built-in Retrieval Augmented Generation (RAG) framework, was evaluated for its ability to deliver AI-generated responses directly to customers.
twoday AI Agent significantly outperformed the compared AI application, specifically the free version of ChatGPT, in the tests. This blog will explore the extent of twoday AI Agent's superior performance and the factors contributing to its success.
Use of twoday AI Agent in a large customer service center
The effectiveness of twoday AI Agent was evaluated in a project at a customer service center with hundreds of employees managing over two million calls and half a million chat messages annually.
When managing large volumes of data, it's essential for the AI solution to be of high quality and deliver reliable answers, as even a small error rate can significantly affect the business, staff, and customers. Consequently, the project adhered to principles of responsible AI and was carefully monitored throughout its implementation.
Read more about twoday AI Agent here.
The significant impact of the RAG framework
The project merged twoday's technology and expertise with the insights of the customer service center. During development, the Retrieval Augmented Generation (RAG) framework was employed, allowing the language model not only to provide accurate answers but also to enhance responses through "prompt engineering."
A key advantage of twoday AI Agent is that all essential features are integrated into the product, making configuration and customization straightforward. As a result, the product is ready for immediate use, allowing methods like RAG to be applied directly without the extensive development work usually required.
In tests, the performance of the AI Agent was compared to the free version of ChatGPT, which does not utilize the RAG framework. All information required to address customer queries is available on the internet and thus accessible to ChatGPT's language models as well. The test results demonstrate that the AI application employing the RAG framework significantly outperforms the compared applications by providing more useful and higher-quality answers.
The project involved several tests in which customer service center employees posed realistic queries to various AI applications. Feedback from these tests was used for analysis and improvement. Performance was assessed based on accuracy and user-friendliness, with employees rating response time and quality on a scale of 1 to 4. A rating of 4 indicated that the generated answer was directly usable for responding to the customer's query.
Results
twoday AI Agent performed significantly better in the test compared to the other AI application. Users evaluated responses from both agents using the same guidelines and questions. Experts assessed that most of AI Agent's responses were suitable for direct presentation to end customers.
"twoday AI Agent's ability to quickly locate industry-specific information from its training data led to significantly better performance in the survey compared to the free version of ChatGPT."
Tony Shepherd, senior consultant in twodays Data & Analytics-team
When comparing the AI applications based on the number of responses rated between 3 and 4 on the 1-4 quality scale, Twoday AI Agent achieved a score of 83%, while ChatGPT, which does not use the RAG framework, reached only 36%.
During the test, planned improvements for Twoday AI Agent were also evaluated, including the integration of additional language models for information retrieval and more efficient utilization of user feedback. It was assessed that with these enhancements, Twoday AI Agent could potentially achieve a score of 95% on the quality scale.
The customer service staff evaluated responses from the twoday AI Agent and the free version of ChatGPT on a quality scale from 1 to 4, where 4 meant the response could be used directly to answer the customer's question.
AI adds a new dimension to customer service
twoday AI Agent not only provided high-quality support to the customer service center but also significantly outperformed the other AI application that did not utilize the RAG framework. The project ensured that the AI solution was optimized through continuous monitoring and improvement by users.
"twoday AI Agent not only provides customized and secure service to our corporate clients but also clearly illustrates how AI's potential can be further harnessed. By utilizing the RAG framework and integrating multiple language models, we have developed a solution that meets the highest standards of accuracy and reliability in customer service.
Project results clearly show that twoday AI Agent can offer excellent support and significantly outperforms other available solutions in handling complex customer queries."
Janne Sipilä, Director of Business Development i twodays Data & Analytics-team
Provide your customers with superior service through automated processes and AI
Create more personalized interactions, receive help with quick and efficient information processing, and provide your customers with superior service. With twoday AI Agent, you can easily get assistance in answering customer queries across all your channels, whether it's on social media, chat, or email.
-
Faster and simpler customer communication, available 24/7, year-round.
-
Accurate and precise answers to customer queries, with no delays.
-
Tailored to your company's brand, language, and tone.
Would you like to learn more about how the twoday AI Agent can assist you and your business? Get in touch with us, and we’ll be happy to help.