BOSTON--(BUSINESS WIRE)--A study by Applause, a world leader in testing and digital quality, reveals that nearly one-third (30%) of users of chatbots, interactive voice response (IVR) services and conversational assistants are dissatisfied with their experiences. In addition, the study has revealed concerns about inherent bias that can negatively influence engagements and outcomes with AI-driven voice applications.
The results are drawn from Applause’s second annual AI and Voice Applications Survey to gain insights into AI user experiences. Conducted in February and March of 2023, the survey collected responses from nearly 5,200 digital quality testing professionals from around the globe.
ChatGPT is proving to be popular among users, with 74% responding positively about their experience with the recently released generative AI application. When asked if ChatGPT had understood their questions and provided them with helpful answers, more than two-thirds (67%) responded with “yes”. When asked whether ChatGPT or similar technology would help them to complete their work-related tasks, the vast majority responded that it would be “very” (43%) or “somewhat” (37%) helpful.
Conversely, when asked about interactions with chatbots in general, up to 30% said they were dissatisfied with the experience - referring to wrong answers (29%) and a lack of understanding (24%) as reasons for dissatisfaction. Respondents were also frustrated by the failure of chatbots to appreciate language variations or the phrasing of questions. However, one-third indicated that they would use chatbots more if they responded more accurately to questions and voice commands.
Despite these complaints, 92% of respondents expect companies to have chatbots on their apps or websites, eliminating the need for a call. A similar number (86%) said they expect to be greeted by an IVR system when they call a business. Nearly half of respondents (44%) expect mobile apps to have voice assistants.
The study also looked at sentiment around the inherent biases that can affect interactions with AI powered chatbots and smart voice assistants. Bias occurs when the underlying algorithm has been trained with poor and limited data. This impacts its decision making and leads to negative or unanticipated experiences. When questioned about bias in generative AI technology like ChatGPT, an overwhelming 86% of survey respondents expressed their concern.
“As AI technology becomes ubiquitous and we rely on it more heavily for mission-critical business functions, it is essential that we focus on quality – both from a data collection standpoint, and from a training and testing perspective. For a second year in-a-row, a vast number of survey respondents shared their frustration with current chatbot interactions. Organizations must focus on bringing real world, unbiased insights into their development processes to drive improvements to their AI experiences and meet their customers’ rising expectations,” said Luke Damian, Chief Growth Officer for Applause.
The AI and Voice Applications Survey is the first of a series of digital quality surveys Applause will conduct this year. In May 2023, the company plans to release its second annual State of Digital Quality Report which analyzes a representative sample of its testing data and reports on the most common flaws in digital experiences in several industries including retail, finance, media and telecommunications, and travel and hospitality.
Applause is a world leader in testing and digital quality. Brands today win or lose customers through digital interactions, and Applause delivers authentic feedback on the quality of digital assets and experiences, provided by real users in real-world settings. Our disruptive approach harnesses the power of the Applause platform and leverages the uTest community of more than one million independent digital testers worldwide. Unlike traditional testing methods (including lab-based and offshoring), Applause responds with the speed, scale and flexibility that digital-focused brands require and expect. Applause provides insightful, actionable testing results that can directly inform go/no go release decisions, helping development teams build better and faster, and release with confidence. Digital-first brands rely on Applause as a best practice to deliver the digital experiences their customers love.
