“”
Posts

Do Humans Interact Differently with Gen AI Bots? Yes, They Do. And We Have the Data to Prove it.

TwitterFacebook

Guest Authored by Clement Ruin, Senior Data Scientist

Some customer service leaders are wary of putting a generative AI bot in front of customers. And look, we get it! There have been a few well-publicized instances of generative AI bots acting, well, a little off occasionally. Even with powerful guardrails put in place to ensure the safe and secure use of the technology designed and maximize customer satisfaction (such as Simplr’s Cognitive Paths), are consumers now predisposed towards being guarded and reserved when interacting with these new types of chatbots?

Well, we’ve had our own generative AI bot “in the wild” for a few months. And we’re very excited to reveal that the data indicates that consumers may be more comfortable and engaged with gen AI bots compared to previous generations of customer service chatbots.

To examine how consumers interact differently with gen AI bots and other chatbots, we looked at two metrics using data accrued from customer interactions with our gen AI bot compared to our older, intent-based bot. 

(As a bit of background: Intent-based chatbots are built on mapping out responses to specific user intents and flows such as “where is my order?” or “I’d like to process a refund”. They cannot autonomously generate responses, thus limiting the types of inquiries they can respond to and how engaging they can be with an end-consumer. Almost all of the bots that consumers interact with today are intent-based.)

The two metrics we looked at were:

  • Numbers of questions asked by the customer to the bot (indicating a willingness to converse with the bot through a lengthier, more involved interaction)
  • Frequency of “small talk” interactions (indicating a desire to engage with the bot outside the direct scope of the inquiry)

Here’s what we found:

  • The number of questions asked by customers increased by 39% when interacting with Simplr’s gen AI chatbot.
  • The number of customers conducting “small talk” with the bot increased by a whopping 250%.

Here are the main reasons why we believe we’re seeing this increase: 

  • As previously discussed, gen AI bots can handle significantly more inquiries and are more sensitive to nuance, ultimately resulting in more successful customer experiences. Repeat users who have had a prior positive interaction with the bot may be more willing to further converse with it in follow-up conversations. 
  • The power of generative AI has allowed us to rethink the overall user experience within the chat. Since gen AI bots are better at understanding the customer and can more efficiently retrieve the right information to solve the customer’s issue, we no longer need to rely as much on “point and click” and menu item approaches to understand what the customer is looking to accomplish. They can just chat away, thus increasing the number of questions they ask the bot.
  • Finally, some customers assume that they are chatting with a human in the first place! While we always advocate the best practice of clearly letting the customer know when they’re interacting with a bot, some customers might still miss that notification and indulge in friendly banter with the bot about the local weather.

Investing in new technologies, especially customer-facing ones, can be stress-inducing. When you add on the fact that generative AI-backed chatbots are so much more powerful than intent-based bots, customer service leaders have the right to need assurances that customers will derive satisfaction and enjoyment out of their interactions with the bot. So far, the data is certainly trending in the right direction. We’re looking forward to seeing this trend continue as customers continue to have more great interactions with the new generation of chatbots.