From Chatbots To Chaos

https://swiy.co/go-from-chatbots-to-chaos

Are your AI guardrails, policies, and guidelines up to date or out of date?

In 2022, Jake Moffatt needed to fly to Toronto for the funeral of a family member. Before booking a flight with Air Canada, he checked their rules to qualify for a bereavement fare. The helpful Air Canada chatbot assured him he could buy a ticket, fly, then request a partial refund. But when he returned home and applied for the refund, Air Canada refused to honour the request, referring to their website policy that clearly stated he had to make the request before travel.

Fortunately, Moffatt had taken a screenshot of the chatbot conversation, which he showed to Air Canada, but the airline still refused his claim, telling Moffatt he should have referred to the website, and then bizarrely claiming the chatbot was “responsible for its own actions”. Not surprisingly, the British Columbia Civil Resolution Tribunal rejected that argument, and ruled that Air Canada had to honour the bereavement refund and pay costs. But the reputational damage to the company- which could have been avoided by simply honouring the claim – was much bigger than those costs.

From a customer experience viewpoint, Air Canada should have just given him the discount anyway, right? It was only about $150, and that would have been much better for the company than the reputational damage when the story made international news!

But from an AI viewpoint, the problem is the chatbot was acting independently. Like other chatbots, it learned from the training data it was given - and advised the customer based on that data - but gave that advice without a human reviewing or overseeing it.

This is more relevant now than ever. People are building AI systems that act autonomously. This is known as “agentic AI”.

You might hear people talking about “AI agents”. It’s a buzzword right now, and is the next step in AI systems.

This is fundamentally different from earlier generative AI, where you could prompt ChatGPT or Copilot, review its response, check it, and then act. Now, AI agents no longer have a human in the loop.

You don’t need much technical expertise to build an AI agent, either. In fact, it’s available right now in ChatGPT and other common AI tools.

That means you can create it yourself.

Or a team member can.

Or you can choose from a list of AI agents others have created.

Make no mistake - AI agents create new opportunities for improving productivity and enhancing your customer experience.

But if you don’t do it the right way, they also create new risks and potential problems for you, your business, and your organisation.

Imagine creating an AI agent that’s “empowered” to give a customer advice - as with Air Canada.

Or asking an AI agent to book a business trip – giving it your credit card details so it can book flights and accommodation.

Or building an AI agent that reviews employee records and automatically send them email or WhatsApp messages about their performance.

It might be very efficient, and it might let people do their work faster.

But it also means you can make mistakes faster!

Is YOUR governance up to date? If not, you’re at risk. I’m running an online presentation soon about what’s new in AI - and what it means for you as a leader to put the right governance in place. You’re welcome to register, and please share this with your colleagues as well.

https://swiy.co/go-from-chatbots-to-chaos

Hosted on Acast. See acast.com/privacy for more information.