AI design needs some 'deep thought'

We all know the answer. 42.

But to quote Deep Thought, "the problem, to be quite honest with you is that you've never actually known what the question was."

When Douglas Adams wrote about Deep Thought in the Hitchhiker's Guide to the Galaxy, he wrote about a machine that humanoids built to do something they couldn't do - work out the meaning to 'life, the Universe and everything'. Unfortunately for all (but possibly fortunately for Arthur Dent), it failed.

I was thinking about this today, whilst reading about Google's recent AI efforts with Google Duplex.

For those of you who may not have seen it, Google Duplex was showcased in their recent I/O keynote. They showed how Duplex can make phone calls on your behalf, mimicking a human so closely that the person on the other end of the call doesn't realise they are speaking with a program.

Technical or unethical?

Technically that's a huge achievement, and something we all knew was coming. Hell, we've been begging for it for years, smart assistants that really understand us? Who doesn't want one?

As a UX/CX consultant, it's easy to see this as the first wave of the disappearance of my job. One day there won't be much of an interface to design - you'll just ask, and get. There won't be human researchers, programs will crunch the data and ask what they don't know, and act accordingly. People like me will mumble about 'the way things used to be', curl up in crusty armchairs and knock back another Margarita. 

Only we still have a few problems to solve, don't we.

The ethical bridge

I received an email recently from someone who said they wanted to talk about UX, relatively urgently. They asked me when I was free for a chat, and since I'm a nice chap (and always interested in growing business, it must be said) I called them back later that day to see where I could help.

It turns out, after several minutes of small talk and intros, that the person was, in fact, a recruiter. And the 'UX' they wanted to talk about was, in fact, the provision of his (non)-unique recruitment skills to any recruitment shortage I might have.

Now whilst this person didn't explicitly lie and tell me that he was looking for business, his email was carefully worded to leave me thinking that was the case - and hence to act as if it were. Once the deception was clear I exited the call as politely and quickly as possible - and vowed to lose his details - we do sometimes talk with recruiters but being deceived as to the nature of the caller as the opening gambit pretty much tops the list of 'ways to lose trust instantly'.

Unfortunately, Duplex was asked to do the same thing. 

By calling people and pretending to be human, it's deceiving them to act accordingly. Any choices they might have made or responses they might have considered had they been in possession of the full facts was denied to them, even if they never became aware of that fact.

In Duplex's case the tool was designed to include noise-terms and pauses and sounds ('ur', 'umm') so that the person it was talking to felt comfortable, and thought they were dealing with a real person. If you want to see the call talking place, watch it here.

As I've long discussed there is an ethical dilemma to much of AI that needs to be carefully built in. Any AI platform must act ethically - but if we, as designers, don't build in the right ethics it will never understand what is right and what is wrong.

First, do no harm

Okay, so those words don't appear within the Hippocratic Oath, but they are close enough. Do no harm. Help the person, don't hurt them.

So, could an AI phone assistant cause harm? 

Sure it could - perhaps only emotional, but the risk is there. Let's say the person who it calls is having a horrendous day. Their partner has been diagnosed with a terminal illness, they are struggling to cope but gamely trying to keep working. They take a call - and struggle.

Most humans would understand and ask the person if they were okay. Would an AI?

What if it just kept trying to book that appointment, seemingly not to care in the least as the person explains they're having the day from hell. How much might that hurt?

But if the person knew it was an AI, they wouldn't expect buckets of empathy and an offer to come round and help, they'd probably just hang up. No harm done.

Designing for AI - two core elements

When you're designing for an AI interface in any form (text, voice or interface-based) there are four key elements you need to consider:

  1. Context-setting
  2. Engagement points & options
  3. Learning paths
  4. Ethical risk analysis

1. Context-setting

AI needs to set context. It needs to let the person know that this is a machine, not a human. Full disclosure isn't a pathway to failure; it's the first step to building trust, understanding, and sometimes even engagement - our user research shows that many people absolutely love to play with and experience smart tech, so you're losing an opportunity to engage if you don't tell them the truth.

You're also potentially setting the AI up to fail - if people think they are talking to a human then they may deal with it in a way the AI hasn't been trained to handle, and kill the engagement right off the bat.

So, firstly, set the context.

2. Engagement points & options

Next comes engagement.

If a lifetime of CX research has taught me one thing, it is this: humans surprise you.

They do the unexpected. They think illogical thoughts. They act in ways that don't always make sense.

And they don't always understand the rules.

So with any smart system or AI interface, it's paramount that the engagement points are as wide as possible. You only need to go to YouTube and search for fails relating to any device (I won't call out Google or Siri particularly) and you'll see what happens when human and AI interpretation goes wrong.

Humans interrupt each other, leave long pauses, and take huge cues from the minutest of details. One tiny example of this happens every day in my family. My own upbringing (English) means I speak with pauses at the end of sentences, so I'll often pause a little between sentences when getting something out. My wife comes from a South American family where people speak fast and with few breaks - so to her, a slight pause is a signal that I've stopped speaking. The outcome is one frustrated husband who only gets half his words out before the conversation moves on - and there's no AI in site, yet.

But it's not just voice systems where this is important. I'm currently working on an AI engine that reviews video for you, identifying the content it finds. But if it does that for you, how does the user engage and ask for something slightly different? How does the user tell the AI that it's misunderstood their need?

Identifying the engagement points and the options available at those points is a key path to success - and making those points as flexible as possible (to deal with tricky human illogicality) is paramount.

3. Learning paths

All AI requires learning to be successful.

Whilst much of that happens in the background and from ongoing interactions, learning from the user within the context of a single interaction has the potential to teach far more.

It's like the difference between user testing and analysing your Google Analytics reports. GA reports give you en-masse data about how people are reacting to your website. It tells you what people are doing, but often not why. User testing on the other hand lets you talk one-on-one with a single customer; whilst it doesn't provide the 50,000 foot view of what everyone is doing, it does let you see exactly what's going wrong, and have a chance to correct it. The depth of feedback and learning at that engagement point is staggering, in comparison to any analytical data-based approach.

And the effect it can have on the customer, to be heard and to have their gripe confirmed and even fixed in that small window just cannot be over-stated. I've seen customers converted into fanboys in moments.

So building one or more learning paths into the tool is a gold-mine, both for longer term improvements within the AI itself and for engagement with and retention of the customer.

4. Ethical risk analysis

My final element to consider is back to my favour subject; ethics.

When AI gets it wrong it can be funny - YouTube is stuffed with examples where assistants have misunderstood and returned something hilarious. The web is full of autocorrect funnies.

But when a self-driving car piles into a pedestrian, it's not quite as funny.

So it's important to perform an ethical risk analysis, and understand how the risk can be mitigated.

The Duplex example I gave above is one. What happens if:

  1. The person receiving the call gets upset and needs someone to listen and/or care?
  2. The person receiving the call has a heart attack mid call and needs someone to call an ambulance?
  3. The AI misunderstands the call and books something that incurs a cancellation fee that the owner is not even aware of.
  4. And what happens to the store if agents repeatedly book appointments that nobody shows up to? Could that extra pressure push them out of business?

Or in the case of the video analysis tool, let's say it analyses a video of children in a park, a fast moving car chase and a dog getting run over - and mixes up the cut so it looks like a child was crushed to death. Could that cause trauma to a viewer? Could they be sued for it?

Ethical risks can be covered by terms and conditions and legalese - to a degree, only. But our responsibility as designers and innovators should never stop at the 'get out of jail (not-quite) free' card.

 

 

Overall Google is doing amazing work, and I love where we're going.

But next time, let's hope the conversation starts with a little more honesty.

Gary Bunker

the Fore