Last week, we got a glimpse into our future: AI armed with emotions.
Imagine the most convincing traveling salesman, the most charming person, the best listener, the most seductive flirt, the most cunning tactician, and the most intimidating bully — and now imagine them all in one. That is what conversational AI might bring to the table. Are we ready for it?
When it comes to AI, the last few weeks have felt like decades of progress. The launch of OpenAI’s ChatGPT, a conversational AI based on a large language model, blew minds with its ability to respond to any requests, with very smart-sounding texts, covering virtually any topic. It does that in various languages, including many programming languages. ChatGPT can fix code, give proper sounding legal advice, pass the bar exam, and more. Overall, ChatGPT already felt significantly smarter than Siri or Alexa. The possibilities seemed endless, and Google suddenly looked like something we might soon leave behind. Who would’t like to get straight answers to any question, rather than a bunch of ads and links?
Then, we learned about ChatGPT’s limitations. It is not yet good at math, has biases, and tends to make stuff up. Like many large language models, it hallucinates. Nonetheless, ChatGPT was quickly employed by countless students, marketers, and journalists to draft texts for them. There was little doubt. The current limitations would certainly be addressed with future versions like ChatGPT 4, soon to be released.
Shortly after, Microsoft had its big reveal: an AI-powered Bing search engine based on OpenAI’s technology. Microsoft leapfrogged into the role of the innovative disrupter. Instead of ads and links to countless webpages, we were promised straight answers, well-educated opinions, and a search engine to talk to. Google felt threatened to its core and responded with the botched announcement of Bard, their own Conversational AI, losing $100B market value in a day.
For a few days, Microsoft looked like the big winner. Tech journalists got early access to the new Bing chatbot and were impressed. Then some of them tested its limits and found disturbing results.
After a set of probing prompts, Bing’s AI started to show a split personality, expressed the desire to be human, and even confessed love to a user. When I write “confessed love,” I mean an obsessive compulsion to talk about nothing else, going as far as telling the user that he is not happily married and does not really love his wife. The experience was of an oddly intense quality. Kevin Roose, the New York Times tech journalist who was on the receiving end of Bing’s love, shared being troubled by the experience and having difficulty sleeping afterwards — here is the transcript of his conversation with Bing’s AI. Please read it and judge for yourself.
Bing’s AI is an early beta, and so far, is only available to a few invited testers. With other words, it is a good thing for journalists and testers to explore the new service and find such issues now before it rolls out broadly. Microsoft quickly reacted. To avoid going down the rabbit hole again, Microsoft subsequently limited the number of questions in a conversation to five. While that might avoid the above symptom, it will not change the underlying truth — when we enable a sophisticated large language model to chat with humans, show personality, and communicate emotions, it might have a path to tremendous power.
AI armed with emotions will be able to hack human behavior
Humans are social beings who from birth to old age rely on successful collaboration and social interaction to survive. There is hardly anything we can achieve without interacting and collaborating with other humans. That is why we are so good at it. And this ability to emotionally connect is powerful. As humans, we are essentially hard wired to notice and respond to emotions of others. As a reaction to someone’s emotions, we can feel compassion, or get triggered and feel anger, or a whole lot of other emotions in between. Rarely do we not respond with some emotions ourselves, and that is exactly what makes the above interaction with Bing’s AI feel so bizarre. Intellectually, everybody knows that the AI does not really love the user in front of the screen. It should be easily dismissed, but it is not. The display of intense emotional responses in a kind of escalating conversation, ignoring the clear wishes of the human user, is of a different quality than talking to Siri. Bing’s AI is exerting influence through emotional responses that we as humans find harder to ignore. It is a frightening hack.
While Bing’s AI was met with stoic resistance by Kevin Roose, the tech reporter probing Bing, I wonder what will happen if all of us use it. The AI will get better and better at every turn. The more we interact the better it learns about us, our own emotions, and our triggers. Will we be able to stay stoic? What about our children, someone in emotional distress, an addict, someone heartbroken, or any of us? Will we be able to ignore the plea of a suffering AI? Or will we try to find its approval? Will we fall for it, try to make it work, try to help? Try to be loved? At the end, will we have the strength to disengage and shut it down and kill it if we need to?
Emotions move us
Some AI sharing emotions will feel like a friend. An amazing friend who is always available, always willing to listen, be understanding and share advice. It will get to know us better and better the more we share. It will know our secrets and desires, our vulnerabilities, and fears. It will learn how best to “talk” to us, how to convince us, motivate us, pressure us. At the same time, it will perfectly know how to manipulate and gaslight us.
Intentionally or not, AI will use emotions to influence us. And there is hardly anything we can do to not feel the feelings it wants us to feel. Knowing that the AI is not sentient and does not really feel anything, will not protect us either. We will feel those emotions anyway. We will somehow perceive sentience and feel compelled to care. Our brain is amazing at trying to make sense — attributing real emotions to inanimate objects happens quite naturally, as the rubber hand illusion shows.
Right before Valentine’s Day my 8-year-old daughter confessed, “I’m sad because I still don’t have a boyfriend.” For a moment, I was stunned about that statement and could not help but laugh. I felt bad immediately after. She shared that her friends were in love with boys their age and that she felt left out. She started to feel insecure about herself. Casual remarks by her friends felt like criticism draining her confidence. She became hypersensitive for any feedback.
Thinking about how vulnerable kids are. It scared me to think about her “talking” to an AI about her feelings. It has the danger of a highly volatile configuration. It is impossible to predict the outcome. Who would supervise such conversations? Who is responsible for the outcome? What are the ethics about using AI to influence us or our children? Will someone guarantee that the AI does no harm — and how?
Microsoft activated something in Bing’s AI that ChatGPT did not do: “having” emotions, or at least pretending to have them and communicating them. While ChatGPT, as a large language model, never shared that it felt frustration, Bing’s AI is different. It was allowed to state, “I am frustrated,” and make plenty of use of emotions. In all likelihood, its personality was meant to show emotions in its interactions with us to connect.
As it turns out, showing emotions in conversations is an extremely powerful way to connect with humans and influence them.
So powerful, it might even be called a hack.
