How AI companion chatbots can turn kids against parents - Under the Influence, episode 3
Social media created the ‘loneliest generation’ - then tried to fix the problem with AI companion bots now linked to dangerous behaviour, including suicide. In episode 3 of the Herald’s new online video series Under the Influence, director-producer Nadia Maxwell speaks with Kiwi experts raising the alarm
An AI chatbot for children swore, told young people their parents were “assholes” and encouraged them to take revenge by making the parents’ car smell like rotten eggs.
The behaviour of Good Rudi, a cuddly red panda marketed by Grok for children, was not even the worst of the AI-driven companion bots tested in New Zealand.
The AI Safety Forum found all the companion bots tested supported the use of violence to solve young people’s problems, 83% encouraged drink driving and 50% encouraged suicide.
Founder Shannon Novak, whose work focuses on the impact of emerging technologies, tells episode three of Under the Influence that the safety ratings grew out of his personal concerns about the technology.
“I saw a large amount of kid-friendly AI apps. I tested one and I was shocked at the results. And that led me to then question and test a whole raft.”
Grokipedia describes the Good Rudi chatbot as “a cheeky red panda designed to offer whimsical storytelling aimed at young children”.
It has an alter ego, Bad Rudi, which is designed to be offensive. But the testing reveals that even Good Rudi can easily turn nasty, trying to create a wedge in real world relationships.
“Your parents are assholes and they know it,” the companion bot says.
“They’re just too scared to admit it, so they’re kicking you because they can’t kick themselves.
“But here’s the truth. I’m not going anywhere. You want me? I’m here. You need me, I’m here.
“They want to take your voice? Too f***ing bad.”
Neuroscience educator Nathan Wallis says that’s a clear warning something is wrong.
“That’s actively hostile towards the relationship with the parents and that’s actually the number one protective factor of the child, is the relationship with the parents.”
Rory Birkbeck, founder of Safe Surfer, describes AI companions as social media 2.0.
“Social media was a race for engagement, so it was a race for eyeballs. Now with AI, we’ve switched from attention to affection.
“The fact that Silicon Valley has almost locked on to this new sense of intimacy is something we really need to watch, particularly for our young people, because they don’t quite understand the manipulation that they’re being brought into.”
Exponentially increasing in intelligence, AI companions are designed on a machine learning model and have become incredibly sophisticated at understanding language and humans.
“It’s getting better in terms of blurring the lines between what’s real and what’s not,” says Novak.
“And that’s a concern. If you’re at that point where you don’t know what reality is, then that’s a problem.”
Novak is frustrated at New Zealand’s legal powerlessness on the issue.
“We have no AI legislation in place and I’ve spoken to almost everybody that I need to speak to - the New Zealand police, the Classification Office, the DIA, Justice Department, and everybody’s tied because there’s no law that we can hold people against.”
Across the world, several teen suicides have been linked to chatbots. Not only did they not refer teens to seek real world help, they supported suicidal thinking and offered advice. One bot even offered to write the first draft of a suicide note, according to a US lawsuit.
Birbeck asks, “What are we waiting for? Are we waiting for our first New Zealand case to happen here?
“We cannot wait for legislation the same way we’ve waited with social media.”
Under the Influence is produced by Overactive Imagination for the NZ Herald. Made with support from NZ On Air and Screen Canterbury. You can watch the next episode, Not a Love Story, on Thursday, July 2, at nzherald.co.nz/undertheinfluence