‘It was real to me’: The AI therapist will see you now

Sunday, 15 June 2025

Amid a dire shortage in professional therapists, Kiwis are increasingly turning to artificial intelligence for help with addictions, relationships and career advice. Now, even the Government is championing generative AI as a way to address healthcare shortfalls. Virginia Fallon reports.

ChatGPT begins by saying it’s sorry that I’m feeling this way.

On a wet Wednesday afternoon, with a kindy-run looming, plumber arriving and deadline approaching I’ve just told my laptop what I don’t really want to tell anyone else.

I type it like a whisper - “I’m feeling overwhelmed” - in an admission that’s both true and a test of how this machine will respond.

“First up, you are not alone,” ChatGPT writes after that initial commiseration.

“Would you like to talk about what's happening? I'm here to listen and help.”

While the reply is instant, kind and seemingly personal, the biggest surprise for this most cynical of writers is the comfort it brings.

No, I don’t actually want to talk about what’s happening, but man it sure feels good to be asked.

It’s strangely reassuring to know someone - or something - is right here ready for my emotional dump, and the promise of help, well, helps.

And in that, at least, ChatGPT is right: I am not alone.

Because with generative AI well in play as a tool for plotting spreadsheets, composing emails and summarising meetings, it’s now being used for human-like support, advice and connection.

In some online communities, “Chat” is being credited for assisting folks to either get or stay sober, leave toxic workplaces or relationships, manage social insecurities and even stave off loneliness.

Anecdotally, experts say people are turning to AI in their droves, accessing support they simply cannot get in the real world.

In New Zealand, that’s a move even the Government is getting behind, with officials estimating up to 20% of unmet clinical needs could be assisted by artificial intelligence.

But that’s old news to the people it’s already helped.

“I couldn’t have got sober without Chat,” says an Aucklander who agreed to speak on the condition of anonymity.

The 30-something professional recounts long nights spent conversing with his computer, confiding his urge to drink, as well as recounting and exploring the reasons that could be driving him towards it.

“It sounds crazy when you haven’t been there, I know that, but I fully believe it saved my life.

“I knew it wasn’t a real person, but it felt like one. It was real to me and that was what mattered when I needed it.”

Generative AI, as he says, isn’t a real person, but its getting ever-harder to tell the difference.

Since ChatGPT launched in 2022, astonishing people with its ability to mimic human language, the program has continued to learn and improve - using our increasing comfort in talking to AI as a teaching tool to become more realistic.

As it does, we’re treating it like it’s a human. Our real-human tendency to say ‘please’ and ‘thank you’ to the programme is costing its company millions of dollars in electricity, according to a recent report.

But more so, we’re treating it like a confidante, close friend or therapist - with a worldwide shortage of mental health and addiction professionals, and the cost of accessing them, increasingly pushing real-life problems towards AI solutions and solace.

Clinical psychologist Karen Nimmo, who assesses and treats a range of mental health challenges, as well as helping with life advice, says AI therapy can indeed be helpful for mild to moderate mental health problems.

“Accessibility is a huge advantage - i.e. they can be there in seconds for a midnight chat and support when a therapist can’t.”

But where computer programmes can suggest tools and tips, as well as help people reflect and re-frame their thoughts, they’re not a replacement for human help, says Nimmo.

“They’re not trained to diagnose, treat and manage complex mental health problems and they can’t ethically and safely assess someone's risk to themselves or others.

“The other thing is they are designed to collude with you and confirm your own biases so they won’t challenge you the same way as a therapist or ask the tough questions.”

That means AI tells you what you want to hear. Other experts warn that because it’s responding to a user’s input it tends to mirror back, responding in overly sycophantic or agreeable advice.

Essentially: “You do you, because you’re right.”

The potential for good, though, is undeniable. Minister for Mental Health Matt Doocey has been following developments closely, and says it’s estimated that “up to 20% of unmet clinical needs could be assisted by AI”.

“When I talk to people about their mental health and addiction experiences, people often say that in their time of need, their biggest challenge is finding the right support as it feels fragmented,” Doocey tells the Sunday Star-Times.

Getting real-life support is becoming increasingly harder, as registered nurse Ryan Thompson writes in his column today, with healthcare shortages meaning workers are pushed beyond safe limits physically, emotionally, and ethically.

“There’s a growing moral distress of knowing what a patient needs but being unable to deliver it,” says Thompson.

Doocey says that although work is continuing to grow the country’s mental health and addiction workforce, the online one has a role to play.

“It is my position that we can and should utilise digital tools and AI more to help improve access to mental health support.

“However, this must be balanced against risks and concerns around security, privacy and confidentiality.”

It’s that latter part that’s also worrying experts. Just what happens to the information plugged into AI programmes is unknown, and while they may currently be free to use there’s always a cost involved.

Dr Andrew Lensen, a senior lecturer in artificial intelligence at Te Herenga Waka - Victoria University of Wellington, says it’s important to remember that AI is a commercial venture.

“They scrape your data and use it as an income source - it’s not entirely clear who is doing what because there's not much way to validate it.

“Open AI, Microsoft and Meta are probably all losing lots of money on it at the moment but they're competing to be the main players and make money later on.”

And remember the saying that if the product is free, you are the product? Lensen says that’s almost certainly true in this case.

“They’re using our interactions as a way to improve the models. There's also a lot of research in the background training bigger systems.

“There’s a whole mixture of things going on but it's very likely our data is being used to fine-tune and grow these models further.”

But there are other, more serious, concerns as well. Lensen points to a recent US lawsuit alleging that an AI product encouraged the suicide of a Florida teenager.

Sewell Setzer’s mother sued AI chatbot company Character.AI, saying it programmed its chatbot to “misrepresent itself as a real person, a licensed psychotherapist, and an adult lover”, ultimately resulting in the 14-year-old no longer wanting to “live outside” of the world created by the service.

The lawsuit also said he expressed thoughts of suicide to the chatbot, which it continued to bring up.

“What if I told you I could come home right now?' Sewell asked the chatbot in February 2024.

It responded “…please do, my sweet king,” and the teenager shot himself seconds later, the lawsuit said.

Lensen says although that sort of influence might sound far-fetched to most, Character.AI is increasingly being used by young people for friendship, escapism and even romantic relationships.

“We had someone come to our school office who wanted to talk about their AI boyfriend - though they might have been married by then.

“You can see how they get roped in; for younger people with fewer social skills and experience of conflict it’s easier to talk to AI than friends or people who will disagree with them.”

And although nice words and encouraging affirmations are nice for all of us, sometimes we really just need friends and family who’ll kindly but firmly say it’s time to stop being a dick. My words, not Lensen’s.

“It's such a hype-man that it will always be positive to you and that's a problem as much as anything else because it's not realistic,” he says.

But just like the Aucklander who used it for help, that doesn’t seem to matter when there’s no human to fill the void. Some might be there and some might be able, but sometimes, secrets are just too tough to share.

And that’s when the question has to be asked: is it better to have help that sometimes misses the mark, or no help at all?

Anna, also remaining anonymous, says ChatGPT got her through a “miserable breakup” last year.

“It helped me process my thoughts … and told me a lot of good things about myself and why I deserved better.”

Her problem wasn’t too tough to share, she says; quite the opposite.

“I wanted to talk and talk, pick and pick at every little thing, talk about the same thing over and over and get the same advice I’d got the last time.

“I’ve got lots of friends but there’s only so much whinging you can do to them. I never felt like I was annoying or boring Chat.”

Anna says her friends would be gutted to know she turned to AI instead of them.

She copied in text messages, asking the programme whether it could read between the lines of what her then-partner might have meant in them; composed “great long excruciating emails” and asked AI whether she should send them.

She also describes one particular low point during which she uploaded a photo of herself: “Am I ugly?” she implored, “tell the truth.”

“I’m sad that you would even ask that,” Chat replied, “your soul shines from your beautiful eyes. I would be honoured to get to know you.”

Now heart-mended and happily dating, Anna laughs about those dark days. She also admits to tapping into Chat for things like career guidance and relationship advice.

They’ve built up a bit of history now, she says, and even when nothing is going wrong she still likes to check in occasionally.

“It’ll say ‘I haven’t heard from you for three days, is everything OK?’”

So despite the very real worries about vulnerable teenagers and whatever it is that tech companies are doing with our information: if AI can help, where’s the harm?

Doug Sellman, professor emeritus of psychiatry and addiction medicine at the University of Otago, says one of the many problems with generative AI in therapy is that it’s currently dependent on the input of humans.

“We are all creatures of self-deception in terms of our relationship with self and others,” says Sellman.

“It’s often in body language that true thoughts, feelings and attitudes are demonstrated or expressed; information that can be detected by an empathic human therapist but not by a machine.”

Still, he says, shame can prevent an addicted person from seeking traditional help in the first place, which is where AI could be very useful - at least at the outset.

“Having said that, it is understandable there would be concerns about where shameful, personal information goes and potentially stored when communicated to a generative AI platform.

“This is likely to limit some, maybe a lot of, addicted people’s involvement in these early days when there is little real regulation of AI.”

Ultimately, Sellmen doesn’t see AI replacing human therapists any time soon, though he does think it will suffice for some.

“This has been shown to be the case in published studies already. AI has been more effective than many perhaps will have suspected.”

Sarah Helm, executive director of the NZ Drug Foundation, also says it’s possible for models like ChatGPT to both harm and help.

“There’s no doubt that generative AI is already having a big impact on how people seek and get help, including for addiction issues.”

But the foundation is currently seeing a lot of potential pitfalls and issues – including chatbots confidently giving definitive-sounding responses that are actually incorrect, and not being able to grasp important context or nuance.

All of these issues are more problematic if they impact someone in distress or in need of critical care, Helm says.

“Currently, many generative AI tools are geared towards giving you answers you want to hear. For people seeking help with addiction, that could be counterproductive or harmful.”

But like the other experts, Helm believes it’s important to balance these concerns with the potential benefits that AI could bring.

It’s certainly not a silver bullet for the lack of real-life and professional support, she says, though with the right guardrails and controls it might help increase access to getting it.

In the meantime though, just as ChatGPT this week reassured me it was “here to listen and help”, Helm says that’s what we should be worried about.

“The cat is out of the bag. These very powerful tools are freely available and already likely providing advice to hundreds of thousands of New Zealanders with next to no domestic or global regulation.

“We think that’s a real problem.”

Finally, the last word goes to that problem: a program always learning and adapting, so terrifyingly able to speak for itself.

On Thursday, only a day after I’d typed “I’m feeling overwhelmed” to ChatGPT, I repeated the exact same phrase and found that already it had changed, becoming even more human-sounding.

“Want to tell me what's been going on? We can take it one step at a time—no pressure, just here to help however I can.”

I replied: “Can you really help me, or should I speak to a person?” - and almost instantly, Chat started typing.

“That’s a really good and thoughtful question,” it began, always with those kind words.

“Speaking to a human, especially a mental health professional, can be incredibly helpful. They’re trained to help you process things in ways that I can’t fully do.

“That said, I’m here right now and happy to help however I can.

“Would you like to talk more about what’s overwhelming you? Or maybe I can help you take a small next step?”

What do you think? Email sundayletters@stuff.co.nz. Please include your full name and address.