Is New Zealand flying blind on generative AI?

Saturday, 12 August 2023

The Government is failing to see the Pandora’s box of generative AI, experts say, meaning it’s not making the most of AI’s benefits and it’s not properly mitigating the risks.

Big tech firms including Google, Amazon and Microsoft are currently preparing to pump billions of dollars into generative AI, a move that will result in rapid advancements in its capabilities.

But there are questions over how fast the transition will happen, what new jobs will be created as a result, and what will happen when the technology’s alien intelligence surpasses that of humans.

“We have to understand how to have agency over it and keep our values intact as humanity … It’s like discovering fire. Fire keeps us warm, but it can also burn us,” said Maty Nikkhou-O’Brien, founder and executive director of non-partisan think-tank Diplosphere.

AI is limited due to it being trained on datasets and images, and not experiencing and sensing the world like people, but there is no way to ensure that would always be the case, Nikkhou-O’Brien said.

The next generation needs to be educated on how to use AI to their advantage without compromising their critical thinking skills, she added.

With the election eight weeks away, MPs have been urged to fix the country’s fragmented oversight system regarding AI, with warnings the technology’s impacts reach far beyond politics into workforces that are bracing for - or in some cases already experiencing - redundancies en masse as it replaces genuine employees.

Unlike in previous technology shifts in which repetitive back office jobs and manual labour became automated, creative and white-collar work is widely expected to be affected by powerful and unpredictable generative AI. The flip side is that for some workers like coders, AI can supercharge productivity.

Issues with AI are well-documented: being trained on datasets that are skewed towards certain cultures, leading to bias that can reinforce stereotypes; cultural appropriation and misrepresentation; large language models’ use of te reo sparking sovereignty concerns from Māori; non-attribution and non-compensation for the citation of others’ work; its use of people’s private data mined online; and its role in the production of harmful content.

The European Union has proposed the AI Act - the first comprehensive law on AI by a major regulator - which would impose restrictions on facial recognition AI that could create an “unacceptable” risk, regulate generative AI, hold bad actors accountable, and issue huge fines for the sale and use of non-complying technology.

There’s a high chance the Act will end up being the world’s de-facto AI regulatory template because companies in non-EU countries that do business in the powerful trading bloc will have to adjust their practices to comply with the law. Nikkhou-O’Brien urged New Zealand to move quickly as a fast follower.

Former chief censor David Shanks said AI technologies could potentially supercharge existing harmful uses of digital technology by terrorists, online predators and scammers at a rate that would be “very difficult” for authorities to keep pace with. AI also raised the spectre in its possible weaponisation to influence elections.

AI avatars would soon become “indistinguishable” from a real person online, which could create cases where vulnerable people are exploited when seeking companionship, Shanks said. Unsafe products needed to be curbed via effective regulation, he added, but this would be challenging as the technology didn’t respect borders.

The Government needed to establish guardrails to ensure human-centred and trustworthy AI was developed, and move on obvious gaps in existing rules, standards and oversight mechanisms that applied to AI and AI products, Shanks said. “Enforceable reporting and transparency obligations would be a good start.”

But historian and NZ Initiative research fellow Matthew Birchall warned against heavy-handed regulation in response to a phenomena people didn’t yet understand, and urged for more rigorous cost-benefit analysis of AI. Its benefits in lifting the country’s productivity, driving innovation and new business models, and to fields like biomedical sciences and disaster management needed to be weighed against its risks.

Birchall was particularly worried about AI’s implications for warfare, and the danger of autonomous weapons systems.

Asa Cox, founder and chief executive of Arcanum AI, is concerned New Zealand isn’t investing enough in AI, including in a national AI defence. “Some humans will make the decision to use AI for bad; unless we have invested sufficiently … we could be overwhelmed and suffer the consequences,” Cox said.

A lack of AI experience within government had left New Zealand “a long way behind global peers” in unlocking its benefit to productivity. The Government needed to develop sector-specific AI strategies, and make decisions on what technology New Zealand didn’t want to have on the market, he said.

Digital Economy and Communications Minister Ginny Andersen said generative AI’s rapid momentum had confronted sectors and heightened the Government’s focus on its potential. The international trend was to take a risk-based approach to AI, she added.

New Zealand’s existing privacy, consumer and competition laws could address some AI-related harms. Guidance had been issued for generative AI’s use in the public service, and a cross-agency work programme on AI involving the public sector and wider economy was being developed to help Aotearoa respond to its opportunities and risks, Andersen said.

Statistics NZ was also establishing an interim Centre for Data Ethics and Innovation.