What’s your P-doom? Six increasingly frightening bits of AI jargon, explained

Tuesday, 16 June 2026

If there is one thing Silicon Valley produces more of than billionaires, businesses and controversies - it’s acronyms.

But behind seemingly innocuous bits of jargon like AGI, GPT, or RSI are enormous — potentially catastrophic — implications for society.

When Google executives start saying things like “we are in the foothills of the singularity,” as they did recently, understanding what they’re actually claiming is critical.

Here are six terms you should understand, arranged in order from interesting but harmless to existentially terrifying.

ChatGPT

The one that started it all. GPT means Generative Pre-Trained Transformer. The Generative part is pretty straightforward, meaning it creates content in response to your prompts.

Pre-trained refers to the vast quantities of data required to teach an AI system to recognise human language and respond (relatively) naturally.

Transformer is the important bit, referring to the underlying innovation first discovered by Google in 2017. It is what most of the AI boom we see today was built upon, following a seminal paper called Attention Is All You Need (available here as extra reading for nerds).

Very basically, the transformer architecture is what allows AI systems to turn human language prompts into maths problems and then back again.

Agentic AI

The big buzzword for 2026, Agentic AI refers to systems that take actions autonomously, over long periods, with little human involvement.

This can be as simple as asking it to scan for certain news each morning and to write a summary for you, or something more complicated, like building an entire app or website based only on a vague prompt.

While agentic AI has so far been focused more on coding, we are starting to see more mainstream versions of AI agents for everyday tasks, like Google's recently unveiled Spark.

The implications for general business are obvious: as increasingly autonomous AI agents start doing everything from answering our emails to booking our holidays, the list of ‘valuable work only humans do’ shrinks fast.

This could radically boost productivity or, to sceptics, lead to widespread job loss.

Recursive Self-Improvement (RSI)

This one might sound boring, but it’s when the jargon starts moving from concerning in a political or economic sense to a potentially existential one.

RSI refers to an AI system that is able to build a better version of itself. We are already seeing the beginning of this; this month, Anthropic revealed more than 80% of the code in its codebase was authored by its AI, Claude.

The fear with RSI is that once AI systems can completely code themselves, progress will speed up exponentially, as each successive AI model builds a better successor and does so faster with each iteration.

RSI could lead to a situation AI researchers sometimes call “liftoff” or an “intelligence explosion”, where the capability of the systems shoots up far faster than our ability to build safeguards.

Importantly, this outcome is not a guarantee, and some argue progress will always be bottlenecked by the enormous energy and physical infrastructure required to train progressively larger models.

Artificial General Intelligence (AGI)

If the current AI boom is a race, AGI is the finish line.

Every AI expert has their own definition, but a generally accepted one is a system as capable as a human across most economically useful tasks.

Whoever builds AGI first will create something feasibly capable of doing most human labour — at least in white-collar work. But with the speed at which robotics is developing, that limitation may not last long either.

When exactly AGI arrives is the multi-trillion-dollar question. Estimates vary wildly, but all of them place it much sooner than our economic or political systems seem ready for.

Anthropic say it could arrive as soon as next year; Elon Musk estimates around 2028; others push the timeline out slightly further, but consensus currently hovers around 2030.

Ben Buchanan, former US President Joe Biden's chief adviser on AI, said based on his many briefings, the White House was preparing for “very capable” AI systems — meaning AGI — in the President’s second term, had he won.

So, whichever government wins the 2026 New Zealand election should probably be planning for an economy radically different than anything we’ve yet seen.

Artificial Superintelligence (ASI)

Also called the “singularity”, creating some form of this is both the nightmare scenario from a safety perspective and, unhelpfully, the stated end goal of every major AI company.

A superintelligent AI would be more intelligent than any combination of humans — even the collective intelligence of humanity as a whole. This is what some in Silicon Valley refer to, with a straight face, as “the digital god”.

Depending on who you ask, the gap between this and AGI might end up being very short.

Any system that is generally as capable as any human, but can work much faster, can very quickly build a much better version of itself (see RSI above).

For people who worry about these things — which arguably should be all of us — controlling a superintelligent AI could be impossible.

Humans have never interacted with something more intelligent than us, let alone an entity as far above us intellectually as we are above a chicken.

For AI cynics, trying to contain a superintelligent AI would be like a toddler drawing a circle on the ground around its parent and saying, “Haha, I’ve trapped you”. If its goals ended up diverging from our own, we would be powerless to stop it.

For optimists, ASI is the last thing humanity ever needs to build. It could be the answer to every challenge: climate change, cancer, even ageing itself. Every single problem humans have ever solved has come about from the application of intelligence.

So, the argument goes, if we scale up the amount of intelligence available almost infinitely, basically anything becomes possible.

Whether we can safely build ASI in order to reap those benefits, may be the question of our time.

P-Doom

A dark in-joke in Silicon Valley, P-Doom refers to “percentage doom”, meaning the probability someone gives that AI will destroy humanity.

Most AI CEOs are asked about their P-Doom at some point. Most dodge the question, but there’s a handy website that tracks those who have gone on record.

Geoffrey Hinton, a Nobel Laureate and the researcher responsible for a lot of the foundational work on AI, has a P-doom of 10–20%. He has recently gone as high as 50-50.

Yoshua Bengio, the world's most cited living scientist, estimates a 20% chance uncontrolled AI kills us all.

And there you have it. Six bits of increasingly troubling AI jargon unpacked - sorry to leave it on such a down note.

But hey, if the AI ever does turn on us, you’ll at least be able to heckle it using correct terminology.