Incidents such as Crowdstrike chaos likely to happen again

Tuesday, 23 July 2024

A Woolworths store at Crofton Downs in Wellington was still warning customers of potential problems on Sunday, but by then it appeared to be business as usual.

Incidents such as the Crowdstrike bungle that ‘blue-screened’ computers around the world are likely to happen again, occasionally, and when they do Kiwi consumers and businesses can only really expect to be spectators.

That is the blunt assessment of Breccan McLeod-Lundy, co-chair of NZRise, an industry body that represents home-grown technology firms.

And, no, business continuity insurance probably won’t cover businesses from the financial fall-out, according to the Insurance Council.

On Friday, an incorrectly formatted update rolled out by Texas-based cybersecurity firm Crowdstrike to its customers caused — by Microsoft’s estimate — 8.5 million computers to crash around the world.

ASB, Jetstar and Woolworths were among the most affected businesses in New Zealand, though all appeared to recover over the weekend.

McLeod-Lundy said the country got off relatively lightly as Crowdstrike is a tool that is mainly used by large corporations.

Crowdstrike appears to have rapidly grown its business in the region, doubling the revenues of its Australian-based subsidiary to A$139 million (NZ$154m) in the year to the end of January last year.

National Cyber Security Centre is warning of scams in the aftermath of the global Crowdstrike outage.

But it is not understood to employ any staff directly in New Zealand, where its address for service is Auckland law firm Baker Tilly Staples Rodway.

Crowdstrike’s security tool is designed to sit within the core “kernel” of computers’ Microsoft operating system.

One question many computer-users have been asking is whether the company should have rolled out an update to all its customers at once, given the potential for mistakes in software at that basic level to “brick” computers.

But McLeod-Lundy said the trade-offs involved in potentially testing updates with a limited subset of users weren’t always straightforward.

If an update was needed to address say a “zero day” vulnerabilty — one that fixes a software vulnerability that hackers found about first and were exploiting — urgency could be the order of the day, he pointed out.

The upshot of that is mistakes such as that made by Crowdstrike on Friday could and probably would happen again “very occasionally”, he said.

“I don’t think there are new lessons to be learnt.

“If you've got automated roll-out of your updates and you decide to install something that's working at that kernel level — which you have to if you want to do that kind of really intense security stuff — then that kind of software, it's possible, will break things.”

The question was whether “the risk of that extra security causing problems is greater than the risk of whatever you think it's protecting against”, McLeod-Lundy said.

“I think there will be some organisations asking themselves whether the amount of extra security they gained from installing something like that was matched by what they took. But once you've made the decision, that's the risk.”

JetStar is probably right to class cancellations caused by Crowdstrike as being outside of its control, Consumer NZ believes.

Some businesses were having to physically send USB sticks to reboot their computers, which was “a huge amount of work”, he said.

As well as being largely powerless to prevent such issues, the remedies businesses can seek appear to be limited.

Software vendors generally attempt to limit their liability as much as possible to ensure they can’t be held responsible for the knock-on damage any faults in their products can cause businesses.

Insurance Council spokesperson Patrick O’Meara said “a typical business interruption policy” was unlikely to cover Friday’s outage either.

A cyber insurance policy might provide “limited cover” and those affected should check their policy or contact their insurance advisor, he said.

He denied that exposed a gap in the insurance industry’s service.

Jetstar had to cancel 150 flights in Australia and New Zealand on Friday as a result of the impact of the faulty update, but all its operations were running smoothly on Monday, a spokesperson said.

“Customers affected by cancellations have been offered ‘additional flexibility’, including the option of free flight moves up to 14 days or a voucher refund,” he said.

It is understood JetStar is not currently offering cash refunds as it views the cancellation of flights directly as a result of the Crowdstrike issue as being outside of its control.

Consumer NZ spokesperson Jessica Walker said it believed that was probably right, given the scale of the outage.

Woolworths NZ said on Monday that its stores and its online shopping service were open and trading as usual.

It wouldn’t comment on any financial impacts, or whether it saw any lessons that could be learnt, by its business or by others.

The Government stood up the National Emergency Management Agency on Friday so it could coordinate a response to the Crowdstrike chaos.

But spokesperson Anthony Frith said it stood down on Saturday after it was determined that the patch fix had been effective and New Zealand was not as badly impacted as many other countries.

Its activities had included discussing the situation, assessing potential risks and providing information to the public via Facebook, he said.

Nema was not aware of any ongoing impacts to critical systems and infrastructure, he said.

In terms of its own response, it was standard practice after any event to reflect on how well things went while also looking for opportunities to make things better, he said. “This is still to be done.”