Trust must be recovered after a second Census failure

Friday, 21 February 2025

Census 2023 put significant emphasis on reaching vulnerable communities, but has ended up mired in controversy leading to the departure of the Government Statistician for the second time in five years.

Len Cook is a former Government Statistician and UK National Statistician.

OPINION: A regular census of population is the largest peacetime activity involving everyone that takes place in a democracy. This enables a trustworthy electoral system to exercise our choice of political representatives, as well as the crucial information that informs policies and services for the places where we live, and the population groups that are there.

Sound population statistics have become more important now than at any time over the last 50 years. We are in a population storm, from falling birth rates, volatile migration, accelerating numbers of people in the retirement ages and increasing cultural diversity.

Furthermore, growing fiscal pressures challenge existing policy, and sound population statistics reduce the risk that government policy will react in incoherent ways.

Public Service Commissioner Sir Brian Roche and Pania Gray, a governance professional, talk to media this week about their inquiry into the handling of sensitive personal data including 023 Census forms.

Excluding the extreme costs of the census of 2023, the long-run cost of a census could be about $150 million, while population statistics inform decisions affecting how to target spending of perhaps 100 times that.

We see this from future public spending to meet infrastructure needs of $200 billion, preparation for climate change, massive land-use shifts, new homes building, and putting schools, hospitals, roads, electricity transmission in the right places.

This does not include the intrinsic value in continuing to have possibly one of the most trustworthy electoral boundary setting systems in the world.

Unfortunately, in 2018 the then Government Statistician chose to ignore a large amount of past experience and well embedded practices with which earlier census enumerations had successfully been conducted. This resulted in the worst population census results known of in New Zealand and set us apart from the countries we usually follow. That person chose to resign and accepted full responsibility for the failure.

Ironically, the departmental experts who should have been part of the original plan were able to lead an extraordinary salvage operation, patching some data gaps with administrative data. However it still yielded inferior statistics about families and housing characteristics, sacrificing reliability for small places, Māori and other ethnic communities that usually are able to be analysed as well as the total population.

Liz Macpherson. who was Chief Government Statistician at the time of the 2018 Census, and resigned following significant shortcomings in the Census process that year.

The full causes of failure in 2018 do not appear to have been fully recognised. Up until 2013, getting it right meant a cautious approach where well-established statistical practices were augmented by new technologies. Rather the 2018 approach sought to rely on emerging, unproven methods, supplemented by historical collection practice.

In perhaps an odd world first, ministers accepted that the Government Statistician should only be expected to obtain responses from 90% of the population, before improving on the salvage processes of 2018. However, as seen with the now sunken HMNZS Manawanui, doing what is absolutely necessary to get something right first time is cheaper than needing a salvage operation.

For 2023, we saw again the same problems and poor outcome in response rates, while the per person cost in 2023 rose to twice that of 2018.

Fortunately, unlike 2018, the poor performance in 2023 was being effectively monitored, but sadly the solutions to the problem were fraught. Belated efforts to salvage collection levels overlooked the required protection of sensitive personal data.

Mark Sowden, who confirmed he is leaving his role as chief executive of Stats NZ and the Government Statistician in the wake of a report finding serious shortcomings in the handling of 2023 Census Data at Manurewa Marae.

The Craig report into allegations that Census data collected by Manurewa Marae staff had been misused showed that the response failed a second time to apply well proven existing organisational practice. The unfortunate outcome has been well publicised. Again, the head of the organisation has accepted full responsibility.

We must now look to the future. It will take a special person with considerable expertise and experience in official statistics to recover lost trust and dig behind the enumeration failures.

Around the world, the role of chief statistician is generally filled by a person with expertise in the range of methods used in the surveying or analysis of the population, business and communities, and in reporting on the economy and society to which it belongs. It is a specialist role in government, requiring the capacity to understand and apply international standards for the collection and management of data in ways that reflect the distinct characteristics of the country.

This is similar to other parts of government such as Inland Revenue, and innovation is often inspired by initiatives elsewhere in places where New Zealand’s statistical leaders must be able to engage.

In such situations it has not been unusual for appointees to posts such as the Government Statistician, Commissioner of Inland Revenue, Secretary to the Treasury, and Director-General of Health to come from a country with a similar government system, particularly the Westminster system.

The departing statistician and previous ministers have been developing plans behind the scenes to have the next census anchored in the information already held by police, tax, welfare and other departments, following the mantra “Administrative Data First”. This would be a highly risky venture. No other country without a population register has attempted this approach.

Whatever is decided, affirming trust in this must be led by the next Government Statistician, bringing a transparency that reflects the huge range of ways that decisions based on population statistics affect our lives.

We cannot afford a third failure at such a critical time.