Survey finds worrying holes in management and accessibility of public records

Tuesday, 28 January 2020

When police collected evidence from the Pike River mine disaster, a potentially crucial exhibit mysteriously disappeared.

An electrical cabinet door blown 100m to the mine surface – which could have provided clues to the cause of the fatal explosion that claimed 29 lives – was a casualty of a record-keeping process described as 'diabolical' by police investigators.

Pike River re-entry minister Andrew Little later said the switchboard door was probably stored somewhere safe. But safe is no use, if no-one can find it.

There, in a nutshell, is why public record-keeping matters, and how it can fail spectacularly without the right systems and culture in place to ensure information is recorded, classified and stored so it can be easily recovered.

* The OIA is broken, but we are not keeping track of how badly

Public records are supposed to be a window on the nation's past. But are they being stashed where they can't be found?

* Hide and Seek: How politicians seek to hide your information away

* Anatomy of a resignation - how a cup of coffee led to Hirschfeld's resignation

* Public sector organisations criticised for poor record-keeping**

The Public Records Act (PRA) requires public offices to keep full and accurate records to enable the government to be held accountable. As chief archivist Richard Foy puts it, it's the flipside to the Official Information Act. You can't access information under the OIA if it was never created, if it's been filed in the wrong place, or if it's stored on ancient technology that has never been updated. Of the 189 complaints to the ombudsman in 2017-18, 82 related to information that did not exist, or could not be found.

Chief archivist Richard Foy wants to see public records managed with the same care as cash and buildings.

'There are issues arising during the process of locating, compiling and releasing information for OIA requests that are the direct result of poor record-keeping practices and inadequate information management systems,' Foy wrote in a submission about an OIA review. 'These issues frustrate the access to official information to which New Zealanders are entitled.'

Until now, there's been no clear picture of how well public organisations take care of their information. But a new survey has revealed major holes through which important information could be slipping out of reach of accountability. On a scale of one to 10, Foy rates our public records management as more than a five, but 'definitely not 10'.

While information is not as obvious as buildings or cash in the bank, it's just as valuable, because Kiwis need a record of decisions that affect their rights and entitlements, Foy says. And it needs to be better looked after.

'You need to have good governance and treat information in a similar way to other assets of your organisation – we do it for money, we do it for property, we do it for our people. Information is probably one of those slightly invisible things – something we know we need to get people to pay more attention to.'

In that he has support from Chief Ombudsman Peter Boshier, who says careless handling of precious information, or vague notes of important meetings, is a 'fundamental undermining of democracy'.

'I believe there is no excuse for poor record-keeping in the digital age. We have moved on from the days of officials placing sensitive files in the back of a basement filing cabinet in the hope they wouldn't be seen.'

THE KEEPERS

Of 27 large organisations surveyed, almost one-third had three or fewer dedicated information management staff.

Police were criticised for 'diabolical' record-keeping in the Pike River mine explosion investigation.

The 2019 Archives New Zealand public sector information management survey covered 254 public bodies, from district health boards to city councils to government departments.

Of all the survey responses, staffing caused Foy the greatest concern. Of the organisations with 500-2999 staff, 18 per cent had no full-time, dedicated keepers of information. Of the 27 organisations with 3000 or more staff, four had just one or fewer dedicated information staff – New Zealand Post, Capital & Coast District Health Board, Bay of Plenty DHB and the University of Auckland. In the case of health boards, clinical records managers were not counted.

Another four organisations with more than 3000 staff had 1-3 information keepers – Fire and Emergency New Zealand, Canterbury District Health Board, Waikato District Health Board and KiwiRail.

'It is especially concerning that some of the larger organisations seem to be significantly under-resourced,' Foy concludes.

Information managers make sure important, high-risk records are properly protected and stored, that information can be recovered in a disaster and that new IT systems organise information in a way that can be easily searched. (Only 23 per cent of survey respondents said information management was fully built into their new IT systems.)

Conservation Department boss Lou Sanson resolved to do better after the ombudsman found limited-access documents might not appear in information searches.

While it's possible one amazing person could shepherd the records of an organisation with more than 3000 staff, it's about as likely as one finance person being able to administer the public finance act, Foy says.

'What are the chances of that?'

IT'S GOT TO BE AROUND HERE SOMEWHERE

RNZ board chairman Richard Griffin tried to argue a voicemail message from broadcasting minister Clare Curran about a meeting she had with RNZ staffer Carol Hirschfield was not a public record. The chief archivist disagreed.

One in six survey respondents said that, in the past year, they struggled to answer OIA requests for information they believed existed, but couldn't find.

In 2012, Sport New Zealand organised a meeting between then minister of sport Murray McCully and Christchurch mayor Bob Parker, to discuss the post-quake rebuild of the city's sports facilities. When someone later asked Christchurch City Council for information about the meeting, the request was refused because no records were found.

The requester complained to the ombudsman, who notified the chief archivist. He sent Sport New Zealand digging deep in its information systems, where some notes of the meeting were eventually discovered. They had not been transferred to a new IT system, so did not come up in information searches.

That's just one way in which bad systems and practices pull a veil over information, obscuring it from view. Chief Ombudsman Peter Boshier's investigations into official information handling in 12 public agencies revealed others – lack of training, issues working between multiple systems and council workshops going undocumented.

During his investigation into the Conservation Department, almost four out of 10 staff surveyed said information was not easily found or collated. Some documents required special permission to access, and there was confusion over whether those documents appeared in searches. That raised the risk the department was telling people information did not exist, when it did, Boshier concluded.

An audit of Christchurch City Council found senior staff were doctoring documents to avoid negative information getting out.

Staff also told Boshier many people did not file emails and other documents into the information management system, or did not stick to document-naming policies designed to ensure that information could be easily found. It's the digital equivalent of Pike River's missing switchboard door – the fact the information probably exists is no help if it's not accessible.

It's a common problem. More than half the public servants replying to Foy's survey said a lack of contextual information – think filing categories, keywords, metadata – risked preventing records from being discovered and interpreted.

Director of government record-keeping, Antony Moss says the Pike River failure is 'a good example of the challenges of volume and complexity and having to have those systems in place and designed in a way that you can actually manage that'.

And it's only going to get worse, as officials create millions of new documents, emails, text messages and memos every day.

Obsolete technology causes headaches for information management.

WHEN IS A PUBLIC RECORD NOT A PUBLIC RECORD?

One in six organisations surveyed said they could not respond to official information requests because the information did not exist.

In March 2018, then minister for broadcasting Clare Curran left a voicemail message on the phone of then RNZ chairman Richard Griffin. It related to a controversial undisclosed meeting between Curran and RNZ senior manager Carol Hirschfeld, which contributed to both women eventually leaving their jobs.

Chief Ombudsman Peter Boshier was scathing in his criticism of Christchurch City Council's handling of public information, saying he had 'serious concerns' about a culture of secrecy among some top staff.

Griffin refused to release the message, saying 'It's my recording and I'm not too interested in handing it over.' But Foy says any record of public business is a public record, whether it's a text, an email, a voicemail or a formal document. In the case of a voicemail, it should be kept either as a recording, or a note should be made, with enough contextual information (metadata) to make it easily searchable.

But many organisations lack good systems to capture business messages sent to personal emails, phones or accounts, which means these kinds of records are not kept.

'We must rise to the challenge of preserving records in all their formats, so they can remain accessible for accountability,' Foy says.

THE WHOLE TRUTH OR A HOLEY TRUTH?

When Boshier investigated how Christchurch City Council treated official information, he discovered 'deeply concerning' manipulation of information to try to keep the bad stuff out of the public eye. Staff said reports were changed, manipulated or intentionally left in draft form despite being completed. Some projects were simply not reported on.

'Once a LGOIMA [official information request] is in it's fine, but there is some culture to limit the creation of documents that may be open to LGOIMA in the future,' one whistleblower said.

This was also an issue raised in the OIA review consultation, with several government insiders saying officials and politicians avoided leaving paper trails of some discussions, so they could not be revealed by an OIA request. That would put them potentially in breach of their record-keeping responsibilities under the Public Records Act.

Foy says any case of officials intentionally not creating accurate records is concerning, but he does not believe it's widespread. Where officials are giving 'free and frank advice', that's already protected from disclosure under the OIA. So there's no excuse for avoiding creating a record at all, he says.

ANYONE FOR A 3½ INCH FLOPPY DISK?

More than a third of organisations surveyed had information stored on obsolete file formats and almost half had records stored on out-of-date business systems.

'Not all information is created equal,' says Moss. If you've lost the key to a drawer containing moth-eaten old jumpers that you're never going to wear again, that's not a major deal. But you don't know that unless you know what's in the drawer.

Given many agencies still don't have a big boss overseeing how information is looked after, or an information asset register to keep track of what they've got, Moss says it's worrying – but not surprising – that some organisations still don't know what their information risks are.

'Many agencies won't have a good enough layer of governance to even know whether they've got some of these problems lurking in their systems … Sometimes it won't matter if you've got old stuff … You might have a whole bunch of things in WordPerfect or WordStar which actually you may never have to tap into again. That's the thing a lot of organisations struggle with. You've got a lot of information and data – you actually need to take quite a strategic approach to managing it like an asset or resource.'

And while old paper records are expected to keep arriving at Archives NZ until about 2040, a new process will be needed to manage the transfer of digital records, Foy says.

'I think the system as a whole is at a very interesting transition point. We know how to get records boxed up, put into trucks, driven to the loading dock, do the accessioning, bring them in. We can do that on a small scale, case by case, with born-digital transfers. We're at that point where we need to work with the system to really create that equivalent in the digital world.'