A mid-May trial of the CovidCard contact tracing technology showed the device tracked fewer than two-thirds of clinically significant contacts when used in a cafe.
While the technology appears to work well in certain environments, like an office space, and regularly logged more contacts than people could remember on their own, it struggled in more complex scenarios like a cafe, a party or a construction site, according to documents released by the Department of Internal Affairs.
Andrew Chen, an expert in technology and society and a research fellow at the University of Auckland's Koi Tū - the Centre for Informed Futures, said the released documents showed the technology needed more trials to prove its efficacy.
"I wouldn't go so far as to say it doesn't work. But given that the previous statements that were publicly made claimed it could record clinically significant contacts with greater than 90 percent, I think this data suggests that that is true only in nice, clean, controlled circumstances," he said.
The card fared best in office scenarios, where more than 90 percent of contacts were recorded and only a small percentage of the contacts recorded were false positives. In more complex scenarios, it struggled. Some of this was environmental - Bluetooth signals can experience interference in places with large amounts of metal (like a cafe) or dust (like a construction site). The cards in the cafe on average recorded only 63 percent of the contacts they should have.
Chen noted that the documents only cover work up until early July, prior to the new hardware developed by the CovidCard team.
"Overall, my reading of this is that more trials were needed and that there was probably room for more development and iterative improvement on the current card design," commented Chen.
"The questions that I believe Government most need to answer, the important question is, can we get people to wear it? There's no point in proceeding if you don't think people will wear it."