Summary

We did the prototype test for Voice ID with storyboards and it turned out that the participants see no added value from the addition of a Voice ID in the telephone channel at the government. This test was done with a small group and there were no visual impairments among the participants. They may think very differently about it. Authentication over the phone can be made easier and less cumbersome via existing means of authentication such as DigiD. After all, this is already known and feels safe and familiar to many people. For people who are less comfortable with digital government services, there is a safety net through a counter where telephone services are not possible due to the lack of telephone authentication. The strongest and most certain conclusion of this test is that creating a Voice ID while you want an answer to your question is not workable and that it is undesirable to increase your Voice ID again. motto: good first time.

Introduction

To test whether people would like to use Voice ID, we set up a prototype test using Storyboard. We chose to do a prototype test via storyboards where we took people through a number of scenarios. In this test we examined what people would think of Voice ID, when they would want to create it, how they would want to create it, how they would like to increase the confidence level and how and where to use Voice ID. 

We obtained the respondents of this test via the online flash poll, where they could indicate that they wanted to help with a test in a follow-up process. 

We thought a prototype in which people would feel that they were going to create and use a real Voice ID was too big a step for now because of the insights we gained in earlier phases of the Voice ID project, such as the Moral Impact Assessment. This showed that the purchase and storage of a Voice ID must be set up properly, safely and reliably.

Why this test?

From the Voice ID project, the question was: "Can we identify citizens who call the government with their own voice?" To this end, we conducted an online Flash Poll at an earlier stage. In addition, we have also asked ourselves the question, Do we really want this? Just because something is technically possible doesn't mean you have to want it. To find out, we performed a moral impact assessment (MIA). Policy has also asked to investigate whether Voice ID, and in particular the onboarding, can be made accessible enough for the target group.

We took the various points of attention that emerged from this to test them with potential users of Voice ID.

What were the research questions?

The research questions we wanted answered during this storyboard test were:

• What do people expect from government telephone services?
• What are the channel preferences of citizens?
• Why do people call? what is the intention? Have the story colored

Why a test via Storyboards?

To get a good idea of what people think of a possible solution, you choose to make a prototype. In this case we used storyboarding: with it you mimic a kind of comic strip after people use Voice ID in a situation that they recognize. In our view, letting people actually record and use a vote printout would be too frightening, since this is a fairly new technology where much is still unclear about responsible design and the costs involved. Can we use this technology? How do we store the data securely? What are the costs of this? The most important question we wanted to answer first was therefore, do people want this? In which situation? And how do people view alternatives such as calling via an app such as DigiD?

How did we do the test?

During the Flash Poll we asked if we could approach people for a follow-up. From this list we have selected a mix of people who would or would not be interested in a Voice ID. We asked by email whether people wanted to participate in a test, this appointment could be scheduled via Calendly and in this way it automatically entered both their own agenda and ours. Here are 4 participants. We did the test through video calling with Microsoft Teams. After an introduction from those present in which the person who underwent the test also introduced himself, we asked if we could record the test for internal use, after which the test started. Through screen sharing we showed the scenarios in Miro and asked the testers a number of questions.

Lessons learned from the test

We noticed that the testers found it very positive that they were really taken through the test. By sharing the screen, the tester did not have to learn any tools, making the test itself very accessible. We did notice that zooming in Miro during screen sharing was necessary because otherwise it was not very readable for the tester. We also noticed that the scenarios were better understood by having the tester read the screens himself, rather than reading the screens aloud, which allowed immediate response.

Results

We did the test through 3 different interviews to get the widest possible picture.

Scenario 1 Questions about AOW

In this story we follow Mrs. Jansen. Mrs. Jansen has a few questions about her AOW pension. To do this, she calls an employee of the SVB. This body pays out the pension. The SVB employee explains that verification questions must be asked before providing personal information. A Voice ID can be created immediately in that process. It can therefore also be used by other government agencies.

Conclusions from the interview

• It is logical that you have to provide additional information for personal questions via control questions, otherwise everyone can do this.
• Control questions don't feel very safe, these can be things that others know or can learn about you.
• Getting the question about Voice ID during a conversation is nice, but the first priority is to get my question answered, then I'm not going to create anything now, but I want an answer to my question. It is also a lot of information that you have to process if you do not count on it.
• Option to create a Voice ID after the conversation is nice, but people will not do so quickly because they already have their answer and want to continue.
• To create a Voice ID, people first want to get more information and not create it immediately, that feels too intrusive and therefore not safe. It would be nice to be offered more information after the conversation.
• Getting an explanation about Voice ID is nice, but people prefer to read more about it to learn more about security and technology. Preferably in a reliable place, such as a government website where scientific information can also be found.
• Promotion for Voice ID could, for example, be done via TV/Radio spots or flyers, which makes it more known, for now it is unknown and unloved.
• The creation process is simple in itself, but it does not feel safe because only control questions are asked without extra steps.
• Is my voice really that unique? And is it really enough just to say a few sentences to be able to recognize my voice in the future?

Scenario 2 Report a move

In this story we follow Mrs. Jansen. Mrs. Jansen wants to report her move to the municipality. She calls an employee of her municipality to do this. There she is told that a move can be reported with her Voice ID, but that a one-time connection with the DigiD app is required for extra certainty about her identity. An alternative is to have the Voice ID raised at the municipal desk, who can also immediately arrange the move.

Conclusions from the interview

• It is logical that you cannot report your move by telephone without additional information.
• To report a move, a higher reliability level is needed, you have to raise this once at the town hall, there you can also report your move, people prefer to do that, you also know where the town hall is. You must be able to explain its usefulness well.
• If people have DigiD, they do not call to report a move, but they do so digitally at the municipality. Our participants prefer to arrange it digitally, but if they were to call to pass it on, Voice ID turns out to be unusable. If they do not digitally increase their Voice ID themselves, they prefer to arrange everything in 1x at the counter. Passing on the move, and then immediately raising the Voice ID for a possible next time.
• Glad I can use my Voice ID. How does the employee actually know this? and is this okay? Don't you have to give permission first to use your VoiceID? It also raised the question “which other organizations can recognize my voice? And what can the government actually do with all this? I want to know more about it and give explicit permission to organizations to use it, just like with the Message Box.
• Not everyone was enthusiastic about raising the confidence level. It didn't make sense. What we heard was: then I find out that it is not safe enough yet, which shocked me. Did something go wrong the first time?
• When I create a Voice ID, I have to get it right the first time so that I can use it everywhere. I would have liked to do the link with DigiD the first time, or even better would have wanted to receive information afterwards so that I could have done it in my own time.
• Increasing via an app is not useful for people with less digital skills and the counter is also very cumbersome. Especially because, as soon as you have to go to the counter, the raising is no longer necessary. Since you can also handle it there without increased Voice ID.
• It makes me nervous that I have to do this on the spot and the operator is waiting for me.

Scenario 3 Opening a checking account

In this story we follow Mrs. Jansen. She would like to open a new checking account with the bank. She knows exactly what kind of account she wants and calls the bank for this. There she is told that she can open the bank account directly with her Voice ID.

Conclusions from the interview

• Why do I have to enter my BSN at the bank? Is this okay?
• How can the bank use my government Voice ID? I don't want that at all.
• This is very fast, handy, but is that right? Surely the bank needs more information than just my Voice ID?

General remarks

• A voice print does not feel very safe, this is largely because it is still very unknown, but also because it is assumed that it is easy to imitate. This is mainly due to the deepfake video samples.
• If there is an opportunity to use Voice ID, there must be a good campaign around it that makes it clear what it is, where you can use it, why and how you can create it.
• If one were to take a Voice ID, they would like to be able to indicate where they want to use it at the government. They do not want, for example, banks to be able to (automatically) use this. A lot can go wrong in banking if it is not safe, but with the government – according to the perception – it is not that bad.
• People would rather opt for something of logging in with numbers, but even better with an app that they already know, such as DigiD.
• Calling via an app also feels cumbersome, people look up the number and call, an authentication step during the call feels more logical.
• Voice ID was also seen as a replacement for the control questions, which felt less safe because others could look up the answers.
• Voice ID makes sense if you call the government more often, not many people do this, especially not for personal services.
• Typing something feels safer than saying something.

Quotes

• I'll go to the counter, that's no problem, I'm willing to put in some effort.
• I feel this is more convenient for the agency I'm calling but not necessarily for myself.
• I don't want a piece of DNA with the government.
• I feel pressured because I have to create it now.
• I want control over my own data. Preferably storage on my own Smartphone.
• I want to be able to decide for each unique moment which data I send.
• I want to manage which government body may have my data. There are many different agencies
• Give me a robot on the phone to answer my questions, if it's complicated it can also be done through a human
• I'm annoyed to ask for BSN, you have to look up again and you're at the back of the queue again
• Rather not use at the bank, it is dramatic if something goes wrong there.

Storyboards

Below is an example of illustration including introduction.
Full storyboard is available via Miro: https://miro.com/app/board/o9J_kov7lJI=/

What alternatives do we already have in mind?

• Contact ID – www.idcontact.nl
• Video calling – project VNG https://www.vngrealisatie.nl/producten/betrouwbaar-beeldbellen
• We could also give the example that you just gave: that you are already logged in with a trusted account on a device that you use to call, such as a smart speaker, whether or not in combination with local voice recognition. For example Amazon/Alexa: https://www.pocket-lint.com/nl-nl/apps/nieuws/amazon/153135-amazon-fresh-legde-kostenbedrijven-uit-hoe-u-kunt-aanmelden

What steps have we not taken that we actually wanted to do?

1. Continued on the storyboard as we heard quite a lot of the same points like freedom of choice, security and not wanting to use it at a bank. We didn't do this anyway as the accumulation of results from the flash poll, MIA and storyboards gave the impression that it was better to stop.
2. We wanted to build a real prototype to let people experience in a real situation what creating a voice ID was and what it felt like to use it. We didn't do this as this was a more expensive solution than storyboarding. In addition, choices had to be made about how and where real voice impressions would be stored, this didn't feel right.

Questions from the Moral Impact Assessment

• Framework of standards, there is not yet a framework of standards for biometrics, a framework must be drawn up for voice biometrics and its application for authentication;
• Purpose limitation, ensure that the voice impressions cannot be used for other purposes;
• No central database with voiceprints, explore the possibility of storing a voiceprint on a device;
• Set up supervisor for voice biometrics;
• Freedom/opt-in: no legal obligation to use voice recognition. Other means of identification must remain possible;
• Clarity where to bet and not to bet. Only use where a match is in the user's favor;
• Equality: design in such a way that no mobile phone is needed;
• Test the technology yourself and don't just believe the manufacturers;
• Roll-out, ensure that the user is involved in advance during the roll-out, also provide a good explanation of what it is, how it works and why to use it;
• Prove to the user that it works and how it works;
• Help with creating Voice ID by users. Enrollment is the crux and it is difficult to make that process simple/accessible;
• Adjust fallback in case of a non-match.

Conclusion

The sum of all points gives us the result that we do not continue with this. According to residents, there is no added value for this solution. In addition, there is a lack of legislation and regulations. The technology itself is currently too unknown for residents and government to feel familiar. The research into contact ID that provides authentication over the phone via DigiD gives a positive picture and will take further shape in 2022. An important point for now is that there is an existing workable alternative, namely a counter.