In May 2023, we started the ST.AI experiment, in collaboration with colleagues from the Social Insurance Bank (SVB). The aim of this experiment was to investigate whether we can use artificial intelligence (AI) to answer questions from citizens in conversation via the SVB website.

Reason and Team

Inspiration sessions are regularly held within the SVB. In 2022, the results of the Novum research into generative AI were also presented. This led to two AI experiments in 2023, including the ST.AI trajectory. We formed a diverse team consisting of a UX Designer, innovation coach and designer, an ethics expert, an algorithm expert and a developer.

What is Conversational AI

Conversational AI is a collective name for technologies that provide linguistic interactions between machines and people. Our goal was to investigate whether we can provide SVB.nl visitors with better information using Conversational AI. You often see this in the form of a chatbot, as in this experiment.

Design of the first experiment

Our UX Designer built a prototype that allowed us to perform a first user test. This user test focused on visitors of the website who receive child benefit. An important assumption was that a Generative AI system can provide these visitors with crucial information. We also thought that using an LLM on the SVB website would improve the provision of information and increase customer satisfaction. The respondents were recruited from the SVB customer panel. The test showed that the respondents appreciate this form of information provision and would use it if it was available. The prototype was limited, but gave us enough insight to start a follow-up experiment.

Design of the second experiment

In April 2024, we started the second experiment. This experiment had to further investigate our assumptions about technical aspects, user behavior and needs. The Azure products “Azure AI Studio”, “Azure OpenAI Studio” and some separate services were chosen to set up the test environment. These turned out not to be well connected. After consultation with Microsoft support, it turned out that these products would be further improved in the coming months. That is why it was decided to temporarily stop the experiment.

Next Steps

We continue to monitor the developments of Microsoft Azure AI Studio and relevant functions and services. The ST.AI experiment is a next step towards even better information provision for our visitors using AI.