I’m learning all the time — conversation-driven development in a chatbot for Erasmus students with Rasa framework | by Joanna Trojak | Feb, 2021

I am sure that some of you are familiar with Test Driven Development or Behaviour Driven Development. I remember both of them from my Software Engineering at FEUP. When I have started doing chatbots, I have discovered something else, conversation-driven development.

According to the article entitled Conversation Driven Development found of Rasa blog:

Conversation-driven development (CDD) is the process of listening to your users and using those insights to improve your AI assistant.

The article states that the biggest problem for chatbot developer is anticipating user’s input. Instead of assuming what they are going to ask the chatbot we give them the opportunity to say what exactly do they want.

The process of Conversation Driven Development is the following:

1. Share
2. Review
3. Annotate
4. Test
5. Track
6. Fix

Based on the project I have done for my master degree dissertation and some other projects, I have prepared a case study of conversation-driven development for a chatbot for Erasmus exchange students coming to the foreign university.

Framework: Rasa + Rasa X

Purpose: The objective of the project is to assist Erasmus students coming to the university from many different countries and trying to navigate their student life in Poland. They require information about the procedures, documents, professors and campus life.

1. Case Study: Building Appointment Booking Chatbot

2. IBM Watson Assistant provides better intent classification than other commercial products according to published study

3.Testing Conversational AI

4. How intelligent and automated conversational systems are driving B2C revenue and growth.

1. Share — according to the process it is good to share your chatbots with test users early on. I usually don’t spend much time deliberating on dialogue flows. I design a simple diagram with some paths to follow, I train the chatbot in Rasa and call my friends asking them to help. Usually, I make my chatbot available through ngrok tunnel but I found that this solution can be unreliable. Now I deploy my chatbot on a public server. http://35.205.219.197/guest/conversations/production/36a32a4440a84b7095ef45d55feb418c
2. Review — once some people talked to your bot, it is time to read the conversations they had. A lot of developers are focused on metrics like how many people have used that intent but it is better to just sit and read through the conversations because people may have interacted with the chatbot, not in the way that you have expected but it could be that their path should be included in the next version. Rasa X has the option of looking at the previous chats.

3. Annotate — after going through conversations it is time see the intents that would be great candidates to improve your chatbot’s NLU. You can see them in NLU inbox and mark them as correct. Otherwise, you can change the desired intent or create a new one.

4. Test — You can use the previous conversations as test stories that will allow to further verify your chatbot. You can write them in the test directory of your Rasa project and run them often to check how well your bot is performing or you could automatically convert the successful dialogues with the chatbot you had into test stories once the bot is on the server and it is connected to the version control.

rasa test

Testing chatbots in Rasa is not just about the stories. You can evaluate the nlu understanding, checking intent classification and so on. I will probably devote a second article just to the art of extensive chatbot testing.

5. Track — you have to track how well your chatbot is doing. For instance, is the chatbot successful in convincing people to use your online store or how often a particular user is taking to the bot and is she satisfied? You can track the sentiment of the conversation with BERT or just simple logistic regression or Bayes rules.

6. Fix — analyse the conversations based on the performance. If they went smoothly they can become part of the testing set. If not, find out what you need to improve. You could need more training data or fix your custom actions.

Conversation driven approach is user-centric. It is certain that it needs time and attention. I have to often jump between many different stages. Nevertheless, I am working on creating a great chatbot which adapts to the user and not another way around. After all, this is what good conversation is all about.

from
https://fatsfixedassettracking.com/im-learning-all-the-time-conversation-driven-development-in-a-chatbot-for-erasmus-students-with-rasa-framework-by-joanna-trojak-feb-2021/1444/