How do chatbots work? What is the Chatbot Architecture 101?

Understanding Chatbot Architecture 101: A Beginner’s Guide

The world of communication is moving away from voice calls to embrace text and images. In fact, a survey by Facebook states that more than 50% of customers prefer to buy from a business that they can contact via chat.¹ Chatting is the new socially acceptable form of interaction. It enables brands to reach out to their customers anywhere, anytime. By providing easy access to service and reducing wait time, chatbots are quickly becoming popular with brands as well as customers.

Chatbots or automated conversational programs offer more personalized ways for customers to access services using a text-based interface. The latest AI-powered chatbots can understand a query (question, command, order, etc.) from a given context by a human (or another bot, inception) and revert with an appropriate response (answer, action, etc.).

But how do chatbots work? What makes it possible for these computer programs to understand user intentions? And how are chatbots able to contextualize responses for different conversations? Let’s find out!

How do chatbots work?

There can be multiple types of chatbots and the way they work changes accordingly. Chatbots can act as virtual assistants, question-answer bots or domain-specific bots. The question-answer chatbots are less complex and require a smaller skillset. They are mostly knowledge-based, and their capabilities are limited to answering only a specific set of questions. On the other hand, chatbots that harness the full potential of AI and ML can mimic human conversation and maximize user experience.

Majority of chatbots are designed to work using one of the following capabilities:

Pattern matcher

Rule-based or scripted and structured chatbots mostly fall under this category. Such chatbots use a knowledge base which contains documents and each document comprises a particular <pattern> and <template>. When the bot receives an input that matches the <pattern>, it sends the message stored in the <template> as a response. The <pattern> can either be a phrase like “What’s your name?” or a pattern “My name is *”, where the ‘*’ is a regular expression. Typically, these <pattern> <template> pairs are manually inserted.

This means the questions a user must ask fall in line with the programming of such QA chatbots. They classify text by segregating a piece of information (word or sentence) or generic tags from multiple categories and produce suitable responses for the end consumer.

For instance, ELIZA, one of the early chatbots created in 1966, is a rule-based bot. It matches patterns to classify text. It parses the input text word by word, looking up for its meaning in the dictionary, ranking it based on importance, and storing it on a keyword stack. The keyword which scores the highest rank of importance is considered as the <pattern> and this pattern is then matched against the documents in the corpus to find the appropriate response. If there are no existing documents corresponding to the input, ELIZA responds using phrases like “I see” or “Please go on”.

A simple pattern matching examples looks something like the following:
```
#PATTERN MATCHING
<category>
    <pattern>What is the capital of \*</pattern>
    
    <template>
        <srai>The capital of <star/></srai>
    </template>
    
</category>
------------------------------------------------
#PRE_STORED PATTERNS
<category>
    <pattern>The capital of the United States of America?</pattern>
    <template>The capital of the United States of America is Washington, D.C.</template>
</category>
<category>
    <pattern>The capital of India?</pattern>
    <template>The capital of India is New Delhi.</template>
</category>
```
Scripted bots such as Eliza are mostly used on Messenger Platforms today. They are mostly deployed to send newsletters or daily content pieces, generate leads, do surveys, etc.
Suitable algorithms

Compared to rule-based models, suitable algorithm based chatbots don’t just match a pattern against <status> or <response>. They choose a pattern matching algorithm and compare the input sentence against the <responses> in the data corpus.

Algorithms play a major role here as they help chatbots in analyzing large datasets. This reduces the work of pattern matchers. A set of words belong to a class and with every input, each word gets counted for its occurrence. It is then counted for its common type and by leveraging algorithms, an overall rank is given to each class. The algorithms in place then provide a score to the class with the highest rank, which is most likely associated with the input sentence.

In short, the chatbot uses the message and context of the conversation for selecting the appropriate response from a predefined list of canned messages. While the highest score only provides relativity and does not guarantee a perfect match, these chatbots provide more predictable results compared to rule-based bots.

The availability of many algorithms has made it easier for developers to build suitable algorithm-based models. These chatbots are more practical and can be deployed to interact with users and deliver the right response. By leveraging Natural Language Understanding (NLU) and a certain set of algorithms, these chatbots can perform transactional tasks and serve specific purposes.

These models work on the algorithms of directed flows, solving user queries in a way that it moves them gradually to a solution. Instead of intent, they follow a systematic approach and are based on pre-existing data. They are commonly used for goal-oriented tasks, keeping in mind user’s expectations from the brand and possible set of queries in a given context.

For instance, alVin is a suitable algorithm based chatbot built with extensive knowledge of LV= Broker’s products, a UK-based provider of financial products and services. Its live chat service aids with transactional tasks and is overlapped with human support representatives, depending on complexity. alVin helps automate support process for direct queries. Human representatives are required to step in only in case of complex scenarios.
Artificial neural networks

An artificial neural network is a computing system that consists of several simple but highly interconnected elements or nodes, called ‘neurons’, which are organized in layers. These further process information using dynamic state responses to external inputs.

The personality of generative-based chatbots is decided by seq2seq artificial neural networks. While the suitable algorithm-based models require a database of possible responses to choose from, artificial neural network-based models build responses on the fly. Instead of using canned responses, the neural network of generative models is a deep learning model dedicated to handling a set of sequences.

The pattern detection to derive a desirable response grows stronger and accurate as the chatbot runs through each layer in the AI neural network. The three interconnected layers of the neural network which help the generative model to analyze and learn data are input layers, hidden layers, and output.

The input layer introduces patterns into the neural network and comprises of one neuron for each component present in the input data. The neuron transmits the information to other connected neurons downstream. It is then communicated to the hidden layer, where all the processing happens through a system of connections. At the end of the process, the hidden layer leads to the output layer, which has one neuron for each possible desired output.

A generative-based chatbot follows this neural network to understand user queries and to carry forward an intent-based communication. It conceives a sequence of context tokens (Input layer) one at a time and updates its hidden state. After processing the whole context sequence, it produces a final hidden state (Hidden layer), which incorporates the sense of context and is used for generating the answer (output layer). Thus, it combines two inputs - state and context.

By considering the chat history or past transactions, such chatbots take into account the state. By analyzing inputs from external data points and classifying these into appropriate layers, chatbots consider the context.

Amazon’s Alexa, Apple’s Siri, Google’s Google Assistant, and Microsoft’s Cortana are examples of generative based chatbots that are trained using many previous conversations, based upon which responses to the user are generated. AI neural networks make generative chatbots good conversational bots that employ deep machine learning techniques to resolve everyday problems in natural, conversational replies.

While these models will always have a response, they might sound arbitrary and not make sense all the time. They run the chance of generating inconsistent replies and are prone to give answers with incorrect grammar and syntax. Admittedly, generative models can be entertaining in nature with small talk capabilities with users. But the primary focus of chatbots is to keep in mind the goal of the customer, to help resolve support queries of users, and to provide them with relevant information.

Hence, a hybrid model with an architecture that comprises of both generative and suitable algorithm system can assist users better in scheduling reminders through a chat conversation. The flow-based suitable algorithm model has high precision and provides a grammatically accurate response but has a low recall. On the other hand, the neural conversational model can cater to a variety of requests, as it generates the responses word by word as opposed to using canned responses.

The hybrid model, by leveraging a crucial element of AI, Natural Language Processing (NLP) and combining a flow-based suitable algorithm system with a neural generative model makes the final system robust enough for a real-world application.
Natural Language Processing (NLP)

A hybrid chatbot at the heart of its framework has NLP, a part of AI and machine learning, which helps it to understand natural language. AI-powered chatbot decodes and processes human-understandable language within the context in which it is spoken. It understands the nuances of human conversation and realizes that commands or queries made by users do not need to be so specific.

Chatbots, infused with NLP, simulate human-like conversation and decode user intent to generate intelligent replies. Unlike generative models, where the predefined flow makes it difficult for chatbots to have open-ended conversations, AI chatbots can engage users on a wide range of topics.

NLP empowers chatbots to parse multiple user intents and minimizes the failures by relying on the following elements:

Decoding intent: NLP breaks down user inputs and comprehends the meaning of the words, positioning, conjugation, plurality, and many other factors that a human conversation can have.

Recognizing utterance: Chatbots with NLP are smart enough to recognize the instances of sentences that a user may give when they are referring to an intent.

Dealing with Entity: Chatbots can identify entities from a field of data or words related to time, location, description, a synonym for a word, a person, a number or anything that specifies an object.

Contextual understanding: NLP helps chatbots to decipher context by analyzing inputs like time, place, conversation history, tone, sentence structure, sentiment, etc. For instance, it is easy for the chatbot to be misled with the user reply “Great! I have to wait for another hour for my food to be delivered.” Hence, the chatbot must process positive, negative and neutral comments to analyze user sentiments effectively.

Digital interactions between brands and customers are becoming more and more complex by the day. Chatbot architectures showcase the intricacies that make conversational interfaces intelligent enough to be at par with these complex digital interactions.

As digital proliferation continues to increase, we will witness a rise in the usage of chatbots as well. Understanding what powers these chatbots will be critical in the coming years for businesses to truly unlock their potential.

Looking to harness conversational AI to automate business processes?