Bots with personality — GenAI Powered Bot Conversations

Chatbots have been around for many years now. Used in augmented business operations, assistants, embedded in products, ChatOps etc., with a ubiquitous underlying goal to mimic a human conversation using natural language. Chatbots have evolved from being glorified decision trees, to include NLP and now ML chatbots that are powered by ML models which learn and train to get better over a period. And support voice a.k.a voice bots.

With Generative AI, the advanced, human like conversational ability of the LLMs (Large Language Models/Foundation Models) is being used to generate newer solutions in this space. Ones that enhance the chat experience and are easy to integrate with the ecosystem to products and solutions to massively accelerate innovation.

This post presents two patterns of conversations using a OpenAI’s gpt-3.5-turbo model. The basic premise of these patterns is:

1. Define a persona for a LLM session to assume, based on a simple JSON manifest that defines a persona.
2. Use the context or a ‘system’ role in a chat API to pass the composite persona prompt to the model, to tune the persona session.
3. Provide sufficient buffer of conversation history for the LLM to maintain the fine tuned conversation context.
4. Enable integration with services, based on the conversation context i.e. invoke other business services within the chat session itself, to enable an action for a user. This feature uses the function calling feature of GPT chat API.

You may come across similar concepts with different names such as ‘Assistants’ or ‘Agents’ in the GenAI world. The patterns described in this post make a specific persona and defines a schema to do that. In addition some of the capabilities mentioned here , may be available in LLM framework. For example, to provide a state to an otherwise stateless session with LLM model, LangChain provide memory buffer to maintain chat history.

The code demonstrated in this post is available in this repo.

https://github.com/kapilraina/GenAIPatterns/tree/master/conversations

Creating a bot persona

How can a bot personality be define? Although there is a great complexity to this question, but for the bot conversation, that knows itself, a some attributes must be define. This structure has been defined in schema file persona_schema.json.

Some Example of personas derived using this schema.

P1. Bot & Bot

To initiate conversation between two bot personas, such as the example provided above, the respective json files are created in a /personas directory. For P1, the personas are prefixed with 2_* (to indicate that two bots are involved!). The program bot_bot_conversation.py, takes the values from the two keys provided in the .env file :

persona1definitionfile=2_experiencedcsr.json
persona2definitionfile=2_angrycustomer.json

In this pattern, the two person files that are created have very little awareness about each other. Through the persona definitions, you can tune the awareness as much as needed.

Flow

P1

bot_bot_conversation.py loads the two personas and creates an eviction buffer of predefined length for conversation. This eviction buffer is used to create the messages payloads for a ‘system’ role in the GPT chat completion API.

Using a predefined and configurable session length, P1 plays a conversation session between the two assumed personas. The Persona1 seed message is prompted by a human user and then with a sufficiently high model temperature and the token limits, the conversations assume different semantics. The session is paused in between for a few seconds to prevent the rate limiting errors from the model for the provided OpenAI API Key.

The model is also provided a set of pre-defined functions. For example in case of 2_angrycustomer.json & 1_experiencedcsr.json, two functions are defined in businessfunctions.py. This module is imported in the program to provide the necessary business actions that can be invoked from the chat session itself.

def processRefund(refundRequest)
def processReturn(returnRequest)

Depending on the conversation that takes places between two bots, the model may suggest to invoke these functions with a generated parameters and bot_bot_conversation.py calls the functions.

For example :

persona1definitionfile=2_experiencedcsr.json
persona2definitionfile=2_angrycustomer.json

persona1definitionfile=2_mike.json
persona2definitionfile=2_dwight.json

Why use this :

• generate test conversation with actual bots
• test the business function invocations
• measure the targets for the chat assistant (for example running sentiment analysis during the length of conversation)
• aids in choosing right conversation LLM models

P2. Human & Bot

The schema to initiate this pattern has been slightly altered. It introduces two new top level elements — “bot” to define the bot persona and “human” to provide the basic name field for human actor to be used in chat.

For example

Flow

P2

bot_human_conversation.py, takes the persona file with env var: botdefinitionfile and prompts for human inputs to the conversation, which is responded to by the bot persona. The function settings is same as P1

Example

botdefinitionfile=1_angrycustomer.json

Why use this:

1. This is the pattern to test the bot with a persona.
2. Test the effectiveness of responses via metrics such as sentiment analysis.
3. Use human responses to generate training data for model
4. Use human responses to fine tune persona definition.

Few points:

1. The programs work with OpenAI API key and beyond the free tier, will be charged.
2. Tune the conversation length to limit number of iterations of the program to control quota usage.
3. Tune the token size to control token quota use.
4. Increase the sleep interval to prevent throttling issues.

Thanks !

P.S : If you decided to run the samples and face some issues, drop a comment and I will try my best to fix. This is all a learning and thinking loud phase for me !