I'm going to use the Fallback topic so that I can use Power Automate to query the Azure OpenAI ChatGPT model (note: you can also make HTTP requests directly in Bot Framework Composer) whenever PVA doesn't find a matching topic for a user utterance.
It's good to understand Chat Markup Language to understand how to scope the conversation and provide instructions to ChatGPT prior to it returning an answer. That way, you can give personality traits to the bot, rules to follow when providing answers, and data or information needed for the model.
Here's an example:
<|im_start|>system
Assistant is an intelligent chatbot designed to help users answer their tax related questions.
<|im_end|>
<|im_start|>user
When do I need to file my taxes by?
<|im_end|>
<|im_start|>assistant
In 2023, you will need to file your taxes by April 18th. The date falls after the usual April 15th deadline because April 15th falls on a Saturday in 2023. For more details, see https://www.irs.gov/filing/individuals/when-to-file
<|im_end|>
<|im_start|>user
How can I check the status of my tax refund?
<|im_end|>
<|im_start|>assistant
You can check the status of your tax refund by visiting https://www.irs.gov/refunds
<|im_end|>
For our example, I'll use a very basic system message that you can customize
<|im_start|>system
Assistant helps Contoso employees with their questions.
<|im_end|>
The idea for our PVA ChatGPT bot is to keep the full conversation when sending new prompts to the ChatGPT API.
To manage a global variable keeping track of the conversation with Azure OpenAI ChatGPT, I'm going to use Bot Framework Composer to do 2 things:
- Create an InitializeFullDialog dialog, that will create and set a new variable, FullDialog, in Power Virtual Agents, that will initially contain the system message.
- Create a SetFullDialog dialog, that will update the FullDialog variable so that it contains the history of interactions (past questions and answers provided by Azure OpenAI ChatGPT).
I start by opening the Bot Framework Composer
If you need help setting things up, refer to this: Getting started with Bot Framework Composer - Power Virtual Agents | Microsoft Learn
I then add a new dialog to my chatbot
I call it "InitializeFullDialog"
I create one new Output for it, that I call FullDialog, of type string:
In BeginDialog, I add a Manage properties > Set a property node, in order set the FullDialog variable to my system message as well as the necessary chat markup language before I add the user utterence:
Property: dialog.result.FullDialog
Value: ='<|im_start|>system\nAssistant helps Contoso employees with their questions.\n<|im_end|>\n<|im_start|>user\n'
I then add another dialog, SetFullDialog, this time with no outputs.
In the BeginDialog, I add a Manage properties > Set a property node, in order set the Power Virtual Agents FullDialog variable as a concatenation of the previous FullDialog value as well as new values that Power Automate pass me back (I'm showing this later in this article, but they're OK to set up in advance) and some chat markup language to piece it together:
Property: virtualagent.ConversationFullDialog
Value: =concat(virtualagent.FullDialog,virtualagent.OriginalTriggerPhrase,'\n<|im_end|>\n<|im_start|>assistant',virtualagent.ChatGPTResponse,'\n<|im_end|>\n<|im_start|>user\n')
Then I publish my bot in Bot Framework Composer:
As I interact in this example with the Azure OpenAI ChatGPT API, I'm using the Power Virtual Agents Fallback topic. This means that whenever a user asks a question that PVA doesn't know how to answer with an existing topic (i.e. there is no match with the topics trigger phrases), it goes to the Fallback topic, and in the Fallback topic I'm calling the Azure OpenAI ChatGPT API.
If you haven't enabled the Fallback topic in your chatbot, learn how here: Use a system fallback topic - Power Virtual Agents | Microsoft Learn
In the Fallback topic, I start by removing the existing nodes as I'm changing its behavior.
Just after the 'Trigger Phrases' node, I add a "Redirect to another topic" node, and select the InitializeFullDialog dialog.
I click on the variable name, name it FullDialog, and change its usage to "Bot (any topic can access)" and also check "External sources can set values":
Next step is to call the Azure OpenAI ChatGPT API using Power Automate.
I add a new node, select Call an action > Create a flow.
I assume you already have an Azure OpenAI account and an API key, but if you don't, check these:
In your cloud flow, you'll first want to add 2 text inputs, UnrecognizedTriggerPhrase and ConversationFullDialog:
Then, to call the Azure OpenAI GPT API, I use the HTTP action, with this configuration
Method: POST
URI: you can get the URI from the ChatGPT playground in the Azure OpenAI Studio, in view code, in curl
Headers:
api-key: {YourAPIKey}
Content-Type: application/jsonBody:
{
"prompt": "{ExpressionBelow}",
"temperature": 0.7,
"top_p": 0.95,
"frequency_penalty": 0,
"presence_penalty": 0,
"max_tokens": 800,
"stop": [
"<|im_end|>"
]
}For the prompt, this is where I need to pass the user question.
To keep context, the trick is to provide past questions and answers, as well as the new question.
So, I append the FullConversation with the UnrecognizedTriggerPhrase and a bit of chat language markup to let it know I expect a response from the bot.:
concat(
triggerBody()['text_1'],
triggerBody()['text'],
'\n<|im_end|>\n<|im_start|>assistant'
)
Now I need to pass back a few things to PVA.
- The ChatGPTResponse is the response from the ChatGPT API.
This is the expression I'm using:
body('HTTP:_Azure_OpenAI_ChatGPT')?['choices'][0]?['text'] - The OriginalTriggerPhrase - that's the individual question that was raised in this flow, that I also need to append to the ConversationFullDialog variable. I use the UnrecognizedTriggerPhrase value in dynamic content.
I give my flow a name, and save it.
Back in PVA, still in my Fallback topic, I select my flow, and can map the UnrecognizedTriggerPhrase and FullDialog to the Power Automate inputs.
I can see that Power Automate has 2 ouputs. I set their Usage to "Bot (any topic can access)":
I now add a new Show a message node, where I select the bot.ChatGPTResponse variable:
The final step is to add a "Redirect to another topic" node, and select the SetFullDialog dialog, that will take care of updating the FullDialog variable with the past questions and answers.
And that's it! 🎉
