Azure OpenAI: Automate build Power Virtual Agent in Power Apps

Hesam Seyed Mousavi, January 30, 2024

We have built Power Virtual Agent in the past calling the public OpenAI API. If you work in a large organization, your security team probably won’t like that because you are sending private company data to a public endpoint.
By: Leo Wang

Azure OpenAI Resource

Before we can use the API in Power Automate, let’s create our own OpenAI resource first.

1. Go to Azure Portal – Azure OpenAI, Select Create Azure OpenAI.

• Select your subscription
• Choose a resource group
• Only use “East US” or “South Central US” Region

2. Keep the rest options default and create the resource group and resource. I selected All Networks but you should speak to the network security to get it configured properly.

3. Once completed, go to the OpenAI resource. Select Go to Azure OpenAI Studio

4. Create new deployment your first model. I have chosen a gpt3.5 turbo model.

5. Once the model is created, open it in Playground. Select View Code.

6. Here you can see the endpoint and API key in the bottom.

7. In Power Automate, you need to provide the following information:

• Endpoint URL into the URI section
• API Key in the header
• User Message
• System Message (role) is optional

Here is an example in the flow just to test your new Azure OpenAI endpoint respond to the API call.

8. For your enterprise Prod deployment, you should use an environment variable to store the URI and API in a solution.

The Secret to Memorize the Conversation

When it comes to making the PVA bot remember the conversation history like ChatGPT, it’s actually quite interesting. We’ve covered this topic in our previous blogs when we were still exploring the early days of OpenAI.

Let’s refresh our memory and dive into it again. The main idea is to give the bot a short-term memory by sending the previous questions and answers as history, along with the new question, to the GPT model.

Why short-term memory? Well, as the conversation progresses, the prompt would get longer and longer, eventually hitting the token limit of the model. The model will start loosing context as the earlier conversation gets deleted.

Here’s how we can achieve this:

1. Create a global variable in PVA to holds the conversation history. Let’s call it conversation variable (CV).
2. Initialise it with empty string. So every time user restarts the chat session, the conversation will be cleared.
3. Capture user’s first question in PVA, append to CV.
4. Pass the CV to Power Automate Flow to call OpenAI API
5. Once the response is received in PVA, append to CV again.
6. Now the CV has the first question and first response.
7. As user ask second question, loop back to the step to append user input to CV.
8. Pass the CV to flow and repeat the steps, you get the idea.

So essentially you are always using this global variable that holds the conversation history to call OpenAI API. You are sending the whole context plus the new question to the model.

Keeping the global variable inside the Power Virtual Agents (PVA) instead of initializing it within the flow ensures that the conversation history is preserved. In PVA, each flow call is treated as a new session, causing variables to be reset. By keeping the variable within the PVA, it retains its value as long as the chat session is active.

Start with the Solution

Since we are going to use the environment variable, we need to build everything inside the same solution.

Power Platform Environment Variable

1. Go to Power Apps Solutions, create a new Solution or open an existing one.
2. In the Solution, click New – More – Environment Variable
3. Select the JSON format and paste your URL and Key.

You can call them whatever name you want.

{ 
 "endpoint": "https://your-openai-endpoint.com",
  "apiKey": "your-api-key"
}

To use in the flow, use the following expression to reference the correct value.

parameters('EnvVarName')['endpoint']
parameters('EnvVarName')['apikey']

Build a flow to call Azure OpenAI

Let’s build a simple flow that triggers by PVA and returns the response.

When you return the value to PVA, remember the difference between OpenAI and Azure OpenAI. The returned value is under ‘content’ instead of ‘text’.

Use body('HTTP')?['choices'][0]?['message']?['content']

Azure OpenAI Response example:
    "body": {
        "id": "chatcmpl-7NtxouUkk2V4aQ8q5ld0I5mRrOWoJ",
        "object": "chat.completion",
        "created": 1685931024,
        "model": "gpt-35-turbo",
        "choices": [
            {
                "index": 0,
                "finish_reason": "stop",
                "message": {
                    "role": "assistant",
                    "content": ", how can I assist you?"
                }
            }
        ]
    }

Create the PVA bot in the Solution

1. Create the bot from Power Apps Solutions.

2. Once the PVA is loaded, create a bot as usual, open Topics – System – ck, delete all the conditions.

3. Let’s start with the global variable. First we will check whether this variable is empty or not.

If it is not empty, that means we have existing chat history.

Otherwise, we will initialise it with an empty string.

4. Next we will take the user’s input and append to the variable. Since user’s input hasn’t triggered any particular topic and landed in the fallback, you can use Syste.Activity.Text. We also added a line to remove unescaped double quotes characters.

Global.VarConversation & " " & Substitute(System.Activity.Text,"""","\""")

5. Next we add the action to call the Power Automate flow, passing the Global.VarConversation variable.

6. Once we received the response, we pass it back to user.

7. And we also initialise the variable again by adding the response.

Global.VarConversation & " " &Topic.Response

8. The next question that triggers the fallback topic, it would check the variable again. If it has conversation history, it won’t get emptied.

Build a Power App with the PVA Bot with Chatbot Control

Please note this is still a preview feature. Follow this Microsoft Learn link to apply early access and configure.

Enable the Chatbot Control

1. First, publish your bot, go to Settings – Bot Details – Advanced and note down the Schema Name
2. Go to the Solutions and add a Canvas App
3. On the command bar, select Settings > Upcoming features.
4. From the Preview tab, set the toggle for Chatbot component to On.

Add to your canvas app

1. Create a canvas app
2. Select Insert – Chatbot

3. Pick the Chatbot you want, it should automaticallly populate the schema. Make sure the bot is already published in PVA.

4. Now you can test it out in the Player and publish it to use it on a mobile. Cool?

5. Once you are happy with the app, publish it and also add it to solution.