What to consider when planning and developing AI bots with Azure and M365

With the rise of AI and availability of products such as Azure OpenAI, Microsoft Copilot and Copilot Studio (previously Power Virtual Agents) it has become easier than ever for organizations to create their own AI bots.

These bots can be configured to help in specific business cases, or they can use proprietary data owned by the organization as a knowledge base. For example, an HR bot can provide information for company employees in a human-like manner.

More sophisticated bots can even serve as AI assistants, where the bot is capable of performing actions for the human interacting with it. As an example, an AI-assistant for social service workers could not only retrieve data, but also reserve appointments with customers and start phone calls.

But the question to ask is, what does it take to get there?

Azure OpenAI, Microsoft Copilot for M365, and other large language model AI offerings, generally provide the bot’s base intelligence, which needs to be utilized to implement your custom business case.

There are several high-level steps which need to be taken in the bot’s development project; some of these may be optional depending on the business case and AI technologies used, but all of them should be taken into consideration:

• Specify the bot’s business case clearly
• Design data integrations for the bot
• Implement data security
• Design and implement the bot user interface
• Test and validate the bot’s performance

Now, let’s take a closer look to each of these steps.

Specify the bot’s business case clearly

First you need to take a sharp look at why you are designing the bot: identify which personnel are expected to use the bot, for what purpose and what the return-of-investment for the bot is. Identify which external data sources will provide your bot with a sufficient knowledge base to fulfill its purpose.

You may create initial prompts to provide examples of what kind of questions the bot is expected to answer to. Determine whether the bot is expected to only provide information for the end users, or whether it should be able to perform actions as well. Decide whether the bot should serve only specific scenarios, or whether you would like to provide a more universal bot experience.

Design data integrations for the bot

If your bot is expected to use external data, this data needs to be brought to the bot using various integration mechanisms. They vary depending on the AI technology you have chosen. If you wish to allow users to add or remove documents or data sources from the bot’s knowledge base, you may need to implement an additional application for this purpose.

Out of Microsoft’s AI offerings, Azure OpenAI has the best capabilities for supporting complex data integrations and formats and combining multiple data sources into one knowledge base. Extending Copilot with external data, on the other hand, allows you to enrich the already extensive knowledge base it has in Microsoft 365. Copilot Studio provides several connectors out of the box which make it easy to create simple bots, but its integration capabilities are limited compared to the other two.

Implement data security

By default, with any AI bots, all external data available to the bot is also available for anyone who uses the bot. If you wish to use sensitive data with your bot, it is important to design and implement security trimming for the data to keep it safe.

Also implement proper security during integration processes to ensure that your data is not leaked, and that the bot’s knowledge base is not diluted with erroneous data.

Azure OpenAI and Microsoft Copilot both support security trimming in their data integration processes, but the technical details differ significantly. With Azure OpenAI, security trimming needs to be implemented in the application layer where the bot application needs to determine the current user’s access rights, and then determine the data sets they are allowed to access. When using Copilot extensions, security trimming is implemented by setting access rights on the data itself, and Copilot uses the current user’s Entra ID account and group memberships to see which data they can use.

Design and implement the bot user interface

Several AI offerings will come with built-in user interfaces, while with others you are expected to create them yourself. Consider where you wish to publish the bot (internally or externally), who should be able to access it and whether there are any other features you would like to include in the application.

Azure OpenAI by itself only provides the background AI service. Microsoft has created a free open-source application which can be used for prototyping bot functionality, but with Azure OpenAI you should always create a purpose-built user interface. Copilot for M365, and Copilot Studio, on the other hand, come with pre-built user interfaces. As such these can be faster to take to production, but their user interface capabilities are also more limited.

Test and validate the bot’s performance

Iterate over its prompts, and its use of external data, until the results are suitable to provide the desired ROI. With AI bots even very small changes in how their commands are worded can cause significantly different results.

Because of this, it is important to test them thoroughly and to not be discouraged if the very first results seem unimpressive. Perform red team testing to ensure that any sensitive data used by the bot is secure.

It is true that the intelligence of large language model is available for everyone with just a few clicks of a mouse. But as you can see, employing it for your special business cases requires careful thought and work. We are more than happy to help your organization with that.

How can we help you?

If you are interested in hearing more about the possibilities of bots, Azure Open AI, or Copilot Studio, please contact us and let’s arrange a meeting with our expert

Contact Sulava!

Read more about Copilot Studio from this blog: Power Virtual Agents and Microsoft Copilot Studio: A Match Made in AI Heaven

See an example on Copilot Studio possibilities, get to know Sulava intrabot from this blog!

Do you want to know more about the topics mentioned in this blog? Come to our courses! For example:

Copilot Studio Basics

This training provides you with a unique opportunity to learn how to harness the potential of Copilot Studio. The training includes practical exercises where you can apply what you’ve learned.

Microsoft Azure AI Fundamentals

In this course you will explore the uses of artificial intelligence through illustrative examples. Come and discover what AI is and how it can be leveraged in business. This foundational course opens your eyes to opportunities in various areas!

Develop Generative AI Solutions with Azure OpenAI Service

How internal solutions are built on top of OpenAI’s models? How can modern AI be integrated into organisation’s systems and applications? This course focuses specifically on these topics!

Data Protection & Copilot

How do you prevent mishaps and ensure that the right information is used for the right purpose when Copilot is in use? Additionally, how do preparatory measures enhance the Copilot user experience?