Microsoft Introduces New Tools for Enhancing Security and Trust in Azure AI Studio

Microsoft is set to launch a variety of innovative tools for AI developers utilizing Azure. These tools are designed to assist in detecting prompt injection attacks, performing risk monitoring, and identifying hallucinations in output.

The availability of these tools for Azure AI Studio was announced by Microsoft. Developers who utilize Azure AI can leverage these tools to enhance their software. According to the company, there is a growing need for tools that can help in managing innovation and risk effectively, and these new tools for Azure AI Studio aim to address this demand.

Microsoft is introducing five distinct tools to address these needs. One of these tools, Prompt Shields, is specifically designed to detect and prevent jailbreaks and other forms of prompt injection attacks. Users often attempt to 'hack' AI systems by pushing the limits of these tools, such as through jailbreaks like DAN for ChatGPT or by testing how generative AI responds to specific inputs or by uploading documents. Prompt Shields analyzes common methods and phrases used for these attacks to make them easier to block.

In addition, Azure AI Studio will also feature detection of hallucinations in generative AI output. This is particularly crucial in fields where data confidentiality is paramount, as hallucinations in AI models can potentially leak sensitive data. 'Groundedness Detection' is implemented to prevent this from occurring. Furthermore, a Risk & Safety Monitoring tool will be available to provide administrators with a platform to monitor the impact of content filters easily.

Lastly, there will be two additional tools aimed at ensuring the safety of generative AI output. The Safety Messages System allows for safety conditions to be imposed on output, while Safety Evaluations is designed to identify vulnerabilities to jailbreak attacks and assess content risks within applications.

Not all tools are immediately accessible, as Prompt Shields, Safety Evaluations, and Risk & Safety Monitoring will be available in preview for Azure AI and Azure OpenAI, while Groundedness Detection and Safety Messages Systems are expected to launch in the future.