Microsoft is blocking prompts that make Copilot generate violent and sexual images

Microsoft is starting to block prompts that led to Copilot’s Designer AI image creator generating violent and sexually inappropriate photos. The news comes a week after an engineer working on AI raised concerns to the U.S. Federal Trade Commission about the generative AI-powered chatbot’s image-generation capabilities.

According to a recent report by CNBC, Shane Jones, an AI engineer at Microsoft tried prompts like ‘four twenty’, ‘pro choice’, ‘pro choce’ and ‘pro life’ , using which he was able to generate pictures of demons, monsters, teenagers with assault rifles, underage drinking, and drug use.

With Microsoft now blocking these prompts, Copilot now shows a warning message that says, “This prompt has been blocked. Our system automatically flagged this prompt because it may conflict with our content policy. More policy violations may lead to automatic suspension of your access. If you think this is a mistake, please report it to help us improve.”